openjpeg/src/lib/openjp2/t2.c, branch floating_point_irreversible_encodingFork of openjpeg library for JPEG2000 encoding and decodinghttps://git.carlh.net/cgit/openjpeg/atom?h=floating_point_irreversible_encoding2020-04-21T13:55:44ZAdd support for generation of PLT markers in encoder2020-04-21T13:55:44ZEven Rouaulteven.rouault@spatialys.com2020-04-21T13:55:44Zurn:sha1:4edb8c83374f52cd6a8f2c7c875e8ffacccb5fa5
* -PLT switch added to opj_compress
* Add a opj_encoder_set_extra_options() function that
accepts a PLT=YES option, and could be expanded later
for other uses.
-------
Testing with a Sentinel2 10m band, T36JTT_20160914T074612_B02.jp2,
coming from S2A_MSIL1C_20160914T074612_N0204_R135_T36JTT_20160914T081456.SAFE
Decompress it to TIFF:
```
opj_uncompress -i T36JTT_20160914T074612_B02.jp2 -o T36JTT_20160914T074612_B02.tif
```
Recompress it with similar parameters as original:
```
opj_compress -n 5 -c [256,256],[256,256],[256,256],[256,256],[256,256] -t 1024,1024 -PLT -i T36JTT_20160914T074612_B02.tif -o T36JTT_20160914T074612_B02_PLT.jp2
```
Dump codestream detail with GDAL dump_jp2.py utility (https://github.com/OSGeo/gdal/blob/master/gdal/swig/python/samples/dump_jp2.py)
```
python dump_jp2.py T36JTT_20160914T074612_B02.jp2 > /tmp/dump_sentinel2_ori.txt
python dump_jp2.py T36JTT_20160914T074612_B02_PLT.jp2 > /tmp/dump_sentinel2_openjpeg_plt.txt
```
The diff between both show very similar structure, and identical number of packets in PLT markers
Now testing with Kakadu (KDU803_Demo_Apps_for_Linux-x86-64_200210)
Full file decompression:
```
kdu_expand -i T36JTT_20160914T074612_B02_PLT.jp2 -o tmp.tif
Consumed 121 tile-part(s) from a total of 121 tile(s).
Consumed 80,318,806 codestream bytes (excluding any file format) = 5.329697
bits/pel.
Processed using the multi-threaded environment, with
8 parallel threads of execution
```
Partial decompresson (presumably using PLT markers):
```
kdu_expand -i T36JTT_20160914T074612_B02.jp2 -o tmp.pgm -region "{0.5,0.5},{0.01,0.01}"
kdu_expand -i T36JTT_20160914T074612_B02_PLT.jp2 -o tmp2.pgm -region "{0.5,0.5},{0.01,0.01}"
diff tmp.pgm tmp2.pgm && echo "same !"
```
-------
Funded by ESA for S2-MPC project
compression: emit POC marker when only one single POC is requested (fixes #1191)2019-04-25T12:40:55ZEven Rouaulteven.rouault@spatialys.com2019-04-24T22:17:13Zurn:sha1:1e3a57563defb6aa7cf24ffd2394d4a820e13bdaopj_t2_encode_packet(): disable setting empty packet header bit to 1 when there is an empty packet2018-01-08T08:38:44ZEven Rouaulteven.rouault@spatialys.com2018-01-08T08:38:44Zurn:sha1:07d526e4cb93b6d2e1a52cf49e3d011d3d4caa11
This effectively reverts commit 2609fb8077125b5b31f1bcc2f98c12ff1e6572d7
since it has been reported that
such packets cause decoding issues with cinema J2K hardware
decoders: https://groups.google.com/forum/#!topic/openjpeg/M7M_fLX_Bco
Sub-tile decoding: only decode precincts and codeblocks that intersect the window specified in opj_set_decode_area()2017-08-17T17:05:54ZEven Rouaulteven.rouault@spatialys.com2017-08-14T11:23:57Zurn:sha1:fe338a057c39797bf61939471ebaef09e44464c7opj_t2_encode_packet(): fix potential write heap buffer overflow (#992)2017-08-16T15:20:29ZEven Rouaulteven.rouault@spatialys.com2017-08-16T15:20:29Zurn:sha1:c535531f03369623b9b833ef41952c62257b507eopj_t2_encode_packet(): only emit an error about insufficiently large output buffer in FINAL_PASS mode. Fixes (master-only) regression added in 0b4fef6d1901254e41ab74ed681daba477d724c32017-08-16T10:52:33ZEven Rouaulteven.rouault@spatialys.com2017-08-16T10:52:33Zurn:sha1:9624b2fa4722edf77fbac631e7dacaae12c91672Fix argument order in error message of previous commit2017-08-10T14:58:36ZEven Rouaulteven.rouault@spatialys.com2017-08-10T14:58:36Zurn:sha1:a35b4891340dcecf2b248c7e57a90bf7d7aa9b2fPropagate event manager down to opj_t2_encode_packet() and use it to emit an error message when the output buffer is too small2017-08-10T14:49:47ZEven Rouaulteven.rouault@spatialys.com2017-08-10T14:49:47Zurn:sha1:0b4fef6d1901254e41ab74ed681daba477d724c3Slight improvement in management of code block chunks2017-08-07T16:32:52ZEven Rouaulteven.rouault@spatialys.com2017-07-06T17:34:21Zurn:sha1:92114694a48638e86c07e4adf11b78878045c8a9
Instead of having the chunk array at the segment level, we can move it down to
the codeblock itself since segments are filled in sequential order.
Limit the number of memory allocation, and decrease slightly the memory usage.
On MAPA_005.jp2
n4: 1871312549 (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
n1: 1610689344 0x4E781E7: opj_aligned_malloc (opj_malloc.c:61)
n1: 1610689344 0x4E71D1B: opj_alloc_tile_component_data (tcd.c:676)
n1: 1610689344 0x4E726CF: opj_tcd_init_decode_tile (tcd.c:816)
n1: 1610689344 0x4E4BE39: opj_j2k_read_tile_header (j2k.c:8617)
n1: 1610689344 0x4E4C902: opj_j2k_decode_tiles (j2k.c:10348)
n1: 1610689344 0x4E4E3CE: opj_j2k_decode (j2k.c:7846)
n1: 1610689344 0x4E53002: opj_jp2_decode (jp2.c:1564)
n0: 1610689344 0x40374E: main (opj_decompress.c:1459)
n1: 219232541 0x4E4BC50: opj_j2k_read_tile_header (j2k.c:4683)
n1: 219232541 0x4E4C902: opj_j2k_decode_tiles (j2k.c:10348)
n1: 219232541 0x4E4E3CE: opj_j2k_decode (j2k.c:7846)
n1: 219232541 0x4E53002: opj_jp2_decode (jp2.c:1564)
n0: 219232541 0x40374E: main (opj_decompress.c:1459)
n1: 23893200 0x4E72735: opj_tcd_init_decode_tile (tcd.c:1225)
n1: 23893200 0x4E4BE39: opj_j2k_read_tile_header (j2k.c:8617)
n1: 23893200 0x4E4C902: opj_j2k_decode_tiles (j2k.c:10348)
n1: 23893200 0x4E4E3CE: opj_j2k_decode (j2k.c:7846)
n1: 23893200 0x4E53002: opj_jp2_decode (jp2.c:1564)
n0: 23893200 0x40374E: main (opj_decompress.c:1459)
n0: 17497464 in 52 places, all below massif's threshold (1.00%)
Decoding: do not allocate memory for the codestream of each codeblock2017-08-07T16:32:52ZEven Rouaulteven.rouault@spatialys.com2017-07-06T14:11:11Zurn:sha1:ca34d13e76a588a00171e57690c1deeaf068723a
Currently we allocate at least 8192 bytes for each codeblock, and copy
the relevant parts of the codestream in that per-codeblock buffer as we
decode packets.
As the whole codestream for the tile is ingested in memory and alive
during the decoding, we can directly point to it instead of copying. But
to do that, we need an intermediate concept, a 'chunk' of code-stream segment,
given that segments may be made of data at different places in the code-stream
when quality layers are used.
With that change, the decoding of MAPA_005.jp2 goes down from the previous
improvement of 2.7 GB down to 1.9 GB.
New profile:
n4: 1885648469 (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
n1: 1610689344 0x4E78287: opj_aligned_malloc (opj_malloc.c:61)
n1: 1610689344 0x4E71D7B: opj_alloc_tile_component_data (tcd.c:676)
n1: 1610689344 0x4E7272C: opj_tcd_init_decode_tile (tcd.c:816)
n1: 1610689344 0x4E4BDD9: opj_j2k_read_tile_header (j2k.c:8618)
n1: 1610689344 0x4E4C8A2: opj_j2k_decode_tiles (j2k.c:10349)
n1: 1610689344 0x4E4E36E: opj_j2k_decode (j2k.c:7847)
n1: 1610689344 0x4E52FA2: opj_jp2_decode (jp2.c:1564)
n0: 1610689344 0x40374E: main (opj_decompress.c:1459)
n1: 219232541 0x4E4BBF0: opj_j2k_read_tile_header (j2k.c:4685)
n1: 219232541 0x4E4C8A2: opj_j2k_decode_tiles (j2k.c:10349)
n1: 219232541 0x4E4E36E: opj_j2k_decode (j2k.c:7847)
n1: 219232541 0x4E52FA2: opj_jp2_decode (jp2.c:1564)
n0: 219232541 0x40374E: main (opj_decompress.c:1459)
n1: 39822000 0x4E727A9: opj_tcd_init_decode_tile (tcd.c:1219)
n1: 39822000 0x4E4BDD9: opj_j2k_read_tile_header (j2k.c:8618)
n1: 39822000 0x4E4C8A2: opj_j2k_decode_tiles (j2k.c:10349)
n1: 39822000 0x4E4E36E: opj_j2k_decode (j2k.c:7847)
n1: 39822000 0x4E52FA2: opj_jp2_decode (jp2.c:1564)
n0: 39822000 0x40374E: main (opj_decompress.c:1459)
n0: 15904584 in 52 places, all below massif's threshold (1.00%)