Fork of openjpeg library for JPEG2000 encoding and decoding https://git.carlh.net/cgit/openjpeg/atom?h=floating_point_irreversible_encoding 2020-05-20T18:31:28Z 2020-05-20T18:31:28Z Even Rouault even.rouault@spatialys.com 2020-05-20T17:24:09Z urn:sha1:adccbc8336ce5c46ca7de85ac364d9c992d337c7 2020-05-20T18:31:28Z Even Rouault even.rouault@spatialys.com 2020-05-20T16:00:45Z urn:sha1:3cd1305596f191a01afdc11f9355f9c6590065dd The previous constant opj_c13318 was mysteriously equal to 2/K , and in the DWT, we had to divide K and opj_c13318 by 2... The issue was that the band->stepsize computation in tcd.c didn't take into account the log2gain of the band. The effect of this change is expected to be mostly equivalent to the previous situation, except some difference in rounding. But it leads to a dramatic reduction of the mean square error and peak error in the irreversible encoding of issue141.tif ! 2020-05-20T18:31:28Z Even Rouault even.rouault@spatialys.com 2020-05-20T11:14:16Z urn:sha1:f38c069547f1c41dc94ec4a273efb07997685c21 2020-05-20T18:31:28Z Even Rouault even.rouault@spatialys.com 2020-05-18T18:17:07Z urn:sha1:3d35d0f3af46ee206a3ea147298aad3d83a7775c 2020-05-20T18:31:28Z Even Rouault even.rouault@spatialys.com 2020-05-18T18:15:07Z urn:sha1:00cff6f5c02deabb64ccb15f15c13fcb773968fe 2020-05-20T18:30:21Z Even Rouault even.rouault@spatialys.com 2020-04-30T09:52:42Z urn:sha1:07d1f775a1ef95496b0c78b18f671dac41983320 Update the bench_dwt utility to have a -decode/-encode switch Measured performance gains for DWT encoder on a Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz (4 cores, hyper threaded) Encoding time: $ ./bin/bench_dwt -encode -num_threads 1 time for dwt_encode: total = 8.348 s, wallclock = 8.352 s $ ./bin/bench_dwt -encode -num_threads 2 time for dwt_encode: total = 9.776 s, wallclock = 4.904 s $ ./bin/bench_dwt -encode -num_threads 4 time for dwt_encode: total = 13.188 s, wallclock = 3.310 s $ ./bin/bench_dwt -encode -num_threads 8 time for dwt_encode: total = 30.024 s, wallclock = 4.064 s Scaling is probably limited by memory access patterns causing memory access to be the bottleneck. The slightly worse results with threads==8 than with thread==4 is due to hyperthreading being not appropriate here. 2020-05-20T18:30:21Z Even Rouault even.rouault@spatialys.com 2020-04-29T09:50:17Z urn:sha1:97eb7e0bf17b476d516262e0af462ec7eeb8f505 - API wise, opj_codec_set_threads() can be used on the encoding side - opj_compress has a -threads switch similar to opj_uncompress 2020-04-21T13:55:44Z Even Rouault even.rouault@spatialys.com 2020-04-21T13:55:44Z urn:sha1:4edb8c83374f52cd6a8f2c7c875e8ffacccb5fa5 * -PLT switch added to opj_compress * Add a opj_encoder_set_extra_options() function that accepts a PLT=YES option, and could be expanded later for other uses. ------- Testing with a Sentinel2 10m band, T36JTT_20160914T074612_B02.jp2, coming from S2A_MSIL1C_20160914T074612_N0204_R135_T36JTT_20160914T081456.SAFE Decompress it to TIFF: ``` opj_uncompress -i T36JTT_20160914T074612_B02.jp2 -o T36JTT_20160914T074612_B02.tif ``` Recompress it with similar parameters as original: ``` opj_compress -n 5 -c [256,256],[256,256],[256,256],[256,256],[256,256] -t 1024,1024 -PLT -i T36JTT_20160914T074612_B02.tif -o T36JTT_20160914T074612_B02_PLT.jp2 ``` Dump codestream detail with GDAL dump_jp2.py utility (https://github.com/OSGeo/gdal/blob/master/gdal/swig/python/samples/dump_jp2.py) ``` python dump_jp2.py T36JTT_20160914T074612_B02.jp2 > /tmp/dump_sentinel2_ori.txt python dump_jp2.py T36JTT_20160914T074612_B02_PLT.jp2 > /tmp/dump_sentinel2_openjpeg_plt.txt ``` The diff between both show very similar structure, and identical number of packets in PLT markers Now testing with Kakadu (KDU803_Demo_Apps_for_Linux-x86-64_200210) Full file decompression: ``` kdu_expand -i T36JTT_20160914T074612_B02_PLT.jp2 -o tmp.tif Consumed 121 tile-part(s) from a total of 121 tile(s). Consumed 80,318,806 codestream bytes (excluding any file format) = 5.329697 bits/pel. Processed using the multi-threaded environment, with 8 parallel threads of execution ``` Partial decompresson (presumably using PLT markers): ``` kdu_expand -i T36JTT_20160914T074612_B02.jp2 -o tmp.pgm -region "{0.5,0.5},{0.01,0.01}" kdu_expand -i T36JTT_20160914T074612_B02_PLT.jp2 -o tmp2.pgm -region "{0.5,0.5},{0.01,0.01}" diff tmp.pgm tmp2.pgm && echo "same !" ``` ------- Funded by ESA for S2-MPC project 2020-04-16T18:33:22Z Even Rouault even.rouault@spatialys.com 2020-04-16T18:33:22Z urn:sha1:221a801a97a3ea968a311f7905c18a1eb7f034c4 2020-02-12T14:55:25Z Even Rouault even.rouault@spatialys.com 2020-02-12T14:55:16Z urn:sha1:84f3bebbff515f2b00ccf0c817930ebb10b91760 Add -IMF switch to opj_compress as well