Fork of openjpeg library for JPEG2000 encoding and decoding https://git.carlh.net/cgit/openjpeg/atom?h=v2.2.0 2017-08-07T16:32:52Z 2017-08-07T16:32:52Z Even Rouault even.rouault@spatialys.com 2017-07-06T17:34:21Z urn:sha1:92114694a48638e86c07e4adf11b78878045c8a9 Instead of having the chunk array at the segment level, we can move it down to the codeblock itself since segments are filled in sequential order. Limit the number of memory allocation, and decrease slightly the memory usage. On MAPA_005.jp2 n4: 1871312549 (heap allocation functions) malloc/new/new[], --alloc-fns, etc. n1: 1610689344 0x4E781E7: opj_aligned_malloc (opj_malloc.c:61) n1: 1610689344 0x4E71D1B: opj_alloc_tile_component_data (tcd.c:676) n1: 1610689344 0x4E726CF: opj_tcd_init_decode_tile (tcd.c:816) n1: 1610689344 0x4E4BE39: opj_j2k_read_tile_header (j2k.c:8617) n1: 1610689344 0x4E4C902: opj_j2k_decode_tiles (j2k.c:10348) n1: 1610689344 0x4E4E3CE: opj_j2k_decode (j2k.c:7846) n1: 1610689344 0x4E53002: opj_jp2_decode (jp2.c:1564) n0: 1610689344 0x40374E: main (opj_decompress.c:1459) n1: 219232541 0x4E4BC50: opj_j2k_read_tile_header (j2k.c:4683) n1: 219232541 0x4E4C902: opj_j2k_decode_tiles (j2k.c:10348) n1: 219232541 0x4E4E3CE: opj_j2k_decode (j2k.c:7846) n1: 219232541 0x4E53002: opj_jp2_decode (jp2.c:1564) n0: 219232541 0x40374E: main (opj_decompress.c:1459) n1: 23893200 0x4E72735: opj_tcd_init_decode_tile (tcd.c:1225) n1: 23893200 0x4E4BE39: opj_j2k_read_tile_header (j2k.c:8617) n1: 23893200 0x4E4C902: opj_j2k_decode_tiles (j2k.c:10348) n1: 23893200 0x4E4E3CE: opj_j2k_decode (j2k.c:7846) n1: 23893200 0x4E53002: opj_jp2_decode (jp2.c:1564) n0: 23893200 0x40374E: main (opj_decompress.c:1459) n0: 17497464 in 52 places, all below massif's threshold (1.00%) 2017-08-07T16:32:52Z Even Rouault even.rouault@spatialys.com 2017-07-06T14:11:11Z urn:sha1:ca34d13e76a588a00171e57690c1deeaf068723a Currently we allocate at least 8192 bytes for each codeblock, and copy the relevant parts of the codestream in that per-codeblock buffer as we decode packets. As the whole codestream for the tile is ingested in memory and alive during the decoding, we can directly point to it instead of copying. But to do that, we need an intermediate concept, a 'chunk' of code-stream segment, given that segments may be made of data at different places in the code-stream when quality layers are used. With that change, the decoding of MAPA_005.jp2 goes down from the previous improvement of 2.7 GB down to 1.9 GB. New profile: n4: 1885648469 (heap allocation functions) malloc/new/new[], --alloc-fns, etc. n1: 1610689344 0x4E78287: opj_aligned_malloc (opj_malloc.c:61) n1: 1610689344 0x4E71D7B: opj_alloc_tile_component_data (tcd.c:676) n1: 1610689344 0x4E7272C: opj_tcd_init_decode_tile (tcd.c:816) n1: 1610689344 0x4E4BDD9: opj_j2k_read_tile_header (j2k.c:8618) n1: 1610689344 0x4E4C8A2: opj_j2k_decode_tiles (j2k.c:10349) n1: 1610689344 0x4E4E36E: opj_j2k_decode (j2k.c:7847) n1: 1610689344 0x4E52FA2: opj_jp2_decode (jp2.c:1564) n0: 1610689344 0x40374E: main (opj_decompress.c:1459) n1: 219232541 0x4E4BBF0: opj_j2k_read_tile_header (j2k.c:4685) n1: 219232541 0x4E4C8A2: opj_j2k_decode_tiles (j2k.c:10349) n1: 219232541 0x4E4E36E: opj_j2k_decode (j2k.c:7847) n1: 219232541 0x4E52FA2: opj_jp2_decode (jp2.c:1564) n0: 219232541 0x40374E: main (opj_decompress.c:1459) n1: 39822000 0x4E727A9: opj_tcd_init_decode_tile (tcd.c:1219) n1: 39822000 0x4E4BDD9: opj_j2k_read_tile_header (j2k.c:8618) n1: 39822000 0x4E4C8A2: opj_j2k_decode_tiles (j2k.c:10349) n1: 39822000 0x4E4E36E: opj_j2k_decode (j2k.c:7847) n1: 39822000 0x4E52FA2: opj_jp2_decode (jp2.c:1564) n0: 39822000 0x40374E: main (opj_decompress.c:1459) n0: 15904584 in 52 places, all below massif's threshold (1.00%) 2017-08-07T16:32:52Z Even Rouault even.rouault@spatialys.com 2017-07-06T11:23:29Z urn:sha1:373520db309430b68dd9ff09cba03a25f711a88e 2017-07-30T16:18:59Z Even Rouault even.rouault@spatialys.com 2017-07-30T16:18:59Z urn:sha1:83342f2aafcab4599b49f780e35fd249e8402b61 2017-07-26T20:22:44Z Even Rouault even.rouault@spatialys.com 2017-07-26T20:22:44Z urn:sha1:6c5fe9407b7768eb2ac55b83511bc103551d207a Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2485 Credit to OSS Fuzz 2017-06-13T10:09:52Z Even Rouault even.rouault@spatialys.com 2017-06-13T10:09:52Z urn:sha1:9cbc9903c3034d1a101d0521402503ead0abcff0 2017-06-12T16:38:11Z Even Rouault even.rouault@spatialys.com 2017-06-12T16:15:23Z urn:sha1:2609fb8077125b5b31f1bcc2f98c12ff1e6572d7 2017-06-12T16:37:50Z Even Rouault even.rouault@spatialys.com 2017-06-12T10:23:55Z urn:sha1:73d1510d473b7dcfccfdee57e0e511e6791d5091 There are situations where, given a tile size, at a resolution level, there are sub-bands with x0==x1 or y0==y1, that consequently don't have any valid codeblocks, but the other sub-bands may be non-empty. Given that we recycle the memory from one tile to another one, those ghost codeblocks might be non-0 and thus candidate for packet inclusion. 2017-06-02T16:24:07Z Even Rouault even.rouault@spatialys.com 2017-06-02T14:49:26Z urn:sha1:532243f1fd9997db63ea7f6b199d21138ccf58a3 This saves comparing the current pointer with the end of buffer pointer. This results at least in tiny speed improvement for raw decoding, and smaller code size for MQC as well. This kills the remains of the raw.h/.c files that were only used for decoding. Encoding using the mqc structure already. 2017-05-09T18:46:20Z Even Rouault even.rouault@spatialys.com 2017-05-09T13:44:46Z urn:sha1:563bd8499e63db976ca8358216138647593354bc