openjpeg/src/lib/openjp2/tcd.c, branch v2.2.0Fork of openjpeg library for JPEG2000 encoding and decodinghttps://git.carlh.net/cgit/openjpeg/atom?h=v2.2.02017-08-07T16:32:52ZSlight improvement in management of code block chunks2017-08-07T16:32:52ZEven Rouaulteven.rouault@spatialys.com2017-07-06T17:34:21Zurn:sha1:92114694a48638e86c07e4adf11b78878045c8a9
Instead of having the chunk array at the segment level, we can move it down to
the codeblock itself since segments are filled in sequential order.
Limit the number of memory allocation, and decrease slightly the memory usage.
On MAPA_005.jp2
n4: 1871312549 (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
n1: 1610689344 0x4E781E7: opj_aligned_malloc (opj_malloc.c:61)
n1: 1610689344 0x4E71D1B: opj_alloc_tile_component_data (tcd.c:676)
n1: 1610689344 0x4E726CF: opj_tcd_init_decode_tile (tcd.c:816)
n1: 1610689344 0x4E4BE39: opj_j2k_read_tile_header (j2k.c:8617)
n1: 1610689344 0x4E4C902: opj_j2k_decode_tiles (j2k.c:10348)
n1: 1610689344 0x4E4E3CE: opj_j2k_decode (j2k.c:7846)
n1: 1610689344 0x4E53002: opj_jp2_decode (jp2.c:1564)
n0: 1610689344 0x40374E: main (opj_decompress.c:1459)
n1: 219232541 0x4E4BC50: opj_j2k_read_tile_header (j2k.c:4683)
n1: 219232541 0x4E4C902: opj_j2k_decode_tiles (j2k.c:10348)
n1: 219232541 0x4E4E3CE: opj_j2k_decode (j2k.c:7846)
n1: 219232541 0x4E53002: opj_jp2_decode (jp2.c:1564)
n0: 219232541 0x40374E: main (opj_decompress.c:1459)
n1: 23893200 0x4E72735: opj_tcd_init_decode_tile (tcd.c:1225)
n1: 23893200 0x4E4BE39: opj_j2k_read_tile_header (j2k.c:8617)
n1: 23893200 0x4E4C902: opj_j2k_decode_tiles (j2k.c:10348)
n1: 23893200 0x4E4E3CE: opj_j2k_decode (j2k.c:7846)
n1: 23893200 0x4E53002: opj_jp2_decode (jp2.c:1564)
n0: 23893200 0x40374E: main (opj_decompress.c:1459)
n0: 17497464 in 52 places, all below massif's threshold (1.00%)
Decoding: do not allocate memory for the codestream of each codeblock2017-08-07T16:32:52ZEven Rouaulteven.rouault@spatialys.com2017-07-06T14:11:11Zurn:sha1:ca34d13e76a588a00171e57690c1deeaf068723a
Currently we allocate at least 8192 bytes for each codeblock, and copy
the relevant parts of the codestream in that per-codeblock buffer as we
decode packets.
As the whole codestream for the tile is ingested in memory and alive
during the decoding, we can directly point to it instead of copying. But
to do that, we need an intermediate concept, a 'chunk' of code-stream segment,
given that segments may be made of data at different places in the code-stream
when quality layers are used.
With that change, the decoding of MAPA_005.jp2 goes down from the previous
improvement of 2.7 GB down to 1.9 GB.
New profile:
n4: 1885648469 (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
n1: 1610689344 0x4E78287: opj_aligned_malloc (opj_malloc.c:61)
n1: 1610689344 0x4E71D7B: opj_alloc_tile_component_data (tcd.c:676)
n1: 1610689344 0x4E7272C: opj_tcd_init_decode_tile (tcd.c:816)
n1: 1610689344 0x4E4BDD9: opj_j2k_read_tile_header (j2k.c:8618)
n1: 1610689344 0x4E4C8A2: opj_j2k_decode_tiles (j2k.c:10349)
n1: 1610689344 0x4E4E36E: opj_j2k_decode (j2k.c:7847)
n1: 1610689344 0x4E52FA2: opj_jp2_decode (jp2.c:1564)
n0: 1610689344 0x40374E: main (opj_decompress.c:1459)
n1: 219232541 0x4E4BBF0: opj_j2k_read_tile_header (j2k.c:4685)
n1: 219232541 0x4E4C8A2: opj_j2k_decode_tiles (j2k.c:10349)
n1: 219232541 0x4E4E36E: opj_j2k_decode (j2k.c:7847)
n1: 219232541 0x4E52FA2: opj_jp2_decode (jp2.c:1564)
n0: 219232541 0x40374E: main (opj_decompress.c:1459)
n1: 39822000 0x4E727A9: opj_tcd_init_decode_tile (tcd.c:1219)
n1: 39822000 0x4E4BDD9: opj_j2k_read_tile_header (j2k.c:8618)
n1: 39822000 0x4E4C8A2: opj_j2k_decode_tiles (j2k.c:10349)
n1: 39822000 0x4E4E36E: opj_j2k_decode (j2k.c:7847)
n1: 39822000 0x4E52FA2: opj_jp2_decode (jp2.c:1564)
n0: 39822000 0x40374E: main (opj_decompress.c:1459)
n0: 15904584 in 52 places, all below massif's threshold (1.00%)
Add documentation for magic values in the code2017-08-07T16:32:52ZEven Rouaulteven.rouault@spatialys.com2017-07-06T11:23:29Zurn:sha1:373520db309430b68dd9ff09cba03a25f711a88eAdd opj_image_data_alloc() / opj_image_data_free()2017-08-07T16:32:52ZEven Rouaulteven.rouault@spatialys.com2017-07-05T23:47:40Zurn:sha1:f58aab9d6a57e48bdc60e15dd373c24de74719a9
As bin/common/color.c used to directly call malloc()/free(), we need
to export functions dedicated to allocating/freeing image component data.
opj_tcd_dc_level_shift_decode: avoid int32 overflow when prec == 31. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2799. Credit to OSS Fuzz2017-07-30T13:22:24ZEven Rouaulteven.rouault@spatialys.com2017-07-30T13:22:24Zurn:sha1:68832af20e3b3710424947e12762b6b52d3b6ac0Fix write heap buffer overflow in opj_mqc_byteout(). Discovered by Ke Liu of Tencent's Xuanwu LAB (#835)2017-07-29T17:13:49ZEven Rouaulteven.rouault@spatialys.com2017-07-29T17:13:49Zurn:sha1:397f62c0a838e15d667ef50e27d5d011d2c79c04opj_tcd_get_decoded_tile_size(): fix potential UINT32 overflow (#854, CVE-2016-5152)2017-07-29T16:38:16ZEven Rouaulteven.rouault@spatialys.com2017-07-29T16:38:16Zurn:sha1:3fbe71369019df0b47c7a2be4fab8c05768f2f32
Fix derived from https://pdfium.googlesource.com/pdfium.git/+/d8cc503575463ff3d81b22dad292665f2c88911e/third_party/libopenjpeg20/0018-tcd_get_decoded_tile_size.patch
opj_tcd_dc_level_shift_decode(): avoid int overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2516. Credit to OSS Fuzz2017-07-28T20:06:26ZEven Rouaulteven.rouault@spatialys.com2017-07-28T20:06:26Zurn:sha1:361c4506fdfb9b6f0e41d86d5d0ea1efb5704ecfopj_j2k_update_image_data / opj_tcd_update_tile_data: fix unaligned load/store (#895)2017-07-27T17:35:35ZEven Rouaulteven.rouault@spatialys.com2017-07-27T17:34:54Zurn:sha1:820fcfe8bb101a2862c076b02c9b6b636ce39d2f
When components don't have the same width, unaligned load/store are possible.
Fixes openjeg-crashes-2017-07-27/id:000000,sig:11,src:001342,op:flip4,pos:162.jp2 of #895
T1 decoder: check code stream errors when predictable termination is enabled and emit a warning when errors are found2017-07-26T19:43:32ZEven Rouaulteven.rouault@spatialys.com2017-07-26T19:39:50Zurn:sha1:94c4b7300cc515330bf798bb45eb0d68e2c84aa0