Fix video range in preview and DCP when source is VIDEO rather than FULL.

It looks like the handling of "video" range (16-235) sources (when
creating "full" range (0-255) outputs) has been broken for some considerable
time.  Full range outputs are used for both preview and DCP creation.

We assumed that after we called sws_setColorspaceDetails FFmpeg would take
care of the conversion from video to full, but its source suggests that
it refuses to convert to full-range RGB (only full-range YUV or
greyscale).  Previously there was a seemingly correct comment (that
FFmpeg would do nothing if source _or_ destination were RGB) but the
check on when to use video_range_to_full_range() was only activated
for RGB sources, not (also) RGB outputs.

It's hard to believe that this has been around unnoticed for so long.

https://dcpomatic.com/forum/viewtopic.php?t=1928&start=10 has a
discussion which pointed out the problem.

diff --git a/src/lib/image.cc b/src/lib/image.cc
index 5866ee5b418a82b01eec841e832a9fd0bb8db86c..dfa093fbe3c38c61d8fa325078f0f073b78c2a6e 100644 (file)
--- a/src/lib/image.cc
+++ b/src/lib/image.cc
@@ -249,7 +249,7 @@ Image::crop_scale_window (
           1 -> destination range JPEG (i.e. "full", 0-255)
 
           But remember: sws_setColorspaceDetails ignores these
-          parameters unless the both source and destination images
+          parameters unless both source and destination images
           are isYUV or isGray.  (If either is not, it uses video range).
        */
        sws_setColorspaceDetails (
@@ -307,9 +307,12 @@ Image::crop_scale_window (
        if (
                video_range == VideoRange::VIDEO &&
                out_video_range == VideoRange::FULL &&
-               av_pix_fmt_desc_get(_pixel_format)->flags & AV_PIX_FMT_FLAG_RGB
+               ((av_pix_fmt_desc_get(_pixel_format)->flags & AV_PIX_FMT_FLAG_RGB) ||
+                (av_pix_fmt_desc_get(out_format)->flags & AV_PIX_FMT_FLAG_RGB))
           ) {
-               /* libswscale will not convert video range for RGB sources, so we have to do it ourselves */
+               /* libswscale will not convert video range if input or output is RGB, as far as I can see,
+                * so we have to do it ourselves.
+                */
                out->video_range_to_full_range ();
        }