Musings.

[dcpomatic.git] / doc / design / decoder_structures.tex

diff --git a/doc/design/decoder_structures.tex b/doc/design/decoder_structures.tex
index 1f7ae0ec48dcef5cb9ff6f5a7338e10639c95dde..3aa85b1ec558771183460edec9ec60d8431a78ef 100644 (file)
--- a/doc/design/decoder_structures.tex
+++ b/doc/design/decoder_structures.tex
@@ -1,4 +1,6 @@
 \documentclass{article}
+\usepackage[usenames]{xcolor}
+\usepackage{listings}
 \title{Decoder structures}
 \author{}
 \date{}
@@ -12,7 +14,8 @@ hides a decode-some-and-see-what-comes-out approach.
 
 With most decoders it is quick, easy and reliable to get a particular
 piece of content from a particular timecode.  This applies to the DCP,
-DCP subtitle, Sndfile and Image decoders.  With FFmpeg, however, this is not easy.
+DCP subtitle, Image and Video MXF decoders.  With FFmpeg, however,
+this is not easy.
 
 This suggests that it would make more sense to keep the
 decode-and-see-what-comes-out code within the FFmpeg decoder and not
@@ -20,7 +23,7 @@ use it anywhere else.
 
 However resampling screws this up, as it means all audio requires
 decode-and-see.  I don't think you can't resample in neat blocks as
-there are fractional samples other complications.  You can't postpone
+there are fractional samples and other complications.  You can't postpone
 resampling to the end of the player since different audio may be
 coming in at different rates.
 
@@ -28,6 +31,10 @@ This suggests that decode-and-see is a better match, even if it feels
 a bit ridiculous when most of the decoders have slightly clunky seek
 and pass methods.
 
+Having said that: the only other decoder which produces audio is now
+the DCP one, and maybe that never needs to be resampled.
+
+
 \section{Multiple streams}
 
 Another thing unique to FFmpeg is multiple audio streams, possibly at
@@ -50,4 +57,103 @@ and the merging of all audio content inside the player.
 
 These disadvantages suggest that the first approach is better.
 
+One might think that the logical conclusion is to take streams all the
+way back to the player and resample them there, but the resampling
+must occur on the other side of the get-stuff-at-time API.
+
+
+\section{Going back}
+
+Thinking about this again in October 2016 it feels like the
+get-stuff-at-this-time API is causing problems.  It especially seems
+to be a bad fit for previewing audio.  The API is nice for callers,
+but there is a lot of dancing around behind it to make it work, and it
+seems that it is more `flexible' than necessary; all callers ever do
+is seek or run.
+
+Hence there is a temptation to go back to see-what-comes-out.
+
+There are two operations: make DCP and preview.  Make DCP seems to be
+
+\lstset{language=C++}
+\lstset{basicstyle=\footnotesize\ttfamily,
+        breaklines=true,
+        keywordstyle=\color{blue}\ttfamily,
+        stringstyle=\color{red}\ttfamily,
+        commentstyle=\color{olive}\ttfamily}
+
+\begin{lstlisting}
+  while (!done) {
+    done = player->pass();
+    // pass() causes things to appear which are
+    // sent to encoders / disk
+  }
+\end{lstlisting}
+
+And preview seems to be
+
+\begin{lstlisting}
+  // Thread 1
+  while (!done) {
+    done = player->pass();
+    // pass() causes things to appear which are buffered
+    sleep_until_buffers_empty();
+  }
+
+  // Thread 2
+  while (true) {
+    get_video_and_audio_from_buffers();
+    push_to_output();
+    sleep();
+  }
+\end{lstlisting}
+
+\texttt{Player::pass} must call \texttt{pass()} on its decoders.  They
+will emit stuff which \texttt{Player} must adjust (mixing sound etc.).
+Player then emits the `final cut', which must have properties like no
+gaps in video/audio.
+
+Maybe you could have a parent class for simpler get-stuff-at-this-time
+decoders to give them \texttt{pass()} / \texttt{seek()}.
+
+One problem I remember is which decoder to \texttt{pass()} at any given time:
+it must be the one with the earliest last output, presumably.
+Resampling also looks fiddly in the v1 code.
+
+
+\section{Having a go}
+
+\begin{lstlisting}
+  class Decoder {
+    virtual void pass() = 0;
+    virtual void seek(ContentTime time, bool accurate) = 0;
+
+    signals2<void (ContentVideo)> Video;
+    signals2<void (ContentAudio, AudioStreamPtr)> Audio;
+    signals2<void (ContentTextSubtitle)> TextSubtitle;
+  };
+\end{lstlisting}
+
+or perhaps
+
+\begin{lstlisting}
+  class Decoder {
+    virtual void pass() = 0;
+    virtual void seek(ContentTime time, bool accurate) = 0;
+
+    shared_ptr<VideoDecoder> video;
+    shared_ptr<AudioDecoder> audio;
+    shared_ptr<SubtitleDecoder> subtitle;
+  };
+
+  class VideoDecoder {
+    signals2<void (ContentVideo)> Data;
+  };
+\end{lstlisting}
+
+Questions:
+\begin{itemize}
+\item Video / audio frame or \texttt{ContentTime}?
+\end{itemize}
+
 \end{document}