Add some docs on the swaroop variant's differences.

[dcpomatic.git] / doc / design / decoder_structures.tex

diff --git a/doc/design/decoder_structures.tex b/doc/design/decoder_structures.tex
index d151aad7ecaa1782f56304babe10c7e014cad9b0..9b9be33cdd4ea0e6e552b8241a8770c623b15db4 100644 (file)
--- a/doc/design/decoder_structures.tex
+++ b/doc/design/decoder_structures.tex
@@ -14,7 +14,8 @@ hides a decode-some-and-see-what-comes-out approach.
 
 With most decoders it is quick, easy and reliable to get a particular
 piece of content from a particular timecode.  This applies to the DCP,
 
 With most decoders it is quick, easy and reliable to get a particular
 piece of content from a particular timecode.  This applies to the DCP,

-DCP subtitle, Sndfile and Image decoders.  With FFmpeg, however, this is not easy.
+DCP subtitle, Image and Video MXF decoders.  With FFmpeg, however,
+this is not easy.

 
 This suggests that it would make more sense to keep the
 decode-and-see-what-comes-out code within the FFmpeg decoder and not
 
 This suggests that it would make more sense to keep the
 decode-and-see-what-comes-out code within the FFmpeg decoder and not


@@ -22,7 +23,7 @@ use it anywhere else.
 
 However resampling screws this up, as it means all audio requires
 decode-and-see.  I don't think you can't resample in neat blocks as
 
 However resampling screws this up, as it means all audio requires
 decode-and-see.  I don't think you can't resample in neat blocks as

-there are fractional samples other complications.  You can't postpone
+there are fractional samples and other complications.  You can't postpone

 resampling to the end of the player since different audio may be
 coming in at different rates.
 
 resampling to the end of the player since different audio may be
 coming in at different rates.
 


@@ -30,6 +31,9 @@ This suggests that decode-and-see is a better match, even if it feels
 a bit ridiculous when most of the decoders have slightly clunky seek
 and pass methods.
 
 a bit ridiculous when most of the decoders have slightly clunky seek
 and pass methods.
 

+Having said that: the only other decoder which produces audio is now
+the DCP one, and maybe that never needs to be resampled.
+

 
 \section{Multiple streams}
 
 
 \section{Multiple streams}
 


@@ -109,17 +113,103 @@ will emit stuff which \texttt{Player} must adjust (mixing sound etc.).
 Player then emits the `final cut', which must have properties like no
 gaps in video/audio.
 
 Player then emits the `final cut', which must have properties like no
 gaps in video/audio.
 

-One problem I remember is which decoder to pass() at any given time:
+Maybe you could have a parent class for simpler get-stuff-at-this-time
+decoders to give them \texttt{pass()} / \texttt{seek()}.
+
+One problem I remember is which decoder to \texttt{pass()} at any given time:

 it must be the one with the earliest last output, presumably.
 Resampling also looks fiddly in the v1 code.
 
 it must be the one with the earliest last output, presumably.
 Resampling also looks fiddly in the v1 code.
 

-Possible steps:
+
+\section{Having a go}
+
+\begin{lstlisting}
+  class Decoder {
+    virtual void pass() = 0;
+    virtual void seek(ContentTime time, bool accurate) = 0;
+
+    signal<void (ContentVideo)> Video;
+    signal<void (ContentAudio, AudioStreamPtr)> Audio;
+    signal<void (ContentTextSubtitle)> TextSubtitle;
+  };
+\end{lstlisting}
+
+or perhaps
+
+\begin{lstlisting}
+  class Decoder {
+    virtual void pass() = 0;
+    virtual void seek(ContentTime time, bool accurate) = 0;
+
+    shared_ptr<VideoDecoder> video;
+    shared_ptr<AudioDecoder> audio;
+    shared_ptr<SubtitleDecoder> subtitle;
+  };
+
+  class VideoDecoder {
+    signals2<void (ContentVideo)> Data;
+  };
+\end{lstlisting}
+
+Questions:
+\begin{itemize}
+\item Video / audio frame or \texttt{ContentTime}?
+\item Can all the subtitle period notation code go?
+\end{itemize}
+
+\subsection{Steps}
+
+\begin{itemize}
+\item Add signals to \texttt{Player}.
+  \begin{itemize}
+    \item \texttt{signal<void (shared\_ptr<PlayerVideo>), DCPTime)> Video;}
+    \item \texttt{signal<void (shared\_ptr<AudioBuffers>, DCPTime)> Audio;}
+    \item \texttt{signal<void (PlayerSubtitles, DCPTimePeriod)> Subtitle;}
+  \end{itemize}
+  \item Remove \texttt{get()}-based loops and replace with \texttt{pass()} and signal connections.
+  \item Remove \texttt{get()} and \texttt{seek()} from decoder parts; add emission signals.
+  \item Put \texttt{AudioMerger} back.
+  \item Remove \texttt{during} stuff from \texttt{SubtitleDecoder} and decoder classes that use it.
+  \item Rename \texttt{give} methods to \texttt{emit}.
+  \item Remove \texttt{get} methods from \texttt{Player}; replace with \texttt{pass()} and \texttt{seek()}.
+\end{itemize}
+
+
+\section{Summary of work done in \texttt{back-to-pass}}
+
+The diff between \texttt{back-to-pass} and \texttt{master} as at 21/2/2017 can be summarised as:
+
+\begin{enumerate}
+\item Remove \texttt{AudioDecoderStream}; no more need to buffer, and resampling is done in \texttt{Player}.
+\item \texttt{AudioDecoder} is simple; basically counting frames.
+\item All subtitles-during stuff is gone; no need to know what happens in a particular period as we just wait and see.
+\item Pass reason stuff gone; not sure what it was for but seems to have been a contortion related to trying to find specific stuff.
+  \item \texttt{Player::pass} back, obviously.
+  \item \texttt{Player::get\_video}, \texttt{get\_audio} and
+    \texttt{get\_subtitle} more-or-less become \texttt{Player}'s
+    handlers for emissions from decoders; lots of buffering crap gone
+    in the process.
+  \item Add \texttt{Decoder::position} stuff so that we know what to \texttt{pass()} in \texttt{Player}.
+  \item Add \texttt{AudioMerger}; necessary as audio arrives at the
+    \texttt{Player} from different streams at different times.  The
+    \texttt{AudioMerger} just accepts data, mixes and spits it out
+    again.
+\item \texttt{AudioMerger} made aware of periods with no content to
+  allow referenced reels; adds a fair amount of complexity.  Without
+  this the referenced reel gaps are silence-padded which confuses
+  things later on as our VF DCP gets audio data that it does not need.
+\item Obvious consumer changes: what was a loop over the playlist
+  length and calls to \texttt{get()} is now calls to \texttt{pass()}.
+  \item Maybe-seek stuff gone.
+  \item Some small \texttt{const}-correctness bits.
+\end{enumerate}
+
+Obvious things to do:
+

 \begin{enumerate}
 \begin{enumerate}

-\item Add signals to \texttt{Player}; remove \texttt{get\_*}
-\item Give player a \texttt{pass()} which calls decoders and sanitises
-  output.
-\item Make transcoder attach to \texttt{Player} and pass output through to encoding.
-\item Make preview attach to \texttt{Player}, buffer the output and then fetch it from a UI thread.
+\item Ensure AudioMerger is being tested.
+\item Ensure hardest-case in video / audio is being tested.
+\item Look at symmetry of video/audio paths / APIs.

 \end{enumerate}
 
 \end{document}
 \end{enumerate}
 
 \end{document}