Work around deadlock when destroying J2KEncoder with a full writer queue (#2784).v2.16.83

This feels like a hack, but I can't think of a nicer way to do it.

The interruption disable makes sense because when we destroy encoder threads
during a DCP encode (because a remote server goes away, for example) we don't
want any frames to be lost due to the encode thread being interrupted between
taking the frame off the queue and sending it to the writer.

When we're destroying the encoder we don't care about this, but I can't see
how you'd differentiate.

Maybe the encoder queue could have two lists: to-do and in-progress;
the encoder thread atomically moves a frame from to-do to in-progress,
but then how do you know when the in-progress ones are orphaned and need
to be re-added to the main queue.

You could make the writer return saying "no" if the queue is full (rather
than blocking and waiting for the queue to empty) but that seems wasteful
as then the frame would be re-encoded.

3 files changed, 35 insertions, 0 deletions

diff --git a/src/lib/j2k_encoder.cc b/src/lib/j2k_encoder.cc
index 7c9777c16..32d2fefc2 100644
--- a/src/lib/j2k_encoder.cc
+++ b/src/lib/j2k_encoder.cc
@@ -69,6 +69,16 @@ J2KEncoder::~J2KEncoder ()
 {
 	_server_found_connection.disconnect();
 
+	/* One of our encoder threads may be waiting on Writer::write() to return, if that method
+	 * is blocked with the writer queue full waiting for _full_condition.  In that case, the
+	 * attempt to terminate the encoder threads below (in terminate_threads()) will fail because
+	 * the encoder thread waiting for ::write() will have interruption disabled.
+	 *
+	 * To work around that, make the writer into a zombie to unblock any pending write()s and
+	 * not block on any future ones.
+	 */
+	_writer.zombify();
+
 	boost::mutex::scoped_lock lm (_threads_mutex);
 	terminate_threads ();
 }
diff --git a/src/lib/writer.cc b/src/lib/writer.cc
index fbe2d248d..7b9defd73 100644
--- a/src/lib/writer.cc
+++ b/src/lib/writer.cc
@@ -144,6 +144,10 @@ Writer::write (shared_ptr<const Data> encoded, Frame frame, Eyes eyes)
 {
 	boost::mutex::scoped_lock lock (_state_mutex);
 
+	if (_zombie) {
+		return;
+	}
+
 	while (_queued_full_in_memory > _maximum_frames_in_memory) {
 		/* There are too many full frames in memory; wake the main writer thread and
 		   wait until it sorts everything out */
@@ -377,6 +381,9 @@ try
 	while (true)
 	{
 		boost::mutex::scoped_lock lock (_state_mutex);
+		if (_zombie) {
+			return;
+		}
 
 		while (true) {
 
@@ -1042,3 +1049,17 @@ Writer::write_hanging_text (ReelWriter& reel)
 	}
 	_hanging_texts = new_hanging_texts;
 }
+
+
+/** Set the writer so that it has no queue and drops any pending or future requests to write images */
+void
+Writer::zombify()
+{
+	boost::mutex::scoped_lock lock(_state_mutex);
+
+	_queue.clear();
+	_queued_full_in_memory = 0;
+	_zombie = true;
+	_full_condition.notify_all();
+}
+
diff --git a/src/lib/writer.h b/src/lib/writer.h
index 6e7f5ca90..f0f1fe69a 100644
--- a/src/lib/writer.h
+++ b/src/lib/writer.h
@@ -127,6 +127,8 @@ public:
 
 	void set_encoder_threads (int threads);
 
+	void zombify();
+
 private:
 	friend struct ::writer_disambiguate_font_ids1;
 	friend struct ::writer_disambiguate_font_ids2;
@@ -229,6 +231,8 @@ private:
 	};
 
 	std::vector<HangingText> _hanging_texts;
+
+	bool _zombie = false;
 };