Try to speed up cancel of encoder threads if they are sleeping for remote backoff.