Try to use the previous frame rather than the current one when padding PTS gaps.