Make backoff and retry a bit nicer; fix crash when accept() returns EAGAIN.