Commit 3475223
authored
Optimize retry_with_backoff
The optimization replaces the blocking `time.sleep()` with the non-blocking `await asyncio.sleep()`, which provides a **84.9% throughput improvement** despite appearing to have slightly higher individual runtime.
**Key Change:**
- `time.sleep(0.0001 * attempt)` → `await asyncio.sleep(0.0001 * attempt)`
**Why This Optimization Works:**
The blocking `time.sleep()` in the original code completely blocks the entire event loop thread, preventing any other async operations from executing during backoff periods. This creates a bottleneck when multiple retry operations run concurrently.
The `await asyncio.sleep()` yields control back to the event loop, allowing other coroutines to execute while waiting. This dramatically improves concurrency - the event loop can process hundreds of other retry operations during any single backoff period.
**Performance Impact Analysis:**
From the line profiler results, the sleep operation went from consuming 90.5% of execution time (46ms) to only 38.7% (2.9ms) - a **15x reduction** in sleep overhead per operation. While individual function calls may take slightly longer due to async overhead, the concurrent throughput increases massively because the event loop isn't blocked.
**Test Case Benefits:**
The optimization particularly benefits test cases with:
- **Concurrent execution** (test_retry_with_backoff_concurrent_* tests) - Multiple retry operations can now overlap their backoff periods
- **High-volume scenarios** (throughput tests with 100-500 concurrent calls) - The event loop can efficiently multiplex between many retry operations
- **Mixed failure patterns** - When some operations need retries while others don't, successful operations aren't blocked by others' backoff periods
This is a classic async optimization where individual latency may increase slightly, but system-wide throughput improves dramatically due to better concurrency utilization.1 parent 9692e75 commit 3475223
1 file changed
+2
-2
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | | - | |
| 15 | + | |
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
21 | | - | |
| 21 | + | |
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| |||
0 commit comments