API Retry Strategies

Learn proven retry patterns for AI APIs. Handle failures gracefully with exponential backoff and jitter.

Fixed Delay

Wait same time between retries

1s → 1s → 1s → 1s

Exponential

Double wait time each retry

1s → 2s → 4s → 8s

Exponential + Jitter

Randomized exponential backoff

1.2s → 3.1s → 6.8s → 9.2s

Retry Delay Comparison

Fixed Delay (1 second)

Exponential Backoff (1, 2, 4, 8)

Exponential backoff gives services time to recover while jitter prevents thundering herd

Implementation

// Retry configuration
const retryConfig = {
  "maxRetries": 5,
  "initialDelay": 1000,
  "maxDelay": 30000,
  "backoff": "exponential",
  "jitter": true,
  "retryableErrors": [
    "ECONNRESET",
    "ETIMEDOUT",
    "429",
    "500",
    "502",
    "503"
  ]
};

// Retry function
async function retryWithBackoff(fn) {
  let delay = config.initialDelay;
  
  for (let i = 0; i < config.maxRetries; i++) {
    try {
      return await fn();
    } catch (err) {
      if (!isRetryable(err)) throw err;
      
      await sleep(delay);
      delay = Math.min(delay * 2, config.maxDelay);
      if (config.jitter) delay *= Math.random();
    }
  }
  throw new Error("Max retries exceeded");
}
        

Key Points

1
Use Exponential Backoff Double delay each retry to give services time to recover
2
Add Jitter Randomize delays to prevent thundering herd problem
3
Set Max Retries Limit retries to prevent infinite loops, typically 3-5 attempts
4
Only Retry Transient Errors Never retry 400, 401, 403 - these indicate client issues

Frequently Asked Questions

What's jitter and why use it?

Jitter adds randomness to retry delays. Without it, all failed requests retry at the same time, overwhelming the recovering service. Jitter distributes retries more evenly.

How many retries should I configure?

3-5 retries is typical. Too few and you fail on transient errors. Too many and you delay recovery while potentially overloading the service.

Should I retry on rate limit errors?

Yes, but respect Retry-After header if present. Rate limits are temporary and worth retrying after the specified delay.

API Retry Strategies

Fixed Delay

Exponential

Exponential + Jitter

Retry Delay Comparison

Implementation

Key Points

Frequently Asked Questions

Related Resources

Exception Handling

Error Codes

Error Messages

Home