Design API Error Handling and Backoff

Goal

Implement a request policy that neither retries forever nor hides authentication failures.

Be able to record HTTP status, API error code, request ID, and attempt count.

Treat 400, 401, 403, and model_not_found as terminal errors requiring corrected input or configuration.
Retry only 429 responses, timeouts, and explicitly temporary service errors.
Use exponential backoff with jitter, such as 1, 2, and 4 seconds, with a maximum of three retries.
Prefer a Retry-After value when the response provides one.
Log request ID, status, error code, and delay without logging tokens or sensitive bodies.
Verify branches with simulated 401, 429, and repeated timeout responses.

Terminal errors return immediately, retryable errors recover within the limit, and exhausted retries leave a useful diagnostic record.

Simultaneous retries create a spike: add random jitter.
Work is duplicated: use idempotency keys for side effects or disable automatic retries.
Logs expose secrets: centrally redact authorization headers and sensitive fields.