Anthropic API: prompt is too long / context window exceeded

Quick fix

Read the commands before running them. Anything that restarts a service, deletes data or changes permissions should be tried on a non-production system first.

# Count tokens first
token_count = client.messages.count_tokens(
  model='claude-3-5-sonnet-latest',
  messages=messages
)
print(token_count.input_tokens)
# Reduce max_tokens, summarize history, use RAG, or use prompt caching for static context

How to diagnose AI errors

AI and LLM errors cluster into four families: quota and rate limiting (429s, insufficient_quota), context window overflow (the prompt plus the requested completion exceeds the model's limit), accelerator memory (CUDA OOM, KV-cache exhaustion), and content policy (a refusal or safety stop rather than a transport failure). The first thing to establish is which family you are in, because the fixes have nothing in common: a 429 wants backoff and a quota increase, a context overflow wants truncation or chunking, and a CUDA OOM wants a smaller batch or a quantised model.

If the quick fix above does not resolve it, work through these steps. They apply to this whole class of error, not just to this one message, which is usually what saves the time.

1. Read the error body, not just the status code. Providers put the real reason in a JSON error.type / error.code field: rate_limit_exceeded and insufficient_quota are both HTTP 429 but mean completely different things.
2. Count your tokens before you send. Use the provider's tokenizer (tiktoken, Anthropic's count-tokens endpoint) rather than guessing from character length, and remember that max_tokens for the response is reserved inside the context window.
3. For local inference, watch VRAM live with nvidia-smi -l 1 while the request runs. Memory that peaks during the forward pass rather than at load time points at batch size or sequence length, not at model weights.
4. Distinguish a refusal from a failure. A stop_reason of refusal, or a finishReason of SAFETY, is a successful HTTP 200. Retrying identical input will produce the same result.
5. Always implement exponential backoff with jitter and honour the retry-after header. Most production LLM incidents are self-inflicted retry storms.

Tools worth reaching for

• tiktoken
• nvidia-smi
• curl -i
• provider status pages

Authoritative references

Primary documentation for this error, worth reading before applying any fix in production.

📖 docs.anthropic.com

Related AI errors

• Anthropic API: 429 rate_limit_error (tokens per minute)The organisation exceeded its input or output token budget for the minute, not its request…
• Anthropic API: invalid_request_error on tool_use / tool_resultTool calling requires strict message pairing: every assistant tool_use block must be answered…
• Anthropic API: Overloaded (529)The Anthropic API is temporarily overloaded and cannot process your request. This is a…
• AWS Bedrock: AccessDeniedException (no access to the model)Bedrock foundation models must be explicitly enabled per account and per region before they…
• Azure OpenAI: DeploymentNotFoundAzure OpenAI addresses a *deployment name* you chose, not the underlying model name. Passing…
• Claude: response blocked / safety refusalThe model declined to answer because the prompt matched a disallowed or ambiguous safety…
• CUDA: driver version is insufficient for CUDA runtime versionThe installed NVIDIA driver is older than the CUDA runtime the framework was built against…
• CUDA: no kernel image is available for execution on the deviceThe binary contains no compiled code for your GPU's compute capability. It happens with a…

See all 35 AI errors →

Browse other categories

• HTTP 494xx client errors, 5xx server errors, redirects, headers and protocol problems.
• JavaScript 42npm resolution, async pitfalls, hydration, memory limits and runtime type…
• Database 41Connections, deadlocks, constraints, replication and memory limits.
• Network 35Refused connections, timeouts, resets, MTU problems and port exhaustion.
• Python 35Imports, virtual environments, encoding, concurrency and dependency conflicts.
• Kubernetes 34CrashLoopBackOff, ImagePullBackOff, OOMKilled, RBAC, scheduling and storage.
• Docker 27Daemon connectivity, disk space, image pulls, ports and architecture mismatches.
• System 26Disk space, systemd units, file descriptors, OOM killer and scheduled jobs.
• Cloud 25IAM permissions, quotas, service limits and credential failures.
• Security 25JWT validation, CSRF, OAuth grants, SELinux, SSH host keys and CSP.
• TLS 24Untrusted authorities, expiry, hostname mismatch, chains and cipher negotiation.
• Frontend 23Hydration mismatches, bundler resolution, layout shift and font loading.

Something missing or wrong?

This entry is maintained by hand. If the fix is out of date, incomplete, or you have a better one, email a correction and it will be reviewed.