SECURITY WARNING: Never run commands you don't understand. Always review code before execution. Use at your own risk.
FixMyError.net
AI 35 errors

AI & LLM API Errors

Rate limits, context windows, GPU memory and model-serving failures.

Understanding AI errors

AI and LLM errors cluster into four families: quota and rate limiting (429s, insufficient_quota), context window overflow (the prompt plus the requested completion exceeds the model's limit), accelerator memory (CUDA OOM, KV-cache exhaustion), and content policy (a refusal or safety stop rather than a transport failure). The first thing to establish is which family you are in, because the fixes have nothing in common: a 429 wants backoff and a quota increase, a context overflow wants truncation or chunking, and a CUDA OOM wants a smaller batch or a quantised model.

How to debug AI errors

  1. Read the error body, not just the status code. Providers put the real reason in a JSON error.type / error.code field: rate_limit_exceeded and insufficient_quota are both HTTP 429 but mean completely different things.
  2. Count your tokens before you send. Use the provider's tokenizer (tiktoken, Anthropic's count-tokens endpoint) rather than guessing from character length, and remember that max_tokens for the response is reserved inside the context window.
  3. For local inference, watch VRAM live with nvidia-smi -l 1 while the request runs. Memory that peaks during the forward pass rather than at load time points at batch size or sequence length, not at model weights.
  4. Distinguish a refusal from a failure. A stop_reason of refusal, or a finishReason of SAFETY, is a successful HTTP 200. Retrying identical input will produce the same result.
  5. Always implement exponential backoff with jitter and honour the retry-after header. Most production LLM incidents are self-inflicted retry storms.

Tools worth reaching for

  • tiktoken
  • nvidia-smi
  • curl -i
  • provider status pages

All 35 AI errors

Other categories