Logging Pipeline Errors
Log4j, Logback, Fluentd, Logstash and CloudWatch ingestion problems.
Understanding Logging errors
Logging failures are dangerous because they are silent: the application keeps running while its telemetry disappears. The recurring causes are buffer overflow under backpressure (the destination cannot keep up), permissions on log files or directories, and ingestion rate limits at the cloud provider. Monitoring the logging pipeline itself is the only reliable way to notice.
How to debug Logging errors
- Check the collector's own logs first: Fluentd, Logstash and the CloudWatch agent all log their own failures, usually to a separate destination.
- Look for backpressure metrics: buffer queue length, retry counts, and dropped-record counters. A full buffer means the destination, not the collector, is the bottleneck.
- Verify write permissions and disk space on the buffer path. A full disk silently stops most collectors.
- For cloud ingestion, check the API rate limit for the log group or stream and batch more aggressively rather than retrying harder.
- Add a heartbeat log line and alert on its absence. This is the only way to detect a pipeline that has stopped without erroring.
Tools worth reaching for
collector self-logsbuffer/queue metricsdf -haws logs describe-log-streamslogger / fluent-cat for test events
All 18 Logging errors
- CloudWatch Logs: Rate exceeded (ThrottlingException)PutLogEvents API calls are being throttled by CloudWatch Logs. The account or log group is exceeding the API rate limit.
- Elasticsearch: Limit of total fields [1000] has been exceeded NewDynamic mapping adds a new field for every new key it sees, and JSON logs whose keys contain IDs, headers or arbitrary…
- Error response from daemon: configured logging driver does not support reading NewThe container sends its logs to a driver such as syslog, gelf, fluentd or awslogs, so they never touch the local disk…
- Fluent Bit: Action/metadata line [1] contains an unknown parameter [_type] NewElasticsearch 8 removed mapping types, and rejects any bulk request that still sends _type. Fluent Bit's es output…
- Fluentd: Buffer overflowFluentd's output buffer is full because the destination cannot accept data fast enough. Logs may be dropped.
- journalctl: No journal files were found Newjournald stores logs in memory when /var/log/journal does not exist, so everything is discarded at reboot. The service…
- Log4j: Appender not foundThe logging configuration references an appender that is not defined. The log4j2.xml or log4j.properties file has an…
- log4j2: ERROR StatusLogger No Log4j 2 configuration file found NewLog4j 2 could not find log4j2.xml on the classpath and fell back to logging errors only to the console, which is why…
- Logback: Failed to create log fileLogback cannot create or write to the log file due to filesystem permissions or the directory not existing.
- logrotate: skipping because parent directory has insecure permissions Newlogrotate refuses to rotate a file in a directory that is group or world writable, because doing that work as root…
- Logstash: Pipeline errorA Logstash pipeline failed to start or process events, usually due to a bad grok pattern, filter configuration, or…
- Loki: entry too far behind / out of order NewLoki rejected log lines whose timestamps are older than the accepted window for that stream. Usually a backfill, a…
- Loki: error at least one label pair is required per stream NewLoki indexes streams by their labels and will not accept a push whose stream has none. The shipper is configured, but…
- Loki: Maximum active stream limit exceeded, reduce the number of active streams NewEvery unique set of labels is its own stream in Loki, and the tenant has gone over its stream limit. That nearly…
- Python: --- Logging error --- TypeError: not all arguments converted NewThe logging module formats messages lazily with the % operator, and a call like logger.info("user", user_id) passes an…
- Python: logging.basicConfig() has no effect and nothing is logged NewbasicConfig only does anything when the root logger has no handlers yet, so it is a silent no operation if any import…
- SLF4J: Failed to load class org.slf4j.impl.StaticLoggerBinder NewThe SLF4J API is on the classpath with no binding behind it, so every log call is discarded through the no operation…
- systemd-journald: Suppressed 1234 messages from /system.slice/app.service Newjournald rate limits each service, discarding the rest of a burst rather than slowing the writer. The lines are gone…
Other categories
- AI 35Rate limits, context windows, GPU memory and model-serving failures.
- Ansible 18Unreachable hosts, become passwords, undefined variables and Jinja2 failures.
- API 20Auth headers, payload limits, versioning, idempotency and webhook signatures.
- Apple 20Command line tools, dyld, Homebrew permissions, notarisation and Keychain.
- Auth 20OIDC, SAML, Auth0, Okta, Keycloak, passkeys and MFA failures.
- BigData 17Spark, Kafka, Airflow, Snowflake, Flink and Databricks failures.
- C# 18NuGet restore, null references, EF Core migrations, async deadlocks and Blazor…
- C++ 20Segfaults, linker errors, memory corruption and template deduction failures.
- Caching 20Cache stampedes, stale content, Varnish and CloudFront failures.
- CI/CD 18GitHub Actions, GitLab CI, Jenkins, CircleCI: permissions, runners and…
- Client 21CORS, mixed content, module resolution, memory limits and framework runtime…
- Cloud 25IAM permissions, quotas, service limits and credential failures.
- Dart 20Null safety, pub version solving and build toolchain problems.
- Database 41Connections, deadlocks, constraints, replication and memory limits.
- DNS 20NXDOMAIN, SERVFAIL, timeouts, propagation and delegation problems.
- Docker 27Daemon connectivity, disk space, image pulls, ports and architecture mismatches.
- Elixir 18GenServer timeouts, supervision failures and Mix compilation problems.
- Email 20Delivery failures, relay denial, authentication, SPF, DKIM and DMARC.
- Frontend 23Hydration mismatches, bundler resolution, layout shift and font loading.
- Git 20Merge conflicts, rejected pushes, detached HEAD, LFS and repository corruption.
- Go 19Nil map assignment, concurrent map access, context cancellation and deadlocks.
- GraphQL 19Validation, depth limits, N+1 queries and fragment problems.
- gRPC 17Status codes, deadlines, message limits, TLS and HTTP/2 transport failures.
- HTTP 494xx client errors, 5xx server errors, redirects, headers and protocol problems.
- ICMP 23Destination unreachable, time exceeded, fragmentation needed and redirects.
- Ingress 20404 default backend, missing TLS secrets, IngressClass and path matching.
- Java 19Class loading, dependency resolution, connection pools and JVM version…
- JavaScript 42npm resolution, async pitfalls, hydration, memory limits and runtime type…
- Kubernetes 34CrashLoopBackOff, ImagePullBackOff, OOMKilled, RBAC, scheduling and storage.
- MessageQueue 20Kafka, RabbitMQ, SQS, NATS and Celery: lag, rebalancing and poison messages.
- Mobile 20Gradle, CocoaPods, Xcode signing, Metro bundler and toolchain mismatches.
- Monitoring 18Prometheus scrapes, Grafana data sources, OpenTelemetry exporters and agent…
- Network 35Refused connections, timeouts, resets, MTU problems and port exhaustion.
- Performance 19GC pauses, thread pool starvation and event loop blocking.
- PHP 20Memory limits, execution timeouts, autoloading, Composer and PDO connections.
- Proxy 20nginx, Envoy, HAProxy, Traefik, Caddy and Cloudflare upstream failures.
- Python 35Imports, virtual environments, encoding, concurrency and dependency conflicts.
- Regex 17Catastrophic backtracking, back references, escaping and engine differences.
- Ruby 20Bundler, migrations, native extensions, encoding and asset compilation.
- Rust 19Borrow checker, ownership moves, trait bounds and lifetime mismatches.
- Scala 17Dependency resolution, binary compatibility and type inference failures.
- Security 25JWT validation, CSRF, OAuth grants, SELinux, SSH host keys and CSP.
- Serverless 20Lambda timeouts, package size limits, VPC networking and cold starts.
- Shell 19Command not found, permissions, quoting, expansion and Makefile syntax.
- Storage 19S3 permissions, NFS mounts, quotas, signed URLs and volume attachment.
- Svelte 20Store subscriptions, load functions and server/client boundaries.
- System 26Disk space, systemd units, file descriptors, OOM killer and scheduled jobs.
- Terraform 18State locks, provider auth, drift, dependency cycles and plan-time unknowns.
- Testing 18Jest, pytest, JUnit, Cypress and Playwright: fixtures, snapshots and timeouts.
- TLS 24Untrusted authorities, expiry, hostname mismatch, chains and cipher negotiation.
- TypeScript 19Assignability, missing declarations, strict null checks and generic constraints.
- Virtualization 20VirtualBox, VMware, Hyper-V, WSL, KVM and hypervisor conflicts.
- Web3 17Gas estimation, nonce management and reverted transactions.
- WebAssembly 17Compile errors, memory bounds and host binding mismatches.
- WebServer 20nginx, Apache, IIS and Caddy: binding, permissions, rewrites and TLS.
- Windows 20Installer failures, missing runtimes, update errors and permission problems.