Monitoring

The Metrics That Actually Predict a Database Incident

CPU and memory graphs tell you a database is unhealthy after the fact. These five signals — tracked as trends, not snapshots — tell you before your users do.

May 22, 2026 · 7 min read · AG Data Team

Most database dashboards are built around CPU, memory, and disk usage. They're not wrong to track — but by the time any of them are red, the incident is usually already underway. The signals that actually give you warning are further upstream, and they're rarely on the default dashboard.

1. Replication lag — as a trend, not a snapshot

A single lag reading of "12 seconds" tells you almost nothing on its own. What matters is the trend: is lag flat, or is it climbing? A replica that's steadily falling further behind under the same workload is telling you it's running out of headroom — on I/O, on apply-worker throughput, or on something contending for the same resources. By the time lag has visibly spiked, the replica has usually already been degrading for a while.

2. Lock wait time and blocked-query count

Long-running transactions holding locks are one of the most common precursors to a cascading failure — one slow query blocks a handful of others, which queue up connections, which exhausts the pool, which turns a slow query into a full outage. Tracking the count of blocked queries and total lock wait time over time surfaces this pattern well before connection exhaustion actually happens.

3. Autovacuum debt

Dead tuple count, table bloat, and how far behind autovacuum has fallen on your busiest tables are leading indicators, not lagging ones. A table that's accumulating dead tuples faster than autovacuum can clean them up will eventually show up as degraded query plans, bloated indexes, and — in the worst case — transaction ID wraparound risk. All of this is visible well in advance if anyone is looking.

4. Query plan regressions

Comparing snapshots of pg_stat_statements over time — not just looking at the current top queries, but diffing against last week's — surfaces plan regressions before they become an incident. A query that suddenly needs a sequential scan where it used to use an index (often after a statistics update, a data distribution shift, or a parameter sniffing issue) shows up here long before it shows up as a page.

5. Connection saturation relative to pool limits

Raw connection count means little without the denominator. What matters is how close the active connection count is to the configured pool limit, and how quickly that gap is closing. This is one of the simplest metrics to track and one of the most commonly ignored, because "connections: 340" doesn't look alarming until you know the limit is 350.

The common thread across all five: none of them are useful as a single point-in-time value. They're useful as trends, tracked continuously, with alerts on rate-of-change rather than fixed thresholds. Paging on "disk 80% full" as a single absolute threshold is often already too late — paging on "disk usage growing 3x faster than its 30-day average" gives you time to act.

Alert fatigue is a design problem, not a tuning problem

Teams that alert on every metric crossing a static threshold tend to end up ignoring alerts altogether within a few months. The fix isn't fewer metrics — it's better-designed alerts: fewer absolute thresholds, more trend and rate-of-change detection, and clear escalation so a genuinely urgent signal doesn't get lost in the noise of a hundred low-priority ones.

Key takeaways

Monitoring Observability PostgreSQL Operations
Work with us

Still finding out about problems from your users first?

We build monitoring around the signals that actually predict trouble, not just the ones that are easy to graph. Let's look at what your dashboards are missing.

Get in touch More from the blog