Monitoring Redis Queue Health: Latency, Memory, and Throughput Dashboards

You just deployed BullMQ to production. Jobs are flowing. Workers are humming. Life is good.

Then your pager goes off at 2:47 PM on a Sunday afternoon. Redis hit its memory limit. Writes are failing. Your entire queue pipeline is stalled, and you're scrambling to figure out what happened while jobs pile up into the hundreds of thousands.

If this scenario sounds familiar, you're not alone. Redis is the backbone of every BullMQ and BeeQueue deployment, but it's also the single most common source of production failures in queue-based architectures. The problem isn't that Redis is unreliable — it's that Redis health is invisible until something breaks.

This post covers the three pillars of Redis queue health monitoring — latency, memory, and throughput — with practical code, dashboard designs, and alert thresholds you can implement today. We'll cover Redis-native tools (INFO, SLOWLOG, LATENCY MONITOR), BullMQ-specific metrics, and how to build effective monitoring dashboards.

Section 1: The Three Pillars of Redis Queue Health

A. Latency — The Silent Killer

Redis is single-threaded. This is its greatest strength (no lock contention, atomic operations, predictable performance) and its greatest vulnerability: one slow command blocks everything.

When a slow command hits your Redis instance, every subsequent operation — every job enqueue, every status check, every worker poll — queues up behind it. The symptom is a gradual, system-wide slowdown that's hard to pinpoint because individual commands look fast in isolation.

Common Latency Sources in Queue Workloads

• BullMQ stall detection: Workers periodically check if jobs are stalled by querying Redis. At scale, thousands of these checks compound.
• Large job payloads: A single 10 MB job payload stored as a hash field can take milliseconds to serialize and transfer, blocking smaller operations.
• Queue operations at scale: lpush, brpoplpush, zadd, and zrem on queues with millions of entries degrade over time.
• Lua EVALSHA scripts: BullMQ uses Lua scripts for atomic operations (moving jobs between states, rate limiting). Heavy script execution starves the event loop.

Metrics to Track

• Command latency: p50, p95, and p99 for each command type
• SLOWLOG entries: Commands exceeding the slowlog threshold
• Latency events: Redis-native latency monitor events (fork, command, fast-command)
• Worker processing time: Time spent in worker process callbacks (high values contribute to backpressure)

Polling Redis SLOWLOG with ioredis

Here's a practical script that polls SLOWLOG every 30 seconds and alerts on slow commands:

import Redis from "ioredis";

const redis = new Redis({
  host: process.env.REDIS_HOST || "localhost",
  port: Number(process.env.REDIS_PORT) || 6379,
});

// Configure SLOWLOG threshold — log any command taking > 100ms
await redis.config("SET", "slowlog-log-slower-than", "100000"); // microseconds

async function pollSlowlog() {
  const entries = await redis.slowlog("GET", 20);
  const criticalEntries = entries.filter(
    (entry: any) => entry[2] > 1_000_000 // > 1 second
  );

  if (criticalEntries.length > 0) {
    console.warn("⚠️ Critical slow commands detected:", {
      count: criticalEntries.length,
      entries: criticalEntries.map((e: any) => ({
        command: e[3].join(" ").substring(0, 200),
        duration: `${(e[2] / 1000).toFixed(0)}ms`,
        timestamp: new Date(e[1] * 1000).toISOString(),
      })),
    });
  }

  console.log(
    `SLOWLOG: ${entries.length} recent entries, ${criticalEntries.length} critical`
  );
}

// Poll every 30 seconds
setInterval(pollSlowlog, 30_000);

Alert Thresholds for Latency

Metric	Investigate	Critical	Immediate Action
P99 command latency	> 50ms	> 200ms	Check SLOWLOG for offending command
SLOWLOG entries (last 5 min)	> 5	> 20	Optimize heavy queries or split queues
Individual SLOWLOG duration	> 500ms	> 1s	Review Lua script or payload size
Worker processing time	> 500ms (p95)	> 2s (p95)	Profile worker code, consider concurrency

B. Memory — The Most Common Failure Mode

Every BullMQ job you ever enqueue lives in Redis until it's completed and cleaned. This is the fundamental reality that makes memory the most critical health metric for queue operations. When Redis runs out of memory, it doesn't gracefully degrade — it either crashes writes (with noeviction) or starts evicting queue data (with any eviction policy).

The Eviction Policy Trap

Redis's maxmemory-policy has profound implications for queue workloads:

• noeviction (default in some configurations): Writes fail outright with OOM errors. BullMQ stops enqueuing jobs. Workers can't update job status. Your queue freezes.
• allkeys-lru: Redis evicts any key, including queue metadata and job data. Jobs disappear silently.
• volatile-lru: Redis evicts only keys with TTL set. But BullMQ doesn't set TTL on most job keys, so this effectively does nothing until eviction pressure is extreme.
• allkeys-random: Same problem as lru — random job loss.

Recommendation: Use allkeys-lru if you can tolerate rare job evictions (and monitor evicted_keys religiously). Use noeviction with aggressive monitoring and a memory buffer if job loss is unacceptable.

Key Redis INFO Memory Fields

When you run redis-cli INFO memory, focus on these fields:

Field	What It Tells You
used_memory	Bytes Redis has allocated (the "working set")
used_memory_rss	Actual physical memory Redis is using (RSS)
mem_fragmentation_ratio	used_memory_rss / used_memory — measures allocator efficiency
maxmemory	Configured memory limit (0 = no limit)
evicted_keys	Keys evicted since Redis started
total_system_memory	Host's total RAM

Polling Redis Memory Metrics

import Redis from "ioredis";

interface RedisMemoryInfo {
  used_memory: number;
  used_memory_rss: number;
  mem_fragmentation_ratio: number;
  maxmemory: number;
  evicted_keys: number;
  total_system_memory: number;
}

async function getMemoryMetrics(redis: Redis): Promise<RedisMemoryInfo> {
  const info = await redis.info("memory");
  const parse = (key: string): number => {
    const match = info.match(new RegExp(`^${key}:(\d+)`, "m"));
    return match ? parseInt(match[1], 10) : 0;
  };

  return {
    used_memory: parse("used_memory"),
    used_memory_rss: parse("used_memory_rss"),
    mem_fragmentation_ratio: parse("mem_fragmentation_ratio") / 100,
    maxmemory: parse("maxmemory"),
    evicted_keys: parse("evicted_keys"),
    total_system_memory: parse("total_system_memory"),
  };
}

async function checkMemory(redis: Redis) {
  const mem = await getMemoryMetrics(redis);
  const usagePercent = mem.maxmemory > 0
    ? (mem.used_memory / mem.maxmemory) * 100
    : (mem.used_memory / mem.total_system_memory) * 100;

  const alerts: string[] = [];

  if (usagePercent > 80) {
    alerts.push(`🔴 Memory usage at ${usagePercent.toFixed(1)}% — investigate`);
  }
  if (usagePercent > 90) {
    alerts.push(`🚨 CRITICAL: Memory usage at ${usagePercent.toFixed(1)}% — take action`);
  }
  if (mem.mem_fragmentation_ratio < 1.0) {
    alerts.push(`⚠️ RSS < used_memory — Redis may be swapping!`);
  }
  if (mem.mem_fragmentation_ratio > 1.5) {
    alerts.push(`⚠️ High fragmentation ratio: ${mem.mem_fragmentation_ratio.toFixed(2)}`);
  }
  if (mem.evicted_keys > 0) {
    const prevEvicted = mem.evicted_keys; // Compare with last poll
    alerts.push(`⚠️ ${mem.evicted_keys} keys evicted since startup`);
  }

  if (alerts.length > 0) {
    alerts.forEach((a) => console.warn(a));
  }
}

setInterval(() => checkMemory(redis), 30_000);

Alerting on Critical Memory Conditions

Condition	Severity	Action
Fragmentation < 1.0	Critical	Redis is swapping to disk; restart or increase RAM
Memory usage > 80%	Warning	Plan capacity increase
Memory usage > 90%	Critical	Immediate cleanup or scale up
Fragmentation > 1.5	Warning	Enable activedefrag yes
Fragmentation > 2.0	Critical	Restart Redis to free fragmentation
evicted_keys > 0/hr	Warning	Check policy, increase maxmemory
evicted_keys > 100/hr	Critical	Jobs being lost; add capacity now

Memory Management Strategies for Queue Workloads

1. Clean completed jobs aggressively — Set removeOnComplete and removeOnFail retention limits:

   const queue = new Queue("orders", {
     defaultJobOptions: {
       removeOnComplete: { age: 3600, count: 1000 },  // keep 1 hour or 1000
       removeOnFail: { age: 86400, count: 100 },       // keep failures 24h
     },
   });
   

2. Enable activedefrag in redis.conf:

   activedefrag yes
   active-defrag-threshold-lower 10
   active-defrag-threshold-upper 100
   active-defrag-cycle-min 1
   active-defrag-cycle-max 25
   

3. Right-size maxmemory — leave 30-50% headroom for BGSAVE (copy-on-write doubles RSS during saves).

4. Monitor fragmentation — high fragmentation means the allocator is struggling. Restarting Redis during low-traffic windows resets it.

C. Throughput — Is Your Queue Keeping Up?

Latency and memory tell you how Redis is performing. Throughput tells you whether your queue is keeping up with demand.

The Three Throughput Vectors

1. Arrival rate: How fast jobs are being enqueued (jobs/second)
2. Processing rate: How fast workers are completing jobs (jobs/second)
3. Queue depth: The backlog of waiting and delayed jobs

When arrival rate exceeds processing rate over sustained periods, queue depth grows unbounded, memory fills, and eventually the system fails. Throughput monitoring lets you see this in slow motion and act before the pager goes off.

BullMQ-Specific Throughput Metrics

BullMQ exposes built-in metrics that are essential for throughput monitoring:

import { Queue, MetricsTime, Worker } from "bullmq";

// Enable metrics when creating a queue
const queue = new Queue("email-notifications", {
  metrics: {
    maxDataPoints: MetricsTime.ONE_WEEK * 2, // Keep 2 weeks of data
  },
});

// Poll job counts every 30 seconds
async function pollQueueMetrics() {
  const [jobCounts, metrics] = await Promise.all([
    queue.getJobCounts(
      "waiting",
      "active",
      "completed",
      "failed",
      "delayed"
    ),
    queue.getMetrics("completed"),
    queue.getMetrics("failed"),
  ]);

  const totalPending = jobCounts.waiting + jobCounts.delayed;
  const completionRate = metrics.data.length > 1
    ? (metrics.data[metrics.data.length - 1].count -
        metrics.data[metrics.data.length - 2].count) /
      (metrics.data[metrics.data.length - 1].timestamp -
        metrics.data[metrics.data.length - 2].timestamp) *
      1000
    : 0;

  console.log({
    waiting: jobCounts.waiting,
    active: jobCounts.active,
    failed: jobCounts.failed,
    delayed: jobCounts.delayed,
    pending: totalPending,
    completionRate: `${completionRate.toFixed(1)} jobs/s`,
    failureRate: `${((jobCounts.failed / (jobCounts.completed || 1)) * 100).toFixed(1)}%`,
  });
}

setInterval(pollQueueMetrics, 30_000);

Direct Redis Key Inspection

For environments where BullMQ's getJobCounts isn't fast enough, you can inspect Redis keys directly:

async function directKeyCounts(redis: Redis, queueName: string) {
  const prefix = `bull:${queueName}`;
  const [waiting, active, completed, failed, delayed] = await Promise.all([
    redis.llen(`${prefix}:wait`),
    redis.llen(`${prefix}:active`),
    redis.scard(`${prefix}:completed`),
    redis.scard(`${prefix}:failed`),
    redis.zcard(`${prefix}:delayed`),
  ]);

  return { waiting, active, completed, failed, delayed };
}

Throughput Dashboard Layout Concept

┌──────────────────────────────────────────────────────────────┐
│  Throughput (last 1 hour)                    ┌────────────┐  │
│  ┌────────────────────────────────────────┐  │ Processing │  │
│  │ ▁▂▃▄▅▆▇█▇▆▅▄▃▂▁ │  │ Time: 142ms│  │
│  │ Arrival ••••• Processing —             │  │ (p50 avg)  │  │
│  └────────────────────────────────────────┘  └────────────┘  │
│                                                              │
│  ┌───────────┐ ┌───────────┐ ┌───────────┐ ┌───────────┐   │
│  │ Waiting   │ │ Active    │ │ Failed    │ │ Slowest   │   │
│  │   2,341   │ │      12   │ │      47   │ │  order-83 │   │
│  │  ↑ 12%    │ │  = 100%   │ │  = 4.7%   │ │  12.4s    │   │
│  └───────────┘ └───────────┘ └───────────┘ └───────────┘   │
└──────────────────────────────────────────────────────────────┘

Section 2: Redis-Native Monitoring Tools

Redis ships with a rich set of introspection tools. Here's the cheat sheet for queue operators:

INFO Command

# Get everything
redis-cli INFO

# Or specific sections relevant to queues
redis-cli INFO memory     # Memory metrics
redis-cli INFO stats      # Total connections, commands processed
redis-cli INFO clients    # Connected clients, max clients
redis-cli INFO commandstats  # Per-command breakdown
redis-cli INFO keyspace   # Key counts per database
redis-cli INFO persistence  # Last save, RDB/AOF status

The commandstats section is especially valuable — it shows how many times each command was called and the total CPU time consumed. If EVALSHA or BRPOPLPUSH top the list, your BullMQ queue is the primary load driver.

SLOWLOG

# Configure threshold (in microseconds)
redis-cli CONFIG SET slowlog-log-slower-than 100000

# Get the last 10 slow commands
redis-cli SLOWLOG GET 10

# Count total slow entries
redis-cli SLOWLOG LEN

# Reset the log
redis-cli SLOWLOG RESET

In queue workloads, pay special attention when BullMQ-related Lua EVALSHA calls appear in the SLOWLOG — this indicates the queue is under heavy load and scripts are taking noticeable time.

LATENCY Family

# Enable latency monitoring (in milliseconds)
redis-cli CONFIG SET latency-monitor-threshold 100

# View the latest latency spike per event
redis-cli LATENCY LATEST

# Dump history for a specific event
redis-cli LATENCY HISTORY command

# Get a human-readable diagnosis
redis-cli LATENCY DOCTOR

CLIENT LIST

redis-cli CLIENT LIST

Watch the omem field (output buffer memory). A worker with omem > 10 MB may have a stalled connection consuming Redis memory. Use CLIENT KILL to terminate stuck workers:

redis-cli CLIENT KILL TYPE normal
redis-cli CLIENT KILL ADDR 10.0.0.1:6379

⚠️ MONITOR Warning

Never run redis-cli MONITOR in production — it reduces throughput by approximately 50% because it outputs every command to the client. Use it only on isolated replica instances for short debugging sessions.

MEMORY USAGE

# Check how much memory a specific key uses
redis-cli MEMORY USAGE bull:email:83

# Compare across queues
redis-cli --scan --pattern 'bull:*:meta' |   xargs -I{} redis-cli MEMORY USAGE {}

Section 3: BullMQ-Specific Monitoring Deep Dive

BullMQ provides a solid metrics API, but you need to configure it and know where to look.

Enabling Built-in Metrics

import { Queue, MetricsTime } from "bullmq";

const queue = new Queue("orders", {
  metrics: {
    maxDataPoints: MetricsTime.ONE_WEEK * 4, // 4 weeks retention
  },
});

Once enabled, getMetrics("completed") and getMetrics("failed") return time-series data you can chart directly.

Interpreting Job Counts

The job count snapshot tells a story:

• waiting + delayed rising: Backlog is growing — you need more workers or faster processing.
• active == concurrency × workers: All capacity is saturated — any new job goes straight to the waiting queue.
• active > capacity: Stalled jobs — workers that grabbed a job but never completed it. Check for worker crashes.
• failed > 5%: Something is systematically failing — investigate error types.
• delayed growing: Jobs are being retried with backoff faster than they're being consumed.

Worker Health Monitoring

import { Worker } from "bullmq";

// A worker with telemetry-aware configuration
const worker = new Worker("orders", async (job) => {
  // Your processing logic here
  return processOrder(job.data);
}, {
  concurrency: 10,
  stallInterval: 30000,   // Default 30s — tune to your job duration
  lockDuration: 60000,    // How long a job can be 'in progress'
});

// Monitor worker events
worker.on("completed", (job) => {
  // Track per-job completion time
});

worker.on("failed", (job, err) => {
  console.error(`Job ${job.id} failed: ${err.message}`);
});

worker.on("stalled", (jobId) => {
  console.warn(`Job ${jobId} stalled — another worker may pick it up`);
});

Aggregating Across Multiple Queues

Most production systems have multiple queues. Here's how to discover and monitor all of them:

async function discoverAllQueues(redis: Redis): Promise<string[]> {
  // BullMQ stores queue metadata under bull:{name}:meta
  const metaKeys = await redis.keys("bull:*:meta");
  return metaKeys.map((key) => key.split(":")[1]);
}

async function monitorAllQueues(redis: Redis) {
  const queueNames = await discoverAllQueues(redis);

  const results = await Promise.all(
    queueNames.map(async (name) => {
      const queue = new Queue(name, { connection: redis });
      const counts = await queue.getJobCounts();
      return { name, ...counts };
    })
  );

  console.table(results);
}

Section 4: Building Effective Dashboards

A good monitoring dashboard gives you three things at a glance: current health, recent trends, and alert-worthy anomalies.

Dashboard 1: Redis Health

Metric	Widget Type	Refresh	Source
Memory usage	Gauge (%, colored)	30s	INFO memory
Fragmentation ratio	Gauge (1.0 = green, >1.5 = yellow, <1.0 = red)	30s	INFO memory
Evicted keys	Counter (delta per hour)	30s	INFO memory
Connected clients	Number + trend line	30s	INFO clients
Command throughput	Sparkline (ops/sec)	30s	INFO stats - instantaneous_ops_per_sec
SLOWLOG count	Badge (0 green, <5 yellow, >5 red)	30s	SLOWLOG LEN
Keyspace	Table per DB (keys, expires, avg_ttl)	60s	INFO keyspace

Dashboard 2: Queue Performance

Metric	Widget Type	Refresh	Source
Job counts	Summary cards (waiting/active/failed/delayed)	30s	getJobCounts
Throughput	Area chart (arrival vs completion rate)	30s	getMetrics
Processing time	Line chart (p50, p95, p99)	30s	Worker events
Failure rate	Percentage + trend	30s	failed / completed
Slowest jobs	Table (top 5 by duration)	60s	Worker 'completed' events
Failing job types	Table (count by error/type)	60s	Worker 'failed' events
Queue depth	Sparkline (waiting + delayed over last hour)	30s	getJobCounts

Dashboard 3: Worker Health

Metric	Widget Type	Refresh	Source
Active workers	Count + badge	30s	getWorkers()
Worker heartbeats	Timeline (last seen per worker)	30s	Worker events
Active jobs per worker	Horizontal bar chart	30s	getWorkers()/Worker events
Stalled jobs	Counter (delta per hour)	30s	Worker 'stalled' events

How QueueHub Implements These

QueueHub brings all three dashboards together in a single interface. Under the hood, it:

• Polls every 30 seconds — a lightweight setInterval loop calls INFO, SLOWLOG, getJobCounts, and getMetrics in parallel
• Redis Info Card — shows memory, fragmentation, connected clients, uptime, and version at a glance
• Metric Cards — job counts with colored trend indicators (green = stable, yellow = rising, red = critical)
• Throughput / Processing Time charts — time-series visualizations built from BullMQ's getMetrics data
• Slowest Jobs / Failing Jobs tables — live-sorted from worker event emissions on the server
• Live Workers view — shows active workers, their current jobs, and heartbeat timestamps

All of this connects to any Redis instance — local, TLS, Valkey, ElastiCache, or via SSH tunnel — with no code changes required.

Alerting Thresholds Reference

Metric	Warning	Critical	Action
Redis memory usage	> 75%	> 90%	Clean completed jobs, scale up, increase maxmemory
mem_fragmentation_ratio	> 1.5	> 2.0 or < 1.0	Enable activedefrag; restart if < 1.0 (swapping)
evicted_keys	> 0/hr	> 100/hr	Change eviction policy, add capacity
Slowlog count (5 min)	> 5	> 20	Optimize queries, check Lua scripts
P99 command latency	> 50ms	> 200ms	Check SLOWLOG, split busy queues
Failed job rate	> 5%	> 20%	Check error types, review worker code
Waiting + delayed jobs	Rising for 1 hr	Doubled in 1 hr	Scale workers, optimize processing
Stalled jobs	> 1/hr	> 10/hr	Check worker health, network latency
Connected clients	> 80% maxclients	> 95% maxclients	Check for connection leaks, restart workers

Production Lessons from the Trenches

These aren't theoretical — they're hard-won lessons from real production incidents:

1. noeviction will crash your queues without warning. Set an appropriate maxmemory and monitor usage. Better yet, use allkeys-lru with a safety cushion and track evicted_keys.

2. BGSAVE doubles RSS via copy-on-write. If your Redis data set is 4 GB and you have a 10 GB maxmemory, a BGSAVE can push RSS to 8+ GB. Always leave 30-50% headroom above your working set.

3. Clean completed jobs aggressively or they'll fill your memory. Default BullMQ retention is unlimited. Always set removeOnComplete and removeOnFail:

// Smart defaults for production
const queue = new Queue("orders", {
  defaultJobOptions: {
    removeOnComplete: { age: 3600 * 24, count: 10000 },  // 24h or 10k
    removeOnFail: { age: 3600 * 24 * 7, count: 5000 },    // 7 days
  },
});

4. stallInterval matters. Tune it to your job duration. If your jobs average 5 seconds, a 30-second stall interval is fine. If jobs take 2 minutes, increase it to avoid false stall detections and unnecessary overhead.

5. Monitor omem in CLIENT LIST. High output-buffer memory indicates a slow consumer. Kill and reconnect stuck workers before they cause an OOM.

6. BullMQ Lua EVALSHA in SLOWLOG means heavy load. These scripts are fast by design — if they're showing up in slowlog, your queue is operating near its limit.

7. MONITOR is a production antipattern. It reduces throughput by ~50%. Never enable it on a production instance. Use redis-benchmark or INFO commandstats for profiling instead.

Conclusion: Three Signals, One Dashboard

Redis queue health comes down to three signals: latency, memory, and throughput. Redis gives you the primitives (INFO, SLOWLOG, LATENCY DOCTOR). BullMQ adds queue-aware metrics (getJobCounts, getMetrics, worker events). But stitching them together into actionable dashboards takes significant engineering effort — effort that's easy to deprioritize until the Sunday afternoon pager call.

That's where QueueHub comes in. QueueHub is a SaaS dashboard designed specifically for BullMQ and BeeQueue operators. It connects to any Redis instance — local, TLS-protected, Valkey-native, ElastiCache, or behind an SSH tunnel — and gives you:

• Real-time Redis health monitoring (memory, fragmentation, evictions, clients)
• Per-queue performance dashboards (throughput, processing time, failure rates)
• Worker health visibility (active workers, stalled jobs, heartbeats)
• Live tables of your slowest and most frequently failing jobs
• Alert-ready thresholds with trend indicators

Don't wait for the next production incident. Try QueueHub for your BullMQ or BeeQueue queues and get full Redis queue visibility in minutes — no code changes, no agents, no configuration headaches.