Monitoring Redis Queue Health: Latency, Memory & Throughput Guide

Your Redis instance has been silently losing memory for three hours. Jobs are failing. Workers are stalling. Your first sign of trouble? A customer complaint.

If this scenario sounds familiar, you are not alone. Redis powers every BullMQ and BeeQueue deployment, yet its health remains invisible until something breaks. Most teams monitor application-level metrics -- CPU, memory, request rates -- but miss the Redis layer entirely until production goes down.

Redis queue health boils down to three critical dimensions:

• Latency -- blocked commands starving workers
• Memory -- OOM failures or silent data loss from eviction
• Throughput -- backlog growing unbounded when arrival rate exceeds processing rate

In this guide, we will cover Redis-native diagnostic tools like INFO, LATENCY, and SLOWLOG, show you how to analyze memory usage by queue, walk through BullMQ-specific monitoring with getJobCounts() and getMetrics(), and survey the tooling landscape from redis-cli to Queue Hub (QueueHub). By the end, you will have a production-ready monitoring strategy for your Redis-backed queues.

TL;DR for skimmers:

• Never use MONITOR in production (50% throughput loss)
• Always set maxmemory-policy noeviction for queue Redis instances
• Track used_memory, instantaneous_ops_per_sec, and evicted_keys from INFO
• BullMQ's getJobCounts() + getMetrics() give you job-level throughput for free
• A good dashboard needs three views: Redis health, queue performance, worker status

---

The Three Pillars of Queue Health

Latency -- The Silent Killer

Redis is single-threaded. One slow command blocks every other command until it finishes. In a queue system, this means workers cannot heartbeat, jobs cannot be fetched, and your entire pipeline grinds to a halt.

Common sources of queue latency include:

• Large job payloads (10MB+ serialization) that saturate the network buffer
• Lua EVALSHA scripts under load -- BullMQ uses these extensively for atomic operations
• BullMQ stall-detection polling at scale, especially with many queues on one Redis instance
• BRPOPLPUSH on queues with millions of entries

The insidious thing about latency is that individual commands look fast in isolation, but queuing effects compound. A 5ms command under load can become a 500ms command when the command queue backs up.

How to Measure Latency

redis-cli --latency -- your first diagnostic:

redis-cli --latency -h myredis.example.com -p 6379
# Output: min: 0ms, max: 87ms, avg: 1.23ms (5611 samples)

This shows real-time round-trip time from client to server. Crucially, run this from the same network as your workers, not your laptop. Expected values: < 2ms on same-region, < 10ms cross-region.

redis-cli --intrinsic-latency -- the floor you cannot go below:

redis-cli --intrinsic-latency 100
# Run ON the Redis server -- measures OS/hypervisor baseline noise

Virtualized hosts (EC2 Xen, Nitro, etc.) typically show 5--40ms of intrinsic latency; physical hosts are usually under 200us. If your measured latency is close to the intrinsic latency, it is a network or OS problem, not a Redis problem.

LATENCY family (Redis 2.8.13+):

# Enable -- log any latency spike > 100ms
redis-cli CONFIG SET latency-monitor-threshold 100

# Check what is happening
redis-cli LATENCY LATEST
redis-cli LATENCY HISTORY command
redis-cli LATENCY DOCTOR   # Human-readable diagnosis

SLOWLOG -- find the offending commands:

redis-cli CONFIG SET slowlog-log-slower-than 10000   # 10ms threshold
redis-cli SLOWLOG GET 20
redis-cli SLOWLOG LEN

Polling SLOWLOG programmatically with ioredis:

import Redis from "ioredis";

const redis = new Redis({ host: process.env.REDIS_HOST, port: 6379 });

// Set threshold to 100ms
await redis.config("SET", "slowlog-log-slower-than", "100000");

async function pollSlowlog() {
  const entries = await redis.slowlog("GET", 20);
  const critical = entries.filter((e: any) => e[2] > 1_000_000); // >1s

  if (critical.length) {
    console.warn(`Ⓒ ${critical.length} critical slow commands:`,
      critical.map((e: any) => ({
        cmd: e[3].join(" ").substring(0, 200),
        duration: `${(e[2] / 1000).toFixed(0)}ms`,
        time: new Date(e[1] * 1000).toISOString(),
      }))
    );
  }
}

setInterval(pollSlowlog, 30_000);

The MONITOR Command -- Why It Is Dangerous

redis-cli MONITOR   # NEVER do this in production

MONITOR streams every single command to the client. It reduces Redis throughput by approximately 50% because the server spends CPU cycles formatting and sending output for every command. The only legitimate use case is an isolated replica during brief debugging sessions (under 60 seconds).

Safe alternative -- INFO commandstats:

redis-cli INFO commandstats
# cmdstat_get:calls=1234567,usec=8123456,usec_per_cmd=6.59
# cmdstat_evalsha:calls=98765,usec=1234567,usec_per_cmd=12.50

Watch for EVALSHA (BullMQ Lua scripts) and BRPOPLPUSH topping the list.

BullMQ-Specific Latency Signals

• Worker stalled event frequency spikes
• High lockDuration misses -> jobs being re-processed unnecessarily
• Queue active count > worker concurrency -> stalled or frozen jobs

Key insight: A stalled job is almost always a latency symptom -- the worker could not heartbeat to Redis in time. If you see stalled jobs climbing, check Redis latency before debugging your worker code.

---

Memory -- The Most Common Failure Mode

Every BullMQ job lives in RAM until explicitly cleaned. When Redis hits maxmemory, writes either fail (under noeviction) or data evaporates (under any eviction policy). For queue workloads, the queue data is the largest, most critical dataset in the instance.

Key INFO Memory Fields

redis-cli INFO memory

Field	What It Tells You	Alarm Threshold
used_memory	Bytes allocated by Redis	>80% maxmemory
used_memory_rss	Actual physical RAM used	Compare to used_memory
mem_fragmentation_ratio	RSS/used_memory efficiency	<1.0 (swapping!) or >1.5
maxmemory	Configured limit (0 = unlimited)	Should never be 0
evicted_keys	Keys evicted since startup	>0/hr is warning
total_system_memory	Host total RAM	Used for headroom calc

Polling memory metrics programmatically:

interface MemoryInfo {
  used_memory: number;
  used_memory_rss: number;
  maxmemory: number;
  mem_fragmentation_ratio: number;
  evicted_keys: number;
  total_system_memory: number;
}

async function getMemoryMetrics(redis: Redis): Promise<MemoryInfo> {
  const info = await redis.info("memory");
  const parse = (key: string): number => {
    const m = info.match(new RegExp(`^${key}:(\d+)`, "m"));
    return m ? parseInt(m[1], 10) : 0;
  };
  return {
    used_memory: parse("used_memory"),
    used_memory_rss: parse("used_memory_rss"),
    maxmemory: parse("maxmemory"),
    mem_fragmentation_ratio: parse("mem_fragmentation_ratio") / 100,
    evicted_keys: parse("evicted_keys"),
    total_system_memory: parse("total_system_memory"),
  };
}

Memory Usage by Queue -- Counting Keys

BullMQ stores queue data under the bull:{queueName}:* key pattern. Different job states use different Redis data structures:

BullMQ Key	Data Type	Purpose
bull:{name}:wait	LIST	Waiting jobs (FIFO)
bull:{name}:active	LIST	Currently processing
bull:{name}:completed	SET	Recently completed job IDs
bull:{name}:failed	SET	Recently failed job IDs
bull:{name}:delayed	ZSET	Delayed jobs (by timestamp)
bull:{name}:{jobId}	HASH	Individual job data + state
bull:{name}:meta	HASH	Queue metadata/version

Direct key inspection by type:

async function inspectQueueKeys(redis: Redis, queueName: string) {
  const prefix = `bull:${queueName}`;
  const [waitLen, activeLen, completedCard, failedCard, delayedCard] =
    await Promise.all([
      redis.llen(`${prefix}:wait`),
      redis.llen(`${prefix}:active`),
      redis.scard(`${prefix}:completed`),
      redis.scard(`${prefix}:failed`),
      redis.zcard(`${prefix}:delayed`),
    ]);
  return { waitLen, activeLen, completedCard, failedCard, delayedCard };
}

Finding stale jobs -- checking TTLs:

BullMQ does not set TTL on most job keys. Jobs with TTLs are typically delayed or rate-limited jobs only:

# Scan for queue job keys and check their TTL
redis-cli --scan --pattern 'bull:*' | head -100 | while read key; do
  ttl=$(redis-cli ttl "$key")
  mem=$(redis-cli memory usage "$key")
  echo "$key | TTL: $ttl | Memory: $mem bytes"
done

Memory-expensive patterns to watch for:

• Completed jobs without removeOnComplete -- the single most common bloat source
• Failed jobs without removeOnFail -- accumulates silently over time
• Large job payloads in HASH fields -- a single 10MB job can block memory for the entire queue
• High-cardinality completed or failed sets -- sets of job IDs that never expire

The Eviction Policy Trap

BullMQ's documentation is unequivocal: use noeviction.

# redis.conf
maxmemory 2gb
maxmemory-policy noeviction

Policy	Effect on Queues	Verdict
noeviction	Writes fail with OOM -- queue freezes, no data loss	Required
allkeys-lru	Redis evicts ANY key -- jobs disappear silently	Data loss
volatile-lru	Only keys with TTL -- BullMQ sets few TTLs, so it is ineffective	False safety
allkeys-random	Random job loss	Data loss
volatile-ttl	Evicts shortest TTL first -- same problem	Unsafe

Workaround if you cannot use noeviction: Set removeOnComplete aggressively, increase maxmemory, and monitor evicted_keys religiously.

Best Practice: Job Retention Configuration

const queue = new Queue("orders", {
  connection,
  defaultJobOptions: {
    removeOnComplete: { age: 3600 * 24, count: 1000 },  // 24h or 1000
    removeOnFail: { age: 3600 * 24 * 7, count: 5000 },  // 7 days
  },
});

Memory headroom rule: Always leave 30--50% headroom above your working set for BGSAVE copy-on-write. If your working set is 2GB, set maxmemory to at least 3--4GB.

Enable activedefrag:

activedefrag yes
active-defrag-threshold-lower 10
active-defrag-threshold-upper 100
active-defrag-cycle-min 1
active-defrag-cycle-max 25

---

Throughput -- Is Your Queue Keeping Up?

Throughput monitoring has three vectors: arrival rate (enqueues per second), processing rate (completions per second), and backlog (waiting + delayed jobs). When arrival exceeds processing over sustained periods, queue depth grows unbounded and memory fills.

BullMQ Built-in Metrics

Enable metrics on the Worker:

import { Worker, MetricsTime } from "bullmq";

const worker = new Worker("email", async (job) => {
  await sendEmail(job.data);
}, {
  connection,
  metrics: {
    maxDataPoints: MetricsTime.ONE_WEEK * 2,  // 2 weeks of 1-min buckets
  },
});

Query via the Queue:

const metrics = await queue.getMetrics("completed", 0, -1);
// {
//   data: number[],     // jobs completed per minute
//   count: number,      // total completed
//   meta: { count, prevTS, prevCount }
// }

Job Counts Dashboard

async function pollQueueMetrics(queue: Queue) {
  const jobCounts = await queue.getJobCounts(
    "waiting", "active", "completed", "failed", "delayed", "paused"
  );

  const waitingRising = jobCounts.waiting > 1000;  // Example threshold
  const failureRate = (jobCounts.failed / (jobCounts.completed || 1)) * 100;

  console.log({
    waiting: jobCounts.waiting,
    active: jobCounts.active,
    completed: jobCounts.completed,
    failed: jobCounts.failed,
    failureRate: `${failureRate.toFixed(1)}%`,
    healthy: failureRate < 5 && !waitingRising,
  });
}

Interpreting Job Count Patterns

Signal	What It Means
waiting + delayed rising	Backlog growing -- need more workers or faster processing
active == concurrency * workers	All capacity saturated
active > capacity	Stalled jobs -- check for worker crashes
failed > 5%	Systematic failure -- investigate error types
delayed growing	Retry backoff consuming jobs slower than failing

Prometheus Export

BullMQ provides a built-in Prometheus metrics endpoint:

import http from "http";
import { Queue } from "bullmq";

const queue = new Queue("my-queue", { connection });
const server = http.createServer(async (req, res) => {
  if (req.url === "/metrics") {
    const metrics = await queue.exportPrometheusMetrics({
      env: "production",
    });
    res.writeHead(200, { "Content-Type": "text/plain" });
    res.end(metrics);
  }
});
server.listen(3000);

Sample Prometheus output:

# HELP bullmq_job_count Number of jobs in the queue by state
# TYPE bullmq_job_count gauge
bullmq_job_count{queue="my-queue",state="waiting"} 5
bullmq_job_count{queue="my-queue",state="active"} 3
bullmq_job_count{queue="my-queue",state="completed"} 12
bullmq_job_count{queue="my-queue",state="failed"} 2

---

Redis-Native Monitoring Tools -- Cheat Sheet

INFO Command -- Sections Worth Memorizing

redis-cli INFO memory       # Memory usage, fragmentation, evictions
redis-cli INFO stats        # connected_clients, total_commands_processed, instantaneous_ops_per_sec
redis-cli INFO clients      # connected_clients, maxclients, blocked_clients
redis-cli INFO commandstats # Per-command call count + CPU time
redis-cli INFO keyspace     # Per-DB key count + TTL stats
redis-cli INFO persistence  # Last BGSAVE, AOF status, RDB changes since last save
redis-cli INFO replication  # Role (master/slave), lag, connected replicas
redis-cli INFO cpu          # CPU usage by Redis process
redis-cli INFO server       # Redis version, uptime, mode

Hit/miss ratio calculation:

redis-cli INFO stats | grep -E "keyspace_(hits|misses)"
# Ratio = keyspace_hits / (keyspace_hits + keyspace_misses)
# >0.8 = healthy cache, but queues are write-heavy -- expect lower

CLIENT LIST -- Watch for Stalled Connections

redis-cli CLIENT LIST
# id=1234 addr=10.0.0.5:54321 fd=8 name= age=1423 idle=0 ...

Key field: omem -- output buffer memory in bytes. A worker with omem > 10MB may have a stalled connection eating Redis RAM. Kill stuck clients with redis-cli CLIENT KILL TYPE normal.

MEMORY USAGE -- Per-Key Memory Profiling

redis-cli MEMORY USAGE bull:email:83
# Returns bytes for a single key

# Scan largest keys by type
redis-cli --bigkeys
redis-cli --memkeys

---

BullMQ-Specific Monitoring Deep Dive

Auto-Detecting Queues

async function discoverQueues(redis: Redis): Promise<string[]> {
  const keys = await redis.keys("bull:*:meta");
  return keys.map((k: string) => k.split(":")[1]);
}

Aggregated Queue Health

async function monitorAllQueues(redis: Redis) {
  const names = await discoverQueues(redis);
  const results = await Promise.all(
    names.map(async (name) => {
      const q = new Queue(name, { connection: redis });
      const counts = await q.getJobCounts();
      return { name, ...counts };
    })
  );
  console.table(results);
}

Worker Health -- Heartbeats and Stalls

const worker = new Worker("orders", async (job) => process(job), {
  connection,
  concurrency: 10,
  lockDuration: 60000,     // Max time a job can be "active"
  stallInterval: 30000,    // How often to check for stalled jobs
});

worker.on("completed", (job) => {
  // Track per-job completion for processing time histogram
});
worker.on("failed", (job, err) => {
  console.error(`Job ${job.id} failed: ${err.message}`);
});
worker.on("stalled", (jobId) => {
  console.warn(`Job ${jobId} stalled -- may be re-processed`);
});

How QueueHub uses this: The live workers view shows active workers, their current jobs, heartbeat timestamps, and stalled-job counters -- all driven by these events.

---

Tooling Landscape

Command-Line Tools

Tool	Use Case
redis-cli --latency	Real-time latency check from worker network
redis-cli --intrinsic-latency 100	OS/hypervisor baseline
redis-cli --bigkeys / --memkeys	Find memory hogs by data type
redis-cli --scan --pattern 'bull:*'	Iterate queue keys without blocking
redis-cli --stat	Live stats refresh (memory, ops/sec, clients)

RedisInsight (by Redis)

RedisInsight is a GUI for Redis featuring a key browser, memory analysis, and command profiling. It connects to local, remote, TLS, and ElastiCache instances. The PROFILER tab provides interactive SLOWLOG visualization with flame charts.

Limitation: RedisInsight is not queue-aware -- you see raw keys, not job counts or throughput metrics.

redis_exporter + Prometheus + Grafana

The redis_exporter (by Oliver Eilhard) exposes all INFO metrics as Prometheus format. Combined with Grafana, you get pre-built Redis dashboards and custom BullMQ panels.

# docker-compose snippet
services:
  redis_exporter:
    image: oliver006/redis_exporter:latest
    environment:
      - REDIS_ADDR=redis://my-redis:6379
    ports:
      - "9121:9121"

  prometheus:
    image: prom/prometheus:latest
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    ports:
      - "9090:9090"

  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"

Queue Hub / QueueHub

QueueHub is a SaaS dashboard purpose-built for BullMQ and BeeQueue. It connects to any Redis -- local, TLS, Valkey, AWS ElastiCache, or private (via agent tunneling) -- with no agents or configuration required. It provides three integrated views:

1. Redis health -- memory gauge, fragmentation, evictions, clients, ops/sec, SLOWLOG badge
2. Queue performance -- job count cards, throughput chart, processing time, failure rate
3. Worker health -- active workers, stalled jobs, heartbeats, per-worker job list

Key differentiator: queue-aware out of the box -- no Prometheus stack, no Grafana setup, no custom dashboards.

When to Use Which Tool

Scenario	Best Tool
Quick one-off latency check	redis-cli --latency
Deep memory profiling	redis-cli --bigkeys + MEMORY USAGE
GUI key browsing	RedisInsight
Long-term metrics + alerting	redis_exporter + Prometheus + Grafana
Daily queue operations	Queue Hub / QueueHub
Production incident triage	QueueHub + redis-cli

---

Best Practices and Alerting Thresholds

Eviction Policy -- No Compromise

Set maxmemory-policy noeviction. This is the single most important configuration for queue Redis instances.

# redis.conf -- production queue Redis
maxmemory 4gb
maxmemory-policy noeviction
activedefrag yes
save 900 1
save 300 10
save 60 10000
appendonly yes
appendfsync everysec

Memory Limit Sizing

1. Calculate your working set: max(completed + failed sets) + avg(queued jobs * payload size)
2. Add 30--50% headroom for BGSAVE copy-on-write
3. Example: 2GB working set -> 3--4GB maxmemory

Alerting Thresholds Reference

Metric	Warning	Critical	Action
Memory usage (% of maxmemory)	> 75%	> 90%	Clean completed jobs, scale up
mem_fragmentation_ratio	> 1.5	> 2.0 or < 1.0	Enable activedefrag; restart if < 1.0
evicted_keys (delta/hr)	> 0/hr	> 100/hr	Change eviction policy, add capacity
SLOWLOG entries (last 5 min)	> 5	> 20	Optimize queries, review Lua scripts
P99 command latency	> 50ms	> 200ms	Check SLOWLOG, split busy queues
Connected clients (% of max)	> 80%	> 95%	Check for connection leaks
Instantaneous ops/sec (spike)	2x baseline	5x baseline	Throttle producers, check for runaway loops
Failed job rate	> 5%	> 20%	Check error types, review worker code
Waiting + delayed jobs	Rising for 1 hr	Doubled in 1 hr	Scale workers, optimize processing
Stalled jobs (delta/hr)	> 1/hr	> 10/hr	Check worker health, network latency
Cache hit ratio (if caching)	< 0.8	< 0.5	Investigate key expiry patterns

Production Lessons from the Trenches

1. BGSAVE doubles RSS via copy-on-write -- always leave memory headroom
2. Clean completed jobs aggressively -- default BullMQ keeps everything forever
3. stallInterval matters -- tune it to match your job duration
4. Monitor omem in CLIENT LIST -- high output-buffer memory signals a slow consumer
5. BullMQ Lua EVALSHA appearing in SLOWLOG means heavy load -- these scripts are fast by design, so slowness indicates real pressure
6. NEVER use MONITOR in production -- use INFO commandstats instead
7. Test Redis failover -- what happens when ElastiCache fails over? Do your workers reconnect?

---

Conclusion -- Three Signals, One Dashboard

Redis queue health boils down to three signals:

• Latency -- measured via redis-cli --latency, SLOWLOG, and BullMQ worker stalled events
• Memory -- tracked through INFO memory, key-type analysis, and eviction monitoring
• Throughput -- exposed by BullMQ's getJobCounts(), getMetrics(), and Prometheus export

Redis gives you the primitives (INFO, SLOWLOG, LATENCY DOCTOR). BullMQ adds queue-aware metrics (getJobCounts, getMetrics). Prometheus and Grafana provide long-term storage and alerting.

But stitching all of these together requires significant setup, configuration, and maintenance. That is where QueueHub comes in.

Try QueueHub for your BullMQ or BeeQueue queues -- full Redis queue visibility in minutes, no agents, no configuration headaches. Get started at QueueHub and see your Redis queue health on one screen: latency, memory, throughput, and worker status, all in one place.