Beyond the Hash Tag: Mastering ioredis Cluster for BullMQ

Every guide about BullMQ on Redis Cluster focuses on the same thing: hash tags, CROSSSLOT errors, and sharding strategies. And those are important — but they're only half the picture. The other half is the client configuration: the ioredis Cluster constructor that BullMQ actually connects through.

When your BullMQ workers mysteriously stop processing jobs 16 hours into a deployment, or your ElastiCache cluster throws connection limits during a deploy, or your Kubernetes pods can't reach the cluster because NAT mapping is wrong — those aren't hash tag problems. They're ioredis configuration problems. And nobody writes that guide.

This post is the missing wiring diagram. We'll cover every ClusterOptions parameter that matters for BullMQ, the silent connection explosion bug, NAT maps for container environments, TLS with cluster mode, and a production checklist you can copy-paste into your next deployment.

---

BullMQ's Key Patterns and the {bullmq} Convention

Before we touch ioredis config, we need to understand how BullMQ uses Redis keys — because this directly affects how you configure the Cluster client.

BullMQ prefixes all its Redis keys. The default prefix is {bullmq}. Those curly braces aren't decorative — they're Redis Cluster hash tags. When Redis Cluster computes the hash slot for a key, it only hashes the text inside the first {...} pair. Everything outside is ignored.

// Without hash tag: keys go to different slots
HASH_SLOT('bullmq:email:wait')   → slot 7625
HASH_SLOT('bullmq:email:active') → slot 14321  // CROSSSLOT!

// With hash tag: keys go to the same slot
HASH_SLOT('{bullmq}:email:wait')   → slot 637
HASH_SLOT('{bullmq}:email:active') → slot 637  // Same slot ✓

BullMQ's Lua scripts (like moveToFinished) operate on multiple keys atomically. Without a hash tag in the prefix, every key would hit a different slot, and you'd get CROSSSLOT errors on every job completion. The {bullmq} default avoids this — but here's the trade-off: all your queues share one hash slot on one cluster node. You get zero sharding benefit within that prefix.

When to Override the Prefix

If you have multiple queues that should be on different shards, give each one a distinct hash tag:

import { Queue, Worker, QueueEvents } from 'bullmq';

// Queue A → slot calculated from 'q:email'
const emailQueue = new Queue('email', {
  connection: clusterConnection,
  prefix: '{q:email}',
});

// Queue B → different slot
const videoQueue = new Queue('video', {
  connection: clusterConnection,
  prefix: '{q:video}',
});

// ⚠️ ALL BullMQ classes for the same queue must use the SAME prefix
const worker = new Worker('email', processor, {
  connection: clusterConnection,
  prefix: '{q:email}',
});

const events = new QueueEvents('email', {
  connection: clusterConnection,
  prefix: '{q:email}',
});

Critical: Never use ioredis's built-in keyPrefix option with BullMQ. BullMQ manages its own prefix via the prefix option on Queue/Worker/QueueEvents. Using both causes double-prefixing — every key becomes {bullmq}{q:email}:email:wait and nothing matches.

Debugging Slot Assignment

If you're unsure where your queues are landing, you can check directly:

import { Cluster } from 'ioredis';

async function checkSlot(cluster: Cluster, key: string): Promise<void> {
  const slot = await cluster.cluster('keyslot', key);
  console.log(`Key "${key}" → slot ${slot}`);
}

// Simulate BullMQ keys
checkSlot(clusterConnection, '{bullmq}:email:wait');
checkSlot(clusterConnection, '{bullmq}:email:active');
checkSlot(clusterConnection, '{q:email}:email:wait');
checkSlot(clusterConnection, '{q:video}:video:wait');

Running this tells you exactly which hash slot each queue belongs to — and therefore which cluster node handles it. You can also use cluster.cluster('slots') to map slot ranges to node addresses.

---

The ioredis Cluster Constructor — Every Option That Matters

This is the heart of the post. The Cluster constructor accepts a comprehensive ClusterOptions object, and most defaults are wrong for BullMQ. Here's the production configuration we'll build toward:

import { Cluster, ClusterOptions, ClusterNode } from 'ioredis';
import dns from 'dns';

const startupNodes: ClusterNode[] = [
  { host: 'mycluster.cluster-xxxxx.us-east-1.cache.amazonaws.com', port: 6379 },
];

const clusterOptions: ClusterOptions = {
  // ── Slot Refresh ──────────────────────────────────
  // How often ioredis re-fetches the cluster slot map.
  // Default: 3000ms. For stable production clusters,
  // increase to reduce chatter.
  slotsRefreshTimeout: 30000,

  // ── Read Scaling ──────────────────────────────────
  // BullMQ operations need strong consistency. Never
  // route reads to replicas — you'll get stale data.
  scaleReads: 'master',

  // ── Max Redirections ──────────────────────────────
  // BullMQ Lua scripts run many commands per job.
  // During resharding, each may hit MOVED/ASK errors.
  // Default 16 is tight for 50+ concurrent workers.
  maxRedirections: 64,

  // ── Ready Check ───────────────────────────────────
  // Don't start processing before the cluster is ready.
  enableReadyCheck: true,

  // ── Cluster Retry Strategy ────────────────────────
  // BullMQ Workers have their own retry mechanism.
  // Don't let the Cluster bootstrap retry forever.
  clusterRetryStrategy: (times: number) => {
    if (times > 10) {
      console.error('[Cluster] Max retry attempts reached');
      return null;
    }
    return Math.min(100 + times * 500, 10000);
  },

  // ── Redis Options (passed to each per-node connection) ──
  redisOptions: {
    // ** REQUIRED by BullMQ **
    maxRetriesPerRequest: null,

    // Keep TCP connections alive
    keepAlive: 30000,

    // Identifiable connection name for CLIENT LIST
    connectionName: 'bullmq-cluster',

    // Blocking commands (BRPOPLPUSH) can wait long
    commandTimeout: 15000,

    // Queue commands during reconnection
    enableOfflineQueue: true,

    // TLS (see Section 5 for provider-specific configs)
    tls: process.env.REDIS_TLS === 'true'
      ? { rejectUnauthorized: false }
      : undefined,
  },
};

const cluster = new Cluster(startupNodes, clusterOptions);

Option-by-Option Breakdown

Option	Default	BullMQ Recommendation	Why
slotsRefreshTimeout	3000ms	30000ms	Stable clusters don't change topology often; less chatter
scaleReads	'master'	'master'	Queue state must be strongly consistent
maxRedirections	16	64	Lua scripts do multiple ops; resharding amplifies redirects
enableReadyCheck	true	true	Don't start workers before cluster is ready
clusterRetryStrategy	built-in	Custom capped	BullMQ handles its own retries
redisOptions.maxRetriesPerRequest	20	null	Non-negotiable. Breaks BullMQ's blocking commands
redisOptions.enableOfflineQueue	true	true	Don't drop commands during cluster reconnect

The single most important option is maxRetriesPerRequest: null. BullMQ uses blocking commands (BRPOPLPUSH) that sit open waiting for data. If ioredis retries these internally (the default behavior with maxRetriesPerRequest: 20), the retry logic interferes with BullMQ's own retry mechanism, causing deadlocks and silent job stalls. Setting it to null tells ioredis: "never retry — the application (BullMQ) will handle it."

---

The Connection Explosion Problem

Here's the bug that eats production deployments.

When you pass an ioredis Cluster instance to BullMQ's Worker, the Worker calls connection.duplicate() internally to create a separate blocking client. This is by design — the main connection handles standard commands, while the duplicate handles the blocking BRPOPLPUSH call.

The problem: Cluster.duplicate() creates a full new Cluster instance — fresh slot map, fresh TCP connections to every shard, fresh everything. Nothing is shared with the original.

// Every Worker internally does:
class Worker {
  constructor(queueName, processor, opts) {
    const connection = opts.connection;
    this.bclient = connection.duplicate();
    //         ↑──────────────────────────────────────┐
    // duplicate() calls new Cluster(startupNodes)    │
    // with a complete new set of TCP connections     │
    // to ALL shards and ALL replicas                 │
  }
}

The math is brutal: 80 workers × 3 shards × 2 connections (pub/sub + blocking) = 480+ TCP connections from a single Node.js process. ElastiCache monitors this as "new-connections-per-minute" spikes during deployments, and you can hit the instance connection limit before workers finish booting.

The Wedged bclient (BullMQ Issue #4138)

The deeper problem: Cluster.duplicate() has a race condition where connect() can reject silently. The .catch(noop) in ioredis swallows the failure. The duplicate ends up in wait state with an empty connection pool — no nodes, no connections. The Worker polls forever, but no job ever gets processed. Throughput goes to zero with zero error logs.

Mitigation Strategies

Strategy 1: Fewer workers per process, more processes.

Instead of 80 workers in one Node.js process (480 connections), run 8 processes with 10 workers each (60 connections per process). This spreads the connection budget and isolates failures.

Strategy 2: Slow down slot refresh.

const cluster = new Cluster(startupNodes, {
  ...options,
  slotsRefreshTimeout: 60000,
});

Each duplicate re-fetches the slot map independently. A longer refresh interval means fewer simultaneous CLUSTER SLOTS calls during boot.

Strategy 3: Monitor for wedged bclients.

setInterval(async () => {
  for (const [name, worker] of workerRegistry) {
    if (!worker.isRunning()) continue;

    const bclient = (worker as any).bclient;
    if (bclient && bclient.status === 'wait') {
      logger.warn(`Worker ${name}: bclient appears stuck (connection pool empty)`);
      await worker.close();
      // restart logic...
    }
  }
}, 60000);

This won't fix the root cause, but it will surface the problem when it happens — which is the first step to debugging it.

---

NAT Maps, DNS, and Kubernetes — Making Cluster Work in Containers

Redis Cluster nodes announce themselves using their internal IP addresses. In Kubernetes, that's a pod IP like 10.42.1.5. In Docker, it's a bridge network IP like 172.17.0.2. Your application may not be able to reach those IPs from outside the pod's network namespace.

ioredis's natMap option translates internal addresses to reachable ones:

const cluster = new Cluster(startupNodes, {
  natMap: {
    '172.17.0.2:6379': { host: '10.0.1.50', port: 6379 },
    '172.17.0.3:6379': { host: '10.0.1.51', port: 6379 },
    '172.17.0.4:6379': { host: '10.0.1.52', port: 6379 },
  },
});

Dynamic NAT Maps for Kubernetes

In Kubernetes, pod IPs change when pods restart. A static natMap goes stale. The solution is a headless Service and dynamic DNS resolution:

import { Cluster } from 'ioredis';
import dns from 'dns/promises';

async function createK8sCluster(
  serviceDns: string,
  shardCount: number,
  port: number
): Promise<Cluster> {
  // Resolve pod IPs via headless Service DNS
  const nodes: ClusterNode[] = [];
  for (let i = 0; i < shardCount; i++) {
    const podDns = `redis-cluster-${i}.${serviceDns}`;
    try {
      const [address] = await dns.resolve4(podDns);
      nodes.push({ host: address, port });
    } catch {
      // Pod doesn't exist yet — skip
    }
  }

  return new Cluster(nodes, {
    redisOptions: {
      maxRetriesPerRequest: null,
      family: 4, // Force IPv4
    },
    dnsLookup: (hostname, callback) => {
      dns.resolve4(hostname)
        .then(addresses => callback(null, addresses[0], 4))
        .catch(err => callback(err));
    },
  });
}

AWS ElastiCache Configuration Endpoint

ElastiCache (Redis OSS Cluster Mode) provides a single configuration endpoint. If your app is in the same VPC, no NAT map is needed — the internal IPs are reachable:

const cluster = new Cluster(
  [{ host: 'mycluster.cluster-xxxxx.us-east-1.cache.amazonaws.com', port: 6379 }],
  {
    redisOptions: {
      maxRetriesPerRequest: null,
      tls: { rejectUnauthorized: false },
    },
    enableReadyCheck: true,
    slotsRefreshTimeout: 30000,
  }
);

Important for cross-region or hybrid setups: If your application is outside the Redis Cluster's VPC, you'll need a NAT map or a proxy (HAProxy, Envoy) in front of the cluster endpoint.

---

TLS with Redis Cluster — Trickier Than Single-Node

With a single-node Redis, TLS is straightforward: pass tls: {} in redisOptions. With Cluster, this same tls config propagates to every per-shard connection — which is what you want, but introduces complexity.

Rediss:// Does Not Work with Cluster

The rediss:// URL scheme tells ioredis to wrap the single connection in TLS. The Cluster constructor doesn't use URL-based configuration for per-shard connections — it passes redisOptions to each underlying Redis instance. So you must use the tls option:

// ❌ This doesn't work with new Cluster()
new Cluster(startupNodes, {
  redisOptions: { maxRetriesPerRequest: null },
});
// Each per-shard connection is plain TCP — error on TLS-enabled clusters

// ✓ This works:
new Cluster(startupNodes, {
  redisOptions: {
    maxRetriesPerRequest: null,
    tls: { rejectUnauthorized: false },
  },
});

Provider-Specific TLS Configurations

AWS ElastiCache (TLS enabled):

ElastiCache node certificates don't match private IPs, so rejectUnauthorized: false is required:

const elasticacheCluster = new Cluster(
  [{ host: 'mycluster.cluster-xxxxx.us-east-1.cache.amazonaws.com', port: 6379 }],
  {
    redisOptions: {
      maxRetriesPerRequest: null,
      tls: { rejectUnauthorized: false },
    },
  }
);

Self-hosted with mutual TLS:

import fs from 'fs';

const selfHostedCluster = new Cluster(
  [
    { host: 'redis-1.internal', port: 6379 },
    { host: 'redis-2.internal', port: 6379 },
    { host: 'redis-3.internal', port: 6379 },
  ],
  {
    redisOptions: {
      maxRetriesPerRequest: null,
      tls: {
        ca: fs.readFileSync('/etc/ssl/certs/redis-ca.pem'),
        cert: fs.readFileSync('/etc/ssl/certs/redis-client.crt'),
        key: fs.readFileSync('/etc/ssl/private/redis-client.key'),
        rejectUnauthorized: true,
      },
    },
  }
);

TLS Handshake Overhead

Each per-shard connection does a full TLS handshake. With 3 shards × 1 replica × 80 workers × 2 connections per worker = 960 handshakes at boot. Budget 50-100ms per handshake — that's 48-96 seconds of connection setup time under contention. This is another argument for fewer workers per process: fewer concurrent TLS handshakes, faster boot.

---

Sentinel vs. Cluster — A Decision Framework

Redis Sentinel provides high availability for a single Redis node (master-replica with automatic failover). Redis Cluster provides both high availability and horizontal scaling. Which one for BullMQ?

Criterion	Choose Sentinel	Choose Cluster
Total job data < 25 GB	✓
Connection count < 5000	✓
Throughput < 10K jobs/sec	✓
Multi-tenant isolation		✓
Data exceeds single-node memory		✓
Multiple queues with different SLAs		✓
Existing team Cluster infrastructure		✓

Sentinel is simpler: ioredis connects via new Redis({ sentinels: [...], name: 'mymaster' }), no hash tags needed, no CROSSSLOT errors, lower connection overhead. For the vast majority of BullMQ deployments, Sentinel (or a single-node ElastiCache with automatic failover) is the right choice.

Cluster becomes necessary when you need sharding — either because memory exceeds a single node, or because per-tenant queue isolation requires different queues on different physical nodes. But recognize the operational complexity you're trading for that capacity.

---

The ioredis Cluster Cheat Sheet for BullMQ

Six things to get right:

1. Always set maxRetriesPerRequest: null in redisOptions
2. Use BullMQ's prefix option with a hash tag — never ioredis keyPrefix
3. Set slotsRefreshTimeout to 30000ms for stable clusters
4. Configure natMap for Docker/Kubernetes deployments
5. Budget for connection explosion — fewer workers per process, more processes
6. Monitor bclient health — wedged duplicates are silent throughput killers

Final checklist for your next deployment:

• maxRetriesPerRequest: null set in redisOptions
• Hash tag {...} in BullMQ prefix on ALL Queue/Worker/QueueEvents
• Consistent prefix across all classes for the same queue
• NAT map configured if running in Docker/Kubernetes
• TLS configured in redisOptions (not URL) for ElastiCache/Valkey
• slotsRefreshTimeout tuned for your failover tolerance
• Connection budget calculated: Workers × shards × 2 (blocking + events)
• bclient health monitoring in place (periodic heartbeat or QueueHub)
• Single-node or Sentinel considered first — Cluster is overkill for most queue workloads

Redis Cluster doesn't magically make BullMQ faster. It's a capacity and isolation tool, not a speed upgrade. With the right ioredis configuration, it will keep your queues running reliably at scale — but it demands attention to detail at the client layer that most guides skip.

---

Take the Guesswork Out of Cluster Monitoring with QueueHub

Seeing what's happening across your cluster shards shouldn't require running redis-cli --cluster check on six different nodes. QueueHub connects directly to your Redis Cluster — whether it's ElastiCache, self-hosted, or Valkey — and gives you a single-pane view of queue health across all shards.

Monitor per-node memory, connection counts, slot distribution, and worker health, all from one dashboard. Stop guessing whether your ioredis config is right — see what your cluster is actually doing.

Try QueueHub Free →