Introduction

BullMQ for Elixir - Full-featured job queue with OTP design.

BullMQ for Elixir is a robust, feature-rich message queue and job scheduling library built on top of Redis. It's a port of the popular BullMQ library, providing full compatibility with existing BullMQ queues from Node.js, Python, and PHP.

The Elixir package is available on Hex.pm. Full API documentation is available on HexDocs.

Features

• High Performance: Leverages Redis for fast, reliable message passing

• Job Scheduling: Schedule jobs to run at specific times or intervals

• Priority Queues: Process high-priority jobs first

• Retry Strategies: Automatic retries with configurable backoff

• Rate Limiting: Control job processing rates

• Parent-Child Jobs: Create complex job hierarchies with dependencies

• Real-time Events: Subscribe to job lifecycle events via Worker callbacks or QueueEvents

• Concurrency Control: Process multiple jobs simultaneously with true BEAM parallelism

• Stalled Job Recovery: Automatically recover jobs from crashed workers

• Telemetry Integration: Built-in observability with Telemetry

• OTP Design: Built using GenServers, Supervisors, and other OTP patterns

• Cross-language Compatibility: Share queues between Elixir, Node.js, Python, and PHP

Installation

Add BullMQ to your mix.exs:

def deps do
  [
    {:bullmq, "~> 0.1.0"},
    {:redix, "~> 1.2"}  # Redis client
  ]
end

Then run:

mix deps.get

Quick Start

Setting Up Redis Connection

Add a Redix connection to your supervision tree:

defmodule MyApp.Application do
  use Application

  def start(_type, _args) do
    children = [
      {Redix, name: :my_redis, host: "localhost", port: 6379}
    ]

    Supervisor.start_link(children, strategy: :one_for_one)
  end
end

Adding Jobs

# Add a job to a queue
{:ok, job} = BullMQ.Queue.add("emails", "send-welcome", %{
  to: "[e-mail]",
  template: "welcome"
}, connection: :my_redis)

IO.puts("Created job: #{job.id}")

Processing Jobs

defmodule MyApp.EmailWorker do
  def process(%BullMQ.Job{name: "send-welcome", data: data}) do
    MyApp.Mailer.send_welcome(data["to"], data["template"])
    {:ok, %{sent: true}}
  end
end

# Start the worker
{:ok, worker} = BullMQ.Worker.start_link(
  queue: "emails",
  connection: :my_redis,
  processor: &MyApp.EmailWorker.process/1,
  concurrency: 5,
  on_completed: fn job, result ->
    Logger.info("Job #{job.id} completed")
  end
)

Architecture

BullMQ for Elixir leverages OTP patterns for reliability:

• GenServer for stateful components (Worker, QueueEvents)

• Supervisor for fault tolerance

• Telemetry for observability

• True Parallelism using multiple BEAM processes for concurrent job processing

Documentation

For comprehensive documentation, see:

• HexDocs - Full API Reference

• Getting Started Guide

• Workers Guide

• Job Options

• Rate Limiting

• Telemetry

Requirements

• Elixir 1.15 or later

• Erlang/OTP 26 or later

• Redis 6.0 or later (6.2+ recommended)