Payload CMS autoRun vs Worker: Practical Queue Guide | Build with Matija
Payload CMS autoRun vs Worker: Practical Queue Guide
Split Payload jobs between in-process autoRun and dedicated workers — queue routing for media, inventory, emails and…
·Updated on:··
Need Help Making the Switch?
Moving to Next.js and Payload CMS? I offer advisory support on an hourly basis.
Payload CMS autoRun vs Dedicated Worker: How to Use Both Effectively
If you are building a Payload CMS app and wondering whether to reach for autoRun or a dedicated bin script worker, the more useful framing is not "which one" but "which one for which queue." autoRun is queue-selective — it does not run every Payload job, only jobs from the queues you explicitly configure it to process. That distinction unlocks a hybrid architecture where autoRun handles light work inside the app process, and a dedicated worker handles heavy work in isolation. This article explains how each runner works, what it costs, and how to split your jobs between them using real examples from a production multi-tenant marketplace build.
I went through this decision while designing the job architecture for a Farmica farm ordering platform — a system that handles media uploads, transactional emails, inventory processing, and order fan-out all within the same Payload app. Getting the queue routing right early made a meaningful difference in how the app behaved under load.
📚 Comprehensive Payload CMS Guides
Detailed Payload guides with field configuration examples, custom components, and workflow optimization tips to speed up your CMS development process.
Payload separates four concepts that are easy to conflate.
A task is a function definition — the shape and logic of some work you want done. A job is a concrete queued instance of that task, stored as a document in the payload-jobs collection. A queue is a named lane that groups related jobs together — something like emails, media, or inventory. A runner is whatever picks jobs off a queue and executes them.
A job does not run because you defined a task. You queue it with payload.jobs.queue(), Payload writes a document to the database, and then a runner picks it up. That database persistence is what makes queues durable across restarts and what lets you route different job types to completely different runners.
The runner configuration is where autoRun and the dedicated bin script diverge.
The Critical Thing autoRun Does Not Do
autoRun does not run every Payload job in the system. It runs jobs from the queues you give it. This is the most important thing to understand before designing your architecture.
then jobs queued to media, inventory, or video will sit in payload-jobs untouched until another runner processes those queues. autoRun will never touch them. Payload's own troubleshooting docs give exactly this example — if you queue to critical but autoRun only processes default, the job will not be picked up.
You can also run multiple autoRun entries in the same app process, each targeting a different queue on its own cron interval:
Each entry is its own independent runner inside the same process. This is the routing mechanism — queue name is the contract between the producer and the runner.
What autoRun Actually Does (and One Trap)
autoRun runs within the Next.js/Payload process. It polls for pending jobs on a cron interval and executes them synchronously within that process. That means it shares CPU, memory, database connections, and I/O with your app.
A few things the docs do not make obvious:
autoRun is not instant. A job queued at 12:00:10 with a * * * * * cron may not run until 12:01. For transactional email this is usually acceptable. For anything user-facing, it is not.
autoRun handles scheduling by default. Unless you set disableScheduling: true, it will also call handleSchedules, which can create jobs from schedule definitions on tasks. If you are running both autoRun and a bin script for the same queue, this can duplicate scheduled jobs. Set disableScheduling: true when you want autoRun to only process already-created jobs.
shouldAutoRun is a kill switch, not a throttle. The docs say that if shouldAutoRun returns false, the cron schedule is stopped — not skipped for one tick, stopped. Do not use it for dynamic backpressure like checking active job count. Use it only for stable environment gating:
This is the correct use case: controlling which app replicas act as runners. On a multi-instance deployment, you set ENABLE_AUTORUN=true only on designated instances so that every web process is not competing to run jobs.
autoRun must not run on serverless. Payload's docs are explicit — it requires a long-running server. On Vercel, use the /api/payload-jobs/run endpoint triggered by Vercel Cron instead.
What a Dedicated Worker Does
A dedicated worker is the Payload bin script running as a separate process:
That process has no Next.js overhead, shares no resources with your web app, and can be deployed, restarted, and scaled entirely independently. You can containerize it with its own resource limits, install native dependencies like sharp or FFmpeg without polluting the main app bundle, and crash it without affecting API response times.
The Payload docs position the bin script as the recommended production approach for dedicated servers. autoRun is described as the simpler alternative — not the stronger isolation pattern.
Real Job Cost Analysis: A Production Example
Here is what the queue breakdown actually looked like on a marketplace platform I built with Payload — a multi-tenant farm ordering system with media uploads, checkout flow, and transactional email.
Heaviest (CPU and I/O)
media queue → generateMediaBlurPlaceholder
Downloads the full image from S3 or HTTP, runs sharp (resize to 10×10, blur, PNG), and writes the blurDataURL back to Payload. This fires on every new image upload. A backfillMediaBlurPlaceholders maintenance task can enqueue up to 500 blur jobs in one go. This was the main heavy queue in normal day-to-day operation and was an immediate candidate for a dedicated worker.
Medium (DB and external APIs)
inventory queue → processOrderInventory
Runs the full inventory hook logic off the request path — reserve, release, or fulfill per line item, with status transitions. Cost scales with line item count. Each order update can run a line-item diff and a status transition inside the same job.
emails queue → sendTransactionalEmail
Resolves template and recipient data via several DB reads, runs a dedup check, then calls Brevo over HTTP. A single new order can queue two transactional emails plus a sendOrderEmails job simultaneously. Under a flash sale or bulk order scenario, this queue is noisier than it looks at rest.
Loads up to 1000 docs, then for each doc across three locales (sl, en, de): one findByID plus one update. This is a lot of database round-trips concentrated in a short window. It is maintenance-triggered and infrequent, but it should never run inside autoRun on a production replica.
default queue → migrateRuLocaleToDe
Raw SQL over many locale tables combined with OpenAI calls per Cyrillic field. One-off migration task, extremely heavy, always runs as a manual dedicated worker invocation.
Light (Fine for autoRun)
Queue / task
Why it is light
logs → persistLog
Single log row write
promo → processOrderPromoUsage
A few scoped reads plus one promo update
The Volume Cap to Watch
With autoRun configured at a limit of 50 jobs per minute across all queues, a large media backfill or an order spike can cause the media and emails queues to lag for several ticks. This is worth monitoring if you bulk-upload images or run a flash sale. The lag is not dangerous, but it is visible in user-facing flows that depend on blurDataURL being populated quickly.
The Decision Framework
The queue name is the contract. The runner only processes the queue you assign to it. Heavy queues must never be included in an in-process autoRun config.
Job type
Queue
Runner
Reason
Transactional email
emails
autoRun
Mostly network I/O, short, predictable
Blur placeholder generation
media
Dedicated worker
CPU and I/O, fires on every upload, can bulk-enqueue 500 at once
Inventory processing
inventory
Dedicated worker
DB-heavy, scales with line item count
Log persistence
logs
autoRun
Single write, negligible cost
Promo usage update
promo
autoRun
Small scoped reads, safe in-process
Slug generation
default
Manual dedicated invocation
Bulk DB round-trips, maintenance-only
One-off data migrations
default
Manual dedicated invocation
Extreme cost, never in autoRun
Video transcoding
video
Dedicated worker
CPU, memory, disk, long-running
The practical split I arrived at:
autoRun inside app process:
emails — transactional, low volume
logs — single-write, negligible
promo — scoped reads, bounded
dedicated worker (media):
media — blur generation, every upload, backfill spikes
dedicated worker (inventory):
inventory — order processing off the request path
manual invocation only:
default — slug generation, migrations
The Code
Queuing to the right queue at the call site:
// File: src/jobs/queue.ts// Light — autoRun will pick this upawait payload.jobs.queue({
task: 'sendTransactionalEmail',
input: { to, templateId, variables },
queue: 'emails',
})
// Light — autoRun will pick this upawait payload.jobs.queue({
task: 'persistLog',
input: { level: 'info', message, context },
queue: 'logs',
})
// Heavy — only the dedicated media worker will pick this upawait payload.jobs.queue({
task: 'generateMediaBlurPlaceholder',
input: { mediaId, url },
queue: 'media',
})
// Heavy — only the dedicated inventory worker will pick this upawait payload.jobs.queue({
task: 'processOrderInventory',
input: { orderId, lineItems, previousDoc },
queue: 'inventory',
})
autoRun configured to handle only the light queues:
The limit on the media worker is intentionally low because each blur job calls sharp, and sharp is CPU and memory intensive. Five concurrent jobs per tick is enough throughput for normal upload volume while keeping the worker from saturating available memory. For backfill spikes, the jobs simply queue up across multiple ticks — the database persistence handles the backlog safely.
FAQ
Does autoRun pick up jobs from all queues automatically?
No. autoRun only processes jobs from the queues you explicitly configure in each autoRun entry. A job queued to media with only an emails entry in autoRun will sit in the database until a runner targeting media picks it up. This is by design — queue name is the routing contract.
What does disableScheduling: true actually do?
It tells autoRun to skip the handleSchedules step, meaning it will only process jobs that are already in the queue rather than also creating new jobs from schedule definitions on tasks. Set it when you want predictable, run-only behavior and are handling schedule creation elsewhere.
Can shouldAutoRun be used to throttle autoRun dynamically?
Carefully — if shouldAutoRun returns false, the cron schedule stops rather than skipping a single tick. It is safe for environment-level gating like process.env.ENABLE_AUTORUN === 'true', but risky for dynamic checks based on current load or active job count.
What happens to jobs in heavy queues if I never set up a dedicated worker?
They accumulate in payload-jobs with waitingForWorker status and never run. Payload does not automatically fall back to processing them inside autoRun. This is actually useful during development — you can queue jobs freely and only spin up the worker when you are ready to process them.
Can I run autoRun locally and a dedicated worker in production?
Yes. Use shouldAutoRun with an environment variable to gate the behavior, or simply omit autoRun from your production config entirely and run everything via bin script. For local development, autoRun on all queues is the fastest way to test the full job lifecycle without additional processes.
Conclusion
autoRun and the dedicated bin script worker are not alternatives to choose between — they are complementary runners designed to handle different categories of work. autoRun handles light, I/O-bound jobs inside the app process with minimal infrastructure. The dedicated worker handles heavy, CPU-intensive, or failure-sensitive jobs in isolation.
The routing mechanism is the queue name. Every job specifies a queue, and every runner specifies which queue it processes. Keeping that discipline — light queues in autoRun, heavy queues in dedicated workers, maintenance queues as manual invocations — gives you the UX benefit where user actions return fast, and the operational benefit where a crashing worker cannot take down your app.
If you are designing a Payload jobs architecture and want to think through the queue routing for your specific workload, drop a question in the comments below.
Thanks,
Matija
I'm Matija Žiberna, a self-taught full-stack developer and co-founder passionate about building products, writing clean code, and figuring out how to turn ideas into businesses. I write about web development with Next.js, lessons from entrepreneurship, and the journey of learning by doing. My goal is to provide value through code—whether it's through tools, content, or real-world software.
