APScheduler Running Twice: Gunicorn Workers, Reloaders, and --preload

APScheduler runs each job once per scheduler, and a BackgroundScheduler started when your app loads creates one scheduler in every process that loads it: each Gunicorn or Uvicorn worker, both processes of Flask's debug reloader, and both processes of Django's runserver autoreloader. APScheduler 3.x has no coordination between processes, so every one of them fires the job. The fix is to run the schedule in exactly one process, either a dedicated one or the single web worker that wins a lock, and to guard the development reloaders separately.

This guide covers APScheduler 3.11.3, the current release on PyPI (checked October 30, 2026); the 4.0 line is still pre-release there, at 4.0.0a6. Server behavior is checked against the source of Gunicorn 26.2.0, Uvicorn 0.54.0, Flask 3.1.3 with Werkzeug 3.1.9, and Django 6.1.2. For APScheduler's general failure modes and events, see Python scheduler monitoring; for duplicates caused by crontab, systemd, or Kubernetes, see why a cron job runs twice.

Count the Schedulers First#

Log the process every time a scheduler starts. start() dispatches EVENT_SCHEDULER_STARTED once it is running (base.py), so add the listener before calling it:

import logging
import os

from apscheduler.events import EVENT_SCHEDULER_STARTED

log = logging.getLogger("scheduler")


def log_scheduler_start(event):
    log.warning("APScheduler started: pid=%s parent=%s", os.getpid(), os.getppid())


scheduler.add_listener(log_scheduler_start, EVENT_SCHEDULER_STARTED)
scheduler.start()

Then compare the duplicate runs:

The last two have one scheduler and are covered under when one scheduler runs the job twice.

Why Every Web Worker Runs Its Own Scheduler#

BackgroundScheduler.start() starts a thread named APScheduler in the calling process and returns (background.py). Each process that runs that code gets its own scheduler object, its own job store, and its own executor. Nothing connects them. The APScheduler 3.x FAQ is direct about the shared-database version of this:

Sharing a persistent job store among two or more processes will lead to incorrect scheduler behavior like duplicate execution or the scheduler missing jobs, etc. This is because APScheduler does not currently have any interprocess synchronization and signalling scheme...

The user guide adds: "Job stores must never be shared between schedulers." With the default in-memory store, every worker holds a full private copy of every job, so each one runs all of them. Moving to SQLAlchemyJobStore doesn't fix this. It replaces independent copies with schedulers racing over the same rows.

The FAQ entry on web applications says to "First read through the previous section" (faq.rst), which is the job store entry above. How many processes you get depends on the server.

Gunicorn#

Without --preload, each worker loads your WSGI app after it has been forked (workers/base.py), so code at import time, or in an app factory, runs once per worker. -w 4 means four schedulers. The worker count defaults to the WEB_CONCURRENCY environment variable when it is set (config.py), so a platform that sets it can turn one scheduler into several without a change to your command.

Worker restarts start schedulers too. With --max-requests, a worker restarts after that many requests (config.py), and its replacement loads the app again, starting a new scheduler as the old one exits with its worker.

Uvicorn#

uvicorn app:app --workers 4 starts four worker processes from a spawn multiprocessing context (_subprocess.py), and each one imports the app in its own server (server.py). As with Gunicorn, the worker count defaults to WEB_CONCURRENCY when you don't pass --workers (config.py). Moving scheduler.start() from module level into a FastAPI or Starlette lifespan handler doesn't help: the lifespan is created in that same per-worker server, right after the app loads.

With --reload or more than one worker, the Uvicorn parent process doesn't load the app at all; only the children do (main.py). Under uvicorn --reload, you therefore have one scheduler at a time, replaced on each reload.

What --preload Does to the Scheduler Thread#

With --preload, Gunicorn loads the app in the master process before forking any workers (arbiter.py), a setting its docs describe as a way to "save some RAM resources as well as speed up server boot times" (config.py). A scheduler started at import now starts its thread in the master. Workers are created with os.fork() (arbiter.py), and POSIX specifies that the child of a multi-threaded process contains "a replica of the calling thread" only (fork).

The result looks like a fix and isn't one:

• Jobs run only in the master, because only the master has a scheduler thread. The duplicates disappear by accident.
• Each worker holds a copy of the scheduler with no thread behind it. Its state was copied from the master, so it still reports itself as started. A job a request handler adds with add_job() goes into that worker's copy, where nothing will run it. With a persistent store, the row reaches the database, but the master's scheduler isn't notified, per the FAQ above.
• Every fork is a fork of a multi-threaded process. Since Python 3.12, os.fork() raises a DeprecationWarning when it detects other threads, and the docs say "it has never been safe to mix threading with os.fork() on POSIX platforms" (os.fork). Gunicorn forks again every time it replaces a worker.

Treat --preload as a memory setting. If the scheduler belongs in the master, it belongs in its own process instead.

Development Reloaders Start a Second Scheduler#

These cases run two schedulers on a laptop even when production runs one.

Flask: flask run --debug and app.run(debug=True)#

flask run --debug turns the reloader on, because --reload defaults to the debug flag (cli.py). The command loads your app before it starts the server (cli.py). Werkzeug's reloader then starts a new interpreter with the same arguments and WERKZEUG_RUN_MAIN=true, waits for it, and starts another when a file change makes it exit with code 3 (_reloader.py). That child loads the app again.

So a scheduler started at import or in create_app() runs in the parent for the whole session and in the child, with the child's copy replaced on every reload. app.run(debug=True) behaves the same way, because use_reloader defaults to the debug setting (app.py).

Werkzeug exposes the check as is_running_from_reloader(), which tests that environment variable (serving.py). --debug sets FLASK_DEBUG early so that factories can read it (cli.py), and the app's DEBUG config starts from that value (sansio/app.py):

from flask import Flask
from werkzeug.serving import is_running_from_reloader


def create_app():
    app = Flask(__name__)
    # ... register blueprints and jobs ...
    if not app.debug or is_running_from_reloader():
        scheduler.start()
    return app

Under flask run --debug, or with FLASK_DEBUG=1 set, this skips the reloader parent and nothing else. It makes the scheduler wait for the reloader in debug mode, so flask run --debug --no-reload starts no scheduler at all. It does nothing about Gunicorn workers, where app.debug is false and every worker passes the check.

The guard doesn't work with app.run(debug=True). get_debug_flag() reads only FLASK_DEBUG (helpers.py), and app.run() sets app.debug from its debug argument only when it is called (app.py), after create_app() has returned. The factory sees app.debug as false in both processes, and both start a scheduler. flask run --reload without --debug fails the same way. For app.run(debug=True), set FLASK_DEBUG=1 in the environment before the script starts, or pass use_reloader=False.

Django: runserver and AppConfig.ready()#

Unless you pass --noreload, manage.py runserver runs django.setup() in the parent process first (management/__init__.py), which calls every AppConfig.ready(). The autoreloader then runs the command again in a child with RUN_MAIN=true (autoreload.py, autoreload.py), and ready() runs again there. A scheduler started in ready() runs twice.

It also starts in places you didn't intend. Django's docs warn that ready() "will run during startup of every management command" (applications), so migrate, shell, and collectstatic each start a scheduler too, one that lasts as long as the command does.

Checking os.environ.get("RUN_MAIN") == "true" skips the parent, but RUN_MAIN is an internal name in django.utils.autoreload (autoreload.py), not a documented setting, and it isn't set under Gunicorn at all. The durable fix is to take the scheduler out of ready() and start it from a management command run as its own process, the dedicated scheduler process below. Django cron monitoring gives the same advice for django-apscheduler.

The Fixes, and What Each Costs#

Every fix trades duplicate runs for a different failure: zero schedulers. Decide which you would rather detect.

A Dedicated Scheduler Process#

This is the FAQ's workaround: "Run the scheduler in a dedicated process" (faq.rst). Use BlockingScheduler, which the user guide recommends "when the scheduler is the only thing running in your process" (userguide.rst), in an entry point such as python -m myapp.scheduler, deployed beside the web service. Remove every start() call from the web app. Python scheduler monitoring has a complete worker of this kind.

What it costs:

• One more process to deploy, supervise, and restart. If it's down, nothing runs and nothing errors.
• No more add_job() from request handlers. The web app can't reach the scheduler's in-memory jobs. The FAQ suggests a remote access mechanism such as RPyC, gRPC, or an HTTP server; many teams define jobs in code instead.
• It must stay at one copy. Scaling the scheduler service to two replicas brings the duplicates back. Distributed cron covers why a one-replica Deployment can briefly be two during a rolling update.

One Web Worker Wins a Lock#

Keep the scheduler inside the web app, and let only the worker that gets an exclusive lock start it. On a single host, fcntl.flock with LOCK_NB fails immediately with an OSError instead of waiting (fcntl):

import fcntl
import logging
import os

log = logging.getLogger("scheduler")
_lock_file = None  # Keep the file open: closing it releases the lock.


def start_scheduler_if_first(scheduler, lock_path):
    global _lock_file
    lock_file = open(lock_path, "w")
    try:
        fcntl.flock(lock_file, fcntl.LOCK_EX | fcntl.LOCK_NB)
    except OSError:
        lock_file.close()
        log.info("pid=%s: another process runs the scheduler", os.getpid())
        return False
    _lock_file = lock_file
    scheduler.start()
    log.info("pid=%s: running the scheduler", os.getpid())
    return True

Call it where you used to call scheduler.start(). The kernel releases the lock when every descriptor for that open file is closed, including when the process exits (flock(2)), so a crashed holder can't leave a stale lock.

What it costs:

• One scheduler per host, not per deployment. Each machine or container has its own lock file. Two app servers mean two schedulers.
• The scheduler lives and dies with one web worker. When Gunicorn restarts the holder, for --max-requests or a timeout, the scheduler goes with it. It comes back only when a newly booted worker takes the lock. Workers that lost the race at startup don't try again.
• Take the lock after the fork. Under --preload, this code runs once, in the master, and flock(2) notes that descriptors duplicated by fork(2) "refer to the same lock," so every worker shares the master's.

Across hosts, PostgreSQL's pg_try_advisory_lock(key bigint) does the same thing: it "will either obtain the lock immediately and return true, or return false without waiting" (advisory lock functions). A session-level advisory lock "is held until explicitly released or the session ends" (advisory locks), so the winning worker must keep that connection open for its whole life, outside any pool that might close or reuse it. If the connection drops, the lock is released while the scheduler thread keeps running, and the next worker to boot can win it too.

Run One Web Worker#

A single web worker also gives you one scheduler per host: -w 1 for Gunicorn, --workers 1 for Uvicorn. Pass the count explicitly, because without the flag both servers use WEB_CONCURRENCY when it is set (Gunicorn, Uvicorn). The price is serving all traffic from one process, and the scheduler still restarts whenever that worker restarts.

When One Scheduler Runs the Job Twice#

max_instances, coalesce, and misfire_grace_time act inside one scheduler. None of them can see another process.

max_instances (default 1) is counted in a dictionary held by each executor (executors/base.py, submit check). It stops a slow run from overlapping the next one in the same scheduler. Four workers with max_instances=1 can still run four copies at once.

Catch-up after a restart is the same-PID, back-to-back pattern. It needs a job that stayed in a persistent store while the scheduler was down and wasn't added again at startup: adding it again with replace_existing=True overwrites the stored row with a next run time computed from now (base.py). Even then, the 3.x default coalesce=True (base.py) submits only the latest missed time (base.py), so a burst of runs means coalesce=False together with a long misfire_grace_time or misfire_grace_time=None. Missed runs after downtime covers the defaults and whether catching up is what you want.

Copies of the job are the pattern that grows with each deploy. A job added without an id gets a random one (job.py). In a persistent store, adding it at startup therefore adds a new row each time, which the user guide flags in bold: you "MUST define an explicit ID for the job and use replace_existing=True or you will get a new copy of the job every time your application restarts!" (userguide.rst). Add both, then remove the old copies, which scheduler.get_jobs() lists under their random IDs.

Even with one scheduler, a run can repeat after a crash or a lock that changes hands. Idempotent scheduled jobs shows how to make a second run of the same occurrence do nothing.

Report Each Run to CronRadar#

Python SDK connections are temporarily unavailable (Python integration), so each job reports with HTTP check-ins. HTTP check-ins don't discover jobs automatically; give each job its own monitor key and its real schedule. Keep the API key in a CRONRADAR_API_KEY environment variable from your deployment's secret store, with a separate CronRadar application and key for each environment.

Python scheduler monitoring has the standard-library check-in helper, which uses HTTP Basic auth with the key as username and an empty password, a five-second timeout, and logs monitoring errors instead of raising them. It also has a wrapper that reports a start, then a completion or a failure, and re-raises the job's exception. For duplicate runs, change only how that wrapper builds its run_id: include the host and PID, so each run in the monitor's history shows which process ran it.

import os
import socket
import uuid


def new_run_id():
    # At most 128 characters; identifies the process, never the job's data.
    return f"{socket.gethostname()[:60]}-{os.getpid()}-{uuid.uuid4().hex[:12]}"

In that post's monitored_backup(), replace run_id = uuid.uuid4().hex with run_id = new_run_id(). The wrapper already sends the same run_id with the start and with the completion or failure, and the API accepts up to 128 characters for it (API reference).

If one expression feeds both CronTrigger.from_crontab() and the monitor's schedule, which CronRadar reads as five-field cron in UTC (schedule syntax), avoid numeric weekdays: from_crontab() in APScheduler 3.x treats 0 as Monday, while standard cron treats it as Sunday (cron trigger).

What this shows, and what it doesn't:

• Duplicates are visible, not alerted. CronRadar records every run it receives and doesn't alert on extra runs. Each run_id appears as its own run in the monitor's history, with its Execution ID, so four runs at the same scheduled time from four PIDs is the per-worker pattern above.
• Zero schedulers is alerted. This is the failure every fix above introduces: a dedicated process that didn't restart, or a lock holder that was recycled. No start arrives, the monitor shows warning once the scheduled time passes, and critical after the grace period, with an alert by email, Slack, webhook, Discord, or PagerDuty.
• A failed run alerts immediately. fail records the result failed and opens an incident.

Don't send an Idempotency-Key derived from the scheduled time to collapse the duplicates. A repeated key returns the original ping without reapplying state (API reference), so the dashboard would show one run while the job still ran twice.

Frequently Asked Questions#

Why does my APScheduler job run once per Gunicorn worker?#

Without --preload, each Gunicorn worker loads your app after it is forked, so a scheduler started at import or in an app factory starts once per worker: -w 4 means four schedulers. Without -w, the count comes from WEB_CONCURRENCY when it is set. Log os.getpid() when the scheduler starts to count them.

How do I run APScheduler with Gunicorn and multiple workers?#

Run the scheduler outside Gunicorn, as one BlockingScheduler process, which is what the APScheduler FAQ recommends. If it must live in the web app, start it only in the worker that wins an exclusive lock, and accept that it restarts whenever that worker does.

Does gunicorn --preload fix duplicate APScheduler jobs?#

It hides them. The scheduler thread starts in the master, and forked workers don't get a copy of it, so jobs run only in the master. Each later fork is a fork of a multi-threaded process, which Python's documentation calls unsafe, and a job added from a worker goes into a scheduler copy with no thread to run it.

Why does APScheduler run twice with flask run --debug?#

The debug reloader loads your app in a parent process, then again in a child process started with WERKZEUG_RUN_MAIN=true. With flask run --debug, start the scheduler only when not app.debug or is_running_from_reloader(), or run a dedicated scheduler process. That check fails under app.run(debug=True), where app.debug is still false when the factory runs; there, set FLASK_DEBUG=1 before the script starts or pass use_reloader=False.

Does max_instances=1 stop duplicate runs?#

Only inside one scheduler. Each process counts its own running instances, so it stops overlap of a slow run but not a second process starting the same job.