Django Cron Jobs: How to Monitor Scheduled Tasks

To monitor Django cron jobs, have each scheduled task report to a monitor that knows its schedule: a start when it begins, a completion when it succeeds, and a failure when it raises. The monitor alerts on the failure, and on a run that doesn't arrive on time. That second alert catches what logs and error trackers can't: a beat process that stopped, a crontab entry lost in a deploy, or a worker killed mid-task.

Django doesn't schedule tasks itself, so where you add the reports depends on the scheduler that runs them. This guide covers the common choices, the failures each one hides, and working code for management commands and Celery tasks.

How Django Developers Schedule Background Tasks#

Django projects usually use one of five approaches, each with different monitoring implications.

Celery beat with django-celery-beat stores periodic task schedules in your database and lets you manage them in the Django admin. You need a broker such as Redis or RabbitMQ, plus separate worker and beat processes.

APScheduler with django-apscheduler stores jobs in the Django database through the ORM. Its README warns that each web server worker process can start its own scheduler, which leads to missed or duplicated runs, and recommends starting a single scheduler from a dedicated management command.

Management commands run by system cron remain the simplest approach: write a command, add a crontab entry, and you're done. There's no built-in visibility, and cron runs the command with a different user, directory, and environment than your shell; see why a cron job isn't running. Running a Python script from cron covers the virtualenv interpreter, paths, and logs.

Django Q2, a fork of Django Q, supports scheduled, cron, and repeated tasks with Django admin integration. It can use Redis, the Django ORM, Amazon SQS, MongoDB, or IronMQ as its broker.

Huey declares periodic tasks with @periodic_task(crontab(...)) from huey.contrib.djhuey, uses Redis by default (SQLite, PostgreSQL, and file storage are also available), and runs its consumer with manage.py run_huey.

If you already run Celery, beat is the natural choice. The examples below focus on Celery and management commands; the reporting pattern is the same for all five.

Why Silent Failures Are the Biggest Threat#

When a Django view raises an exception, your error tracker captures it. A scheduled task that never starts raises nothing. These failure modes catch teams off guard:

Beat stops, or two beats run. Celery requires a single scheduler per schedule. If beat stops, no tasks are sent and nothing fails. If two run, tasks run twice.

A time zone change doesn't reset the database schedule. Celery's periodic tasks guide notes that the django-celery-beat database scheduler doesn't reset when time zone settings change. Reset it yourself with PeriodicTask.objects.update(last_run_at=None).

A killed worker loses its task. By default, Celery acknowledges a task just before running it. If the worker process is killed by the out-of-memory killer or a KILL signal, Celery's workers guide says currently executing tasks are lost unless they use acks_late. The task neither completes nor fails.

Every web worker starts a scheduler. Starting APScheduler inside the web application starts one scheduler per web server worker process, and APScheduler can't coordinate them.

Worker memory grows until the process is killed. Celery's worker_max_tasks_per_child and worker_max_memory_per_child replace pool processes after a number of tasks or above a memory threshold. Neither has a limit by default.

Better logging doesn't solve these. You need monitoring that alerts when an expected event doesn't happen.

Monitoring Options for Django Scheduled Tasks#

Several services watch scheduled tasks from outside your application:

• Sentry Crons suits teams already using Sentry. With monitor_beat_tasks=True, Sentry creates monitors for Celery beat tasks automatically; its docs name the default beat scheduler and RedBeat, for crontab-compatible schedules. The @sentry_sdk.monitor decorator covers other tasks. Sentry Crons vs a dedicated cron monitor compares the two approaches.
• Cronitor's Python SDK discovers Celery tasks, including beat tasks, with cronitor.celery.initialize(app). Its README notes that django-celery-beat isn't supported yet; the @cronitor.job decorator covers those tasks and management commands.
• Healthchecks.io is an open-source monitoring service written in Python and Django, available hosted or self-hosted. Each check has a ping URL with /start and /fail variants.
• CronRadar monitors scheduled jobs over HTTP. The first check-in that carries a cron expression creates the monitor, start and terminal events share a run_id, and alerts go to email, Slack, a webhook, Discord, or PagerDuty. Python and Celery SDK setup is temporarily unavailable, so Django projects report over HTTP, as shown below.

All four work the same way: the task checks in, and the service alerts when a check-in is late or reports a failure.

Implementing Monitoring: Code Examples#

A Reporting Helper#

These examples use CronRadar's HTTP API with HTTP Basic authentication. Store the ingestion key in CRONRADAR_API_KEY through your secret manager, with a separate CronRadar application and key for each environment. The helper never raises, so a monitoring outage can't fail the task:

# monitoring/cronradar.py
import logging
import os
from urllib.parse import quote

import requests

log = logging.getLogger(__name__)

def report(monitor_key, event=None, *, schedule, run_id=None, attempt=None, message=None):
    """Send one check-in: event is "start", "complete", "fail", or None for a heartbeat."""
    url = f"https://cron.life/api/v1/ping/{quote(monitor_key, safe='')}"
    if event:
        url += f"/{event}"
    # requests leaves out parameters whose value is None.
    params = {"schedule": schedule, "run_id": run_id, "attempt": attempt}
    if message:
        params["message"] = message[:1000]
    try:
        response = requests.post(
            url,
            params=params,
            auth=(os.environ["CRONRADAR_API_KEY"], ""),
            timeout=5,
        )
        response.raise_for_status()
    except Exception as error:
        # Log and continue: monitoring must not fail or change the task.
        log.warning(
            "CronRadar %s for %s not recorded (%s)",
            event or "heartbeat", monitor_key, type(error).__name__,
        )

Send the exception type as the failure message, never arguments, payloads, or log output. The same pattern works with Healthchecks.io if you swap the URL and authentication: its ping URL accepts /start and /fail suffixes and an rid run ID (Pinging API).

Monitoring Django Management Commands#

A management command run by cron reports from handle() and re-raises its own exception, so cron still sees a non-zero exit:

# audit/management/commands/prune_audit_log.py
import uuid

from django.core.management.base import BaseCommand

from audit.retention import prune_expired_entries
from monitoring.cronradar import report

MONITOR_KEY = "prune-audit-log"
SCHEDULE = "15 3 * * *"  # Same as the crontab entry, in UTC

class Command(BaseCommand):
    help = "Delete audit log entries past their retention period"

    def handle(self, *args, **options):
        run_id = uuid.uuid4().hex
        report(MONITOR_KEY, "start", schedule=SCHEDULE, run_id=run_id)
        try:
            deleted = prune_expired_entries()
        except Exception as error:
            report(MONITOR_KEY, "fail", schedule=SCHEDULE, run_id=run_id, message=type(error).__name__)
            raise
        report(MONITOR_KEY, "complete", schedule=SCHEDULE, run_id=run_id)
        self.stdout.write(f"Deleted {deleted} audit log entries")

And the crontab entry, with absolute paths and output kept in a log:

15 3 * * * cd /srv/app && /srv/app/.venv/bin/python manage.py prune_audit_log >> /var/log/app/prune_audit_log.log 2>&1

Cron usually evaluates entries in the server's local time zone, while CronRadar evaluates schedules in UTC. If the server isn't on UTC, convert the expression. The cron expression guide explains the five fields, and the schedule docs list what CronRadar accepts.

Celery Task Base Class#

For Celery tasks, a base class reports every run without touching task code. before_start requires Celery 5.2 or later:

# inventory/tasks.py
from celery import Task, shared_task

from inventory.models import Supplier
from monitoring.cronradar import report

class MonitoredTask(Task):
    """Reports each run to CronRadar. Set monitor_key and cron_schedule on the task."""

    monitor_key = None
    cron_schedule = None

    def _report(self, event, task_id, message=None):
        if self.monitor_key:
            report(
                self.monitor_key,
                event,
                schedule=self.cron_schedule,
                run_id=task_id,  # Celery keeps the task ID across retries
                attempt=self.request.retries,
                message=message,
            )

    def before_start(self, task_id, args, kwargs):
        self._report("start", task_id)

    def on_success(self, retval, task_id, args, kwargs):
        self._report("complete", task_id)

    def on_failure(self, exc, task_id, args, kwargs, einfo):
        # Runs when the task fails for good; a retry calls on_retry instead.
        self._report("fail", task_id, message=type(exc).__name__)

@shared_task(base=MonitoredTask, monitor_key="sync-inventory", cron_schedule="*/15 * * * *")
def sync_inventory():
    """Sync inventory levels with suppliers."""
    for supplier in Supplier.objects.active():
        supplier.sync_inventory()

Celery sets any keyword argument passed to the task decorator as an attribute of the task class, which is how monitor_key and cron_schedule reach the hooks. Keep the beat schedule identical:

# settings.py (with app.config_from_object("django.conf:settings", namespace="CELERY"))
from celery.schedules import crontab

CELERY_TIMEZONE = "UTC"
CELERY_BEAT_SCHEDULE = {
    "sync-inventory": {
        "task": "inventory.tasks.sync_inventory",
        "schedule": crontab(minute="*/15"),
    },
}

With django-celery-beat's database scheduler, define the same crontab in the admin instead. The Celery monitoring guide shows the same reporting done with Celery signals, plus Flower and Prometheus for the worker side.

Sentry Crons with Celery Beat#

If you use Sentry, configure it once in settings.py:

# settings.py
import os

import sentry_sdk
from sentry_sdk.integrations.celery import CeleryIntegration
from sentry_sdk.integrations.django import DjangoIntegration

sentry_sdk.init(
    dsn=os.environ["SENTRY_DSN"],
    integrations=[
        DjangoIntegration(),
        CeleryIntegration(
            monitor_beat_tasks=True,
            exclude_beat_tasks=["celery.backend_cleanup"],
        ),
    ],
)

For grace periods, runtime limits, or tasks beat monitoring doesn't cover, use the decorator below Celery's:

# orders/tasks.py
import sentry_sdk
from celery import shared_task

monitor_config = {
    "schedule": {"type": "crontab", "value": "0 4 * * *"},
    "timezone": "UTC",
    "checkin_margin": 10,  # Minutes late before the check-in counts as missed
    "max_runtime": 30,  # Minutes a run may take before it counts as failed
    "failure_issue_threshold": 3,  # Consecutive failures before an issue opens
    "recovery_threshold": 2,  # Successful check-ins before the issue resolves
}

@shared_task
@sentry_sdk.monitor(monitor_slug="nightly-order-sync", monitor_config=monitor_config)
def sync_orders_to_warehouse():
    """Sync pending orders to the warehouse system."""
    ...

See Sentry's Crons configuration for every option.

Cronitor with Celery#

Cronitor's SDK discovers Celery tasks when you initialize it after configuring the app:

# myproject/celery.py
import os

import cronitor.celery
from celery import Celery

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")

app = Celery("myproject")
app.config_from_object("django.conf:settings", namespace="CELERY")
app.autodiscover_tasks()

cronitor.celery.initialize(app, api_key=os.environ["CRONITOR_API_KEY"])

Because that doesn't cover django-celery-beat yet, decorate those tasks instead. Keep @shared_task on top: Celery registers the function it decorates, so a decorator above it wouldn't run in the worker.

# billing/tasks.py
import os

import cronitor
from celery import shared_task

cronitor.api_key = os.environ["CRONITOR_API_KEY"]

@shared_task
@cronitor.job("send-invoices", attributes={"schedule": "0 6 * * *", "grace_seconds": 300})
def send_invoices():
    """Email the day's invoices."""
    ...

Adding Timeout Protection#

A task that hangs never fails, and it never completes either. Give Celery tasks time limits in the decorator:

# inventory/tasks.py, continued: MonitoredTask is defined above.
from inventory.etl import extract_from_sources, load_to_warehouse, transform_records

@shared_task(
    base=MonitoredTask,
    monitor_key="nightly-etl",
    cron_schedule="0 1 * * *",
    soft_time_limit=1500,  # Raises SoftTimeLimitExceeded inside the task
    time_limit=1800,  # Kills and replaces the worker process
)
def run_etl_pipeline():
    """Extract, transform, and load daily data."""
    extract_from_sources()
    transform_records()
    load_to_warehouse()

The soft limit raises an exception the task can clean up after, and on_failure reports it. The hard limit kills the worker process, so code inside the task doesn't get to run; if no failure is reported, the monitor still sees a start that never finished. For management commands, wrap the cron entry in timeout 30m.

Best Practices for Production Monitoring#

Configure Meaningful Grace Periods#

Set grace periods from actual task behavior, not optimistic estimates. If a task usually finishes in 2 minutes but sometimes takes 10, allow 15. In CronRadar, pass gracePeriod in seconds (minimum 60); Sentry uses checkin_margin in minutes and Cronitor uses grace_seconds.

Monitor the Monitor#

Beat itself can stop. Add a heartbeat task that beat sends every five minutes:

# inventory/tasks.py
from celery import shared_task

from monitoring.cronradar import report

@shared_task
def beat_heartbeat():
    """A missing check-in means beat, the broker, or the workers stopped."""
    report("celery-beat-heartbeat", schedule="*/5 * * * *")

Add it to CELERY_BEAT_SCHEDULE with crontab(minute="*/5"). If the heartbeat stops, scheduled work has stopped somewhere between beat and the workers.

Track Duration Trends#

A task that succeeds but takes ten times longer than usual often comes before a failure. When you send start and complete events, CronRadar records each run's duration; Sentry and Cronitor track duration too.

Separate Environments#

Give staging and production separate monitors so a staging failure doesn't alert the production on-call. In CronRadar, create a separate application and API key for each environment. In Cronitor, set cronitor.environment or CRONITOR_ENVIRONMENT; in Sentry, pass environment to sentry_sdk.init().

Troubleshooting Common Issues#

Task Shows as Running but Never Completes#

Symptoms: The monitor shows a run in progress indefinitely, with no completion or failure.

Causes: The worker process was killed (out of memory, KILL, or a hard time limit); a network or database call without a timeout; an infinite loop.

Solutions:

1. Set soft_time_limit and time_limit on the task, and a timeout on every network call.
2. For tasks that are safe to run twice, enable task_acks_late and task_reject_on_worker_lost so a task whose worker dies is re-queued. Celery warns that this can cause message loops.
3. Send a start event, so the monitor can flag a run that started and never finished.

Duplicate Task Executions#

Symptoms: A task runs more than once per scheduled interval; the database shows duplicate records.

Causes and fixes: usually two beat processes, or Redis redelivering a task that outlives the visibility timeout. Celery beat running twice covers both and the settings that prevent them. Make tasks safe to repeat as well, with upserts or idempotency keys.

Tasks Not Starting After Deployment#

Symptoms: Beat is running but tasks never execute; there are no errors in the logs.

Causes: The worker hasn't imported the task's module (Celery registers a task only when its module is imported); beat wasn't restarted after a schedule change; time zone settings changed under django-celery-beat.

Solutions:

1. List the tasks each worker has registered: celery -A myproject inspect registered.
2. Restart beat and the workers on every deployment.
3. After changing time zone settings with django-celery-beat, run PeriodicTask.objects.update(last_run_at=None).

Monitoring Pings Fail but Task Succeeds#

Symptoms: The task completes, but the monitor shows it late or missed.

Causes: Outbound HTTPS is blocked; the API key is missing or wrong (HTTP 401); the per-key limit of 60 requests a minute or 1,000 an hour was reached (HTTP 429); the check-in went to a different monitor key.

Solutions:

1. Keep the five-second timeout. A longer one only delays the task during a network problem; the helper already logs the failure and lets the task continue.
2. Search the worker logs for CronRadar ... not recorded warnings and the exception type.
3. Test outbound connectivity from the worker host: curl --max-time 5 https://cron.life/alive.

Frequently Asked Questions#

How do I run cron jobs in Django?#

Django has no built-in scheduler. Write a management command and run it from system cron, or use a scheduler library: Celery beat (with django-celery-beat to manage schedules in the admin), django-apscheduler, Django Q2, or Huey. Run exactly one scheduler process for each set of jobs.

How do I know if a Django scheduled task ran?#

Have the task report its start and completion to a monitor that knows its schedule. The monitor alerts on a reported failure and on a run that doesn't arrive in time. Logs only show runs that happened; they can't show the one that never started.

Can I use django-apscheduler with Gunicorn?#

Yes, but don't start the scheduler inside the web process. Each Gunicorn worker would start its own scheduler, which can duplicate or miss runs. Start one scheduler from a dedicated management command under a process supervisor. APScheduler running twice explains why runserver and AppConfig.ready() start extra schedulers too. Python scheduler monitoring covers APScheduler's own failure modes.

Does CronRadar have a Django package?#

No. CronRadar's Python and Celery SDK setup is temporarily unavailable, so Django projects report over HTTP with a helper like the one above, giving each task its own monitor key and schedule. The Celery integration docs carry the current availability notice.

Summary#

Django scheduled tasks will fail; what matters is whether a monitor or a user tells you first. Monitoring needs three pieces: a scheduler you can instrument, a service that knows each task's schedule, and start, completion, and failure reports that never break the task.

If you already use Sentry, Sentry Crons ties cron alerts to your error data. Cronitor discovers Celery tasks automatically outside django-celery-beat. Healthchecks.io is open source and self-hostable. CronRadar creates monitors from HTTP check-ins that carry a schedule; start with the reporting helper above. The specific tool matters less than having one: a task that runs unsupervised can stop without anyone noticing.