Celery Task Retries: self.retry, autoretry_for, and Final-Failure Alerts

Celery retries a task only when the task asks for it: raise self.retry(exc=exc) in an except block, or autoretry_for=(SomeError,) on the decorator. Both allow 3 retries, so 4 runs, wait 180 seconds between them unless you pass a countdown or set retry_backoff, and then re-raise the original exception, which is the point where task_failure fires; a retry never fires it. A worker process that dies mid-task takes a separate path, controlled by acks_late and task_reject_on_worker_lost. By default that path doesn't run the task again; when either path does rerun it, the task starts from its first line.

This guide covers Celery 5.6.3, the current release on PyPI (checked October 30, 2026); source links point to the v5.6.3 tag. Which failures deserve a retry, and how other schedulers retry, is in how to retry failed cron jobs. Flower, Prometheus, and worker health are in Celery task monitoring.

How self.retry() Works#

# billing/tasks.py
from celery import shared_task

from billing.ledger import LedgerUnavailable, push_ledger_batch


@shared_task(bind=True, max_retries=5, default_retry_delay=120)
def push_ledger(self, batch_id):
    try:
        push_ledger_batch(batch_id)
    except LedgerUnavailable as exc:
        # Overrides default_retry_delay for this attempt only.
        raise self.retry(exc=exc, countdown=60)

retry() publishes a new message with the same task ID to the same queue, then raises Retry, which the worker treats as a state change rather than an error (retrying). The raise in front is for readability, since retry() raises either way. Decorator options become attributes of the task class (base.py), so max_retries=5 above is the same as setting it on a Task subclass.

The defaults live on Task (task.py) and in the autoretry wrapper (autoretry.py):

When Celery Gives Up#

self.request.retries is 0 on the first run and goes up by one on each retry (request attributes). retry() gives up when request.retries + 1 exceeds max_retries (task.py), so the default allows 4 runs, and max_retries=5 allows 6. What it raises then depends on exc:

• With exc, it re-raises that exception, so the result backend, logs, and task_failure see LedgerUnavailable, not a Celery wrapper.
• Without exc, it raises MaxRetriesExceededError, whose message is Can't retry <task>[<id>] args:<args> kwargs:<kwargs>. That puts the task's arguments into every place the exception goes, so pass exc.

Three more rules from the same method:

• retry(max_retries=...) applies to that call only. It doesn't carry over to later attempts, and passing None means "use the default", so the only way to retry forever is max_retries=None on the task itself.
• Called as a plain function, retry() raises at once. It raises the exc you passed, or Retry if you passed none, instead of scheduling anything (task.py). Under apply() or task_always_eager, the retry runs immediately in the same process (task.py).
• If the retry can't be published, the retry is lost without a failure. When publishing the new message fails, for example because the broker is unreachable, retry() raises Reject(requeue=False) (task.py). The worker logs a warning, doesn't requeue the current message (request.py), and fires neither task_retry nor task_failure.

autoretry_for and Exponential Backoff#

autoretry_for wraps the task's run method. When the method raises a listed exception, the wrapper calls task.retry(exc=exc, **retry_kwargs) for you; Retry, Ignore, and anything in dont_autoretry_for pass through untouched (autoretry.py). An exception that isn't listed fails on the first attempt.

# billing/tasks.py
import os

import requests
from celery import shared_task
from requests.exceptions import ConnectionError as RequestsConnectionError
from requests.exceptions import Timeout


@shared_task(
    autoretry_for=(RequestsConnectionError, Timeout),
    max_retries=5,
    retry_backoff=60,       # Delay ceilings of 60, 120, 240, 480, then 600 seconds
    retry_backoff_max=600,  # The default, shown for clarity
)
def sync_supplier_prices():
    response = requests.post(os.environ["SUPPLIER_SYNC_URL"], timeout=30)
    # A 4xx or 5xx raises HTTPError, which isn't listed: it fails on the first attempt.
    response.raise_for_status()
    return response.status_code

HTTPError, ConnectionError, and Timeout are separate subclasses of RequestException in requests 2.34.2 (exceptions.py), and raise_for_status() raises HTTPError for any 4xx or 5xx status (models.py). Listing RequestException would retry all three, 4xx responses included. To retry a family except some members, list both: autoretry_for=(OSError,) with dont_autoretry_for=(FileNotFoundError,) retries every OSError except a missing file, which Python defines as an OSError subclass (exception hierarchy).

What the Backoff Computes#

With retry_backoff on, the wrapper sets the countdown from get_exponential_backoff_interval:

ceiling(n) = min(retry_backoff_max, factor * 2^n)   where n = request.retries (0 on the first failure)
delay(n)   = random integer from 0 to ceiling(n)    if retry_jitter (the default)
           = ceiling(n)                             otherwise

retry_backoff=True makes the factor 1, and a number is used as the factor in seconds, rounded down, with a minimum of 1. Jitter doesn't add noise around the ceiling: it picks a whole number of seconds anywhere from zero up to it, so the average wait is about half of it (retry_jitter).

Two rows deserve a second look. retry_backoff=True at the default max_retries finishes all its retries within seven seconds, too soon for most outages to clear, so pass a factor in seconds instead. And the first row disagrees with the documentation: the tasks guide says that with retry_backoff off "autoretries will not be delayed", but the 5.6.3 wrapper then passes no countdown, and retry() falls back to default_retry_delay, so automatic retries wait three minutes each. Set retry_kwargs={"countdown": 10} for a different fixed delay.

Every delay is a minimum. The retry message carries an ETA, the earliest time it may run, and Celery only guarantees it runs "at some time after the specified date and time" (ETA and countdown), when a worker is free.

acks_late and Worker Loss Are a Different Path#

retry() only handles exceptions the task raises. Celery's FAQ is blunt about the other case: retries are "for expected errors that is catch-able with the try block", and "if the task raises an exception it's still acknowledged!", while acks_late is for running the task again "if the worker (for some reason) crashes mid-execution" (retry or acks_late). The three acknowledgement settings and their defaults are in Celery task monitoring; this is how they interact with retries, from the worker's on_failure:

The lost-task row is from the workers guide; under acks_late the message hasn't been acknowledged yet, so it's redelivered to another worker. The last column is the only automatic rerun after a pool process dies, and it isn't counted as a retry: the broker delivers the same message again, and nothing in that path changes request.retries, so max_retries doesn't limit it. A task that reliably kills its own process gets requeued every time, the message loop the configuration reference warns about.

Long retry delays interact with redelivery too. The Redis caveats warn that "ETA/countdown/retry tasks where the time to execute exceeds the visibility timeout" are "executed again, and again in a loop". The visibility timeout is one hour by default on Redis and 30 minutes on SQS, so keep each retry delay well under it; retry_backoff_max at its default of 600 seconds does.

Make Each Attempt Safe to Run Again#

A retry and a redelivery both run the task from its first line; idempotent cron jobs covers keying each side effect so a second run doesn't repeat it. The task ID stays the same for every retry of one run, because retry() copies it into the new message, and a redelivery is the same message. That makes self.request.id a usable idempotency key for side effects within one run, such as the key on a payment API call.

It doesn't protect against two separate sends of the same work, such as two beats each sending the 06:00 run: those are two task IDs. For that, key on the period the run covers, as idempotent cron jobs shows with a runs table. Celery's own worker_deduplicate_successful_tasks (default off) covers one narrow case: it skips a redelivered late-acknowledged message whose task ID is already SUCCESS in a persistent result backend. A run that died halfway isn't SUCCESS, so it runs again in full.

Retries for Beat-Scheduled Tasks#

Each time an entry is due, beat calls apply_async for it (beat.py) without looking at earlier runs. Every occurrence is a new task with its own ID and its own retry budget, and the 03:00 run of an hourly task is sent on time even if the 02:00 run is still waiting for its fourth attempt. Three consequences:

• A long chain overlaps the next run. With five retries backing off to 600 seconds, a */15 task can have one occurrence's retry running beside, or after, the next occurrence. If the job writes "latest" values, an old retry that finishes last overwrites newer data. Keep the whole chain, retry waits plus run times, shorter than the interval, or prevent overlap with a lock as in preventing overlapping cron jobs.
• A retry inherits the entry's expires. A beat entry can set "options": {"expires": ...} so a backlog doesn't run stale copies. The retry message keeps the original expiry time, which by then is an absolute timestamp (amqp.py), and a worker marks an expired task REVOKED instead of running it (expiration), checking again when the task is about to execute (request.py). A retry that comes due after the expiry never runs, and the chain ends in task_revoked, not task_failure.
• A retry runs late, so "now" is the wrong clock. A nightly job that exports "yesterday" from datetime.now() exports the wrong day if a retry runs after midnight. Fix the period on the first attempt and pass it on, since retry() accepts replacement args and kwargs for the next message (task.py):

# reports/tasks.py
from datetime import datetime, timedelta, timezone

from celery import shared_task

from reports.export import ExportTargetUnavailable, upload_daily_export


@shared_task(bind=True, max_retries=4, default_retry_delay=300)
def export_daily_orders(self, day=None):
    # The first attempt picks the day; every retry receives it as an argument.
    if day is None:
        day = (datetime.now(timezone.utc).date() - timedelta(days=1)).isoformat()
    try:
        upload_daily_export(day)
    except ExportTargetUnavailable as exc:
        raise self.retry(exc=exc, kwargs={"day": day})

kwargs replaces the original keyword arguments rather than merging with them (task.py), so pass all of them. A first attempt that itself starts late, after a backlog, still computes from its own start time; idempotent cron jobs covers naming the period by the work instead of the clock.

Which Hooks Fire on a Retry and Which on the Final Failure#

From the tracer (trace.py) and the worker's request handling (request.py):

So task_failure is the final-failure hook: it fires once per run when Celery stops retrying, never for a retry, and it's the only signal or task method that also covers a dead pool process. A Task.on_failure method on a base class misses that case, because it runs in the process that died; Celery's docs point to the worker-side Request.on_failure, which runs in the main process, as a way to detect failures that Task.on_failure doesn't (custom requests). after_return is skipped for RETRY (handlers), so it runs only at the end of a run, though not for a dead pool process.

For per-attempt visibility, use task_retry or task_postrun. task_retry also fires, without reason or einfo, when a running task is cancelled and retried, so a handler must not assume those arguments exist (task_retry).

Report Only the Final Failure to CronRadar#

Celery's hooks run only while workers run and beat sends. A schedule-based check from outside covers the rest: no start by the expected time means the run never happened, whichever part failed. CronRadar's Celery SDK is unavailable (checked September 20, 2026; the Celery integration docs carry the current notice), so report with HTTP check-ins. HTTP check-ins don't discover beat schedules: each task needs its own monitor key and its real schedule.

The signal reporter in Celery task monitoring already reports a start for every attempt, nothing for a retry, and a fail only when Celery gives up. This version changes three things for retries:

• The fail handler takes task_id from the task_failure signal. When a pool process dies, the worker's main process sends task_failure (request.py), and that process never pushed this run's request, so sender.request is an empty default context (task.py) and sender.request.id can't be the run_id.
• A task_revoked handler reports a retry that expired, which ends the chain without task_failure.
• Each mapping carries a grace period next to the schedule.

The request code is the standard-library helper from Celery beat not running, with an attempt parameter added: HTTP Basic auth with the key from CRONRADAR_API_KEY, and errors logged instead of raised.

# proj/monitoring.py
import base64
import logging
import os
import urllib.parse
import urllib.request

from celery.signals import task_failure, task_prerun, task_revoked, task_success

log = logging.getLogger(__name__)

# Celery task name -> (monitor key, the beat entry's schedule as five-field cron in UTC,
# grace period in seconds). Schedule and grace apply when a check-in creates the monitor.
MONITORED_TASKS = {
    "reports.tasks.export_daily_orders": ("export-daily-orders", "30 2 * * *", 900),
    "billing.tasks.sync_supplier_prices": ("sync-supplier-prices", "*/15 * * * *", 300),
}


def check_in(task_name, event, *, run_id, attempt=None, message=None):
    """Report one lifecycle event to CronRadar. Never raises into Celery."""
    monitor = MONITORED_TASKS.get(task_name)
    if monitor is None:
        return
    key, schedule, grace_period = monitor
    params = {
        "schedule": schedule,
        "gracePeriod": grace_period,
        "run_id": run_id,
        "attempt": attempt,
        "message": message,
    }
    query = urllib.parse.urlencode(
        {name: value for name, value in params.items() if value is not None},
        quote_via=urllib.parse.quote,
    )
    url = f"https://cron.life/api/v1/ping/{urllib.parse.quote(key, safe='')}/{event}?{query}"
    try:
        token = base64.b64encode(f"{os.environ['CRONRADAR_API_KEY']}:".encode()).decode()
        request = urllib.request.Request(url, data=b"", method="POST")
        # Unlike add_header(), this header is not copied to a redirect.
        request.add_unredirected_header("Authorization", f"Basic {token}")
        with urllib.request.urlopen(request, timeout=5):
            pass
    except Exception as error:
        # A monitoring problem must never fail the task or change its result.
        log.warning("CronRadar %s for %s not recorded (%s)", event, key, type(error).__name__)


@task_prerun.connect
def on_attempt_start(sender=None, task_id=None, task=None, **kwargs):
    # Every attempt shares the task ID; request.retries is 0 on the first.
    check_in(sender.name, "start", run_id=task_id, attempt=task.request.retries)


@task_success.connect
def on_success(sender=None, **kwargs):
    check_in(sender.name, "complete", run_id=sender.request.id, attempt=sender.request.retries)


@task_failure.connect
def on_final_failure(sender=None, task_id=None, exception=None, **kwargs):
    # Never sent for a retry. For a dead pool process it comes from the worker's
    # main process, where sender.request isn't this run, so use task_id.
    # The exception type only: never arguments, payloads, or log output.
    check_in(sender.name, "fail", run_id=task_id, message=type(exception).__name__)


@task_revoked.connect
def on_revoked(sender=None, request=None, expired=False, **kwargs):
    # A retry that outlived the entry's expires ends here, not in task_failure.
    if expired:
        check_in(sender.name, "fail", run_id=request.id, message="Expired")

Load it with app.conf.imports = ["proj.monitoring"], which imports the module when the worker starts. With the prefork pool, the task_failure call for a dead pool process and the task_revoked call run in the worker's main process (custom requests, task_revoked), so a slow check-in there blocks the main process rather than one pool process. timeout=5 limits each blocking operation, such as the connection attempt, not the check-in as a whole (urlopen).

What CronRadar records:

• Each attempt is part of one run. The task ID is the run_id and request.retries is the zero-based attempt (HTTP API reference). CronRadar groups the events that share a run_id into one run and counts its attempts from the distinct attempt values on its starts. It never retries your work itself.
• A retry opens no incident. The failed attempt sends no fail, and the next attempt's start updates the same run.
• The final failure alerts immediately. fail marks the monitor critical with the result failed and opens an incident, alerting through email, Slack, webhook, Discord, or PagerDuty. The alert arrives when the chain ends: for export_daily_orders, at least 20 minutes after the first error (four 300-second waits) plus run times. A later successful run records a recovery.
• Grace covers the first attempt only. Every check-in, each attempt's start included, moves the next expected time to the schedule's next occurrence, so the grace period has to cover how late the first attempt can start, not the retry chain. How a missed run moves from warning to critical is in Celery beat not running.
• A run that ends without a hook stays open. A lost retry message or a killed worker leaves a start with no result, flagged as stuck if no check-in arrives by the next expected time plus grace. If the workers are otherwise fine, the next occurrence's start resets that window, and the lost run stays in the run history labeled running, with no result.
• A chain longer than the interval can alert late. If an old run's final fail arrives after a newer run succeeded, it still opens an incident, for the older run. Short chains avoid that.

Each mapping's schedule is its beat entry written as five-field cron in UTC (schedule syntax); Celery beat not running has a table translating crontab() entries, including ones in other time zones.

Frequently Asked Questions#

How many times does Celery retry a task by default?#

Zero, unless the task calls self.retry() or lists exceptions in autoretry_for. When it does, max_retries defaults to 3, so a task that keeps failing runs 4 times. max_retries=None retries forever.

What is the default delay between Celery retries?#

180 seconds, from default_retry_delay, for both self.retry() without a countdown and autoretry_for without retry_backoff. With retry_backoff=True, the ceilings are 1, 2, and 4 seconds, and the default jitter picks a random delay from zero up to each one.

When does Celery raise MaxRetriesExceededError?#

When the retry limit is reached and retry() was called without exc. With exc, Celery re-raises that exception instead, which is what autoretry_for always does. The MaxRetriesExceededError message includes the task's arguments, so pass exc to keep them out of logs and alerts.

Does task_failure fire on every retry?#

No. A retry fires task_retry and on_retry. task_failure fires once, when Celery stops retrying or the exception isn't one it retries, and also when the pool process running the task dies and the message isn't requeued. It doesn't fire for a retry that expired, which sends task_revoked.

Does acks_late retry a task that raised an exception?#

No. With acks_late, a task that raises is still acknowledged by default, and only retry() or autoretry_for runs it again. acks_late matters when the worker dies mid-task: with task_reject_on_worker_lost as well, a dead pool process puts the message back in the queue, without counting a retry.