Hangfire Automatic Retry: Configure Attempts and Alert on Failure
Contents
Hangfire 1.8 retries a failed background job 10 times by default, then leaves it in the Failed state. The retries come from AutomaticRetryAttribute, a filter Hangfire registers globally, and the delays between them grow from seconds to almost two hours, so a job that keeps failing takes over four hours to give up. Change the policy with [AutomaticRetry(Attempts = 3)] on a job method or class, or by replacing the global filter. For recurring jobs, the same setting decides whether old runs collide with new ones and how late a final-failure alert arrives.
Everything below was checked against the source of Hangfire 1.8.25, the current release on October 24, 2026.
What AutomaticRetry Does by Default#
Hangfire adds new AutomaticRetryAttribute() to GlobalJobFilters.Filters at startup, so every job gets it unless something overrides it. Its constructor sets these defaults:
| Property | Default | Effect |
|---|---|---|
Attempts |
10 |
Retries after the first failure |
DelaysInSeconds |
not set | Delays come from a built-in formula |
OnAttemptsExceeded |
Fail |
The job stays in the Failed state |
LogEvents |
true |
A warning per retry and an error when retries run out |
OnlyOn, ExceptOn |
not set | Every exception is retried |
Ten Retries Mean Eleven Runs#
Attempts counts retries, not executions. The filter stores a RetryCount job parameter and schedules another run while RetryCount + 1 is no greater than Attempts, so the default allows one run plus ten retries, and Attempts = 3 allows four runs in total.
The counter belongs to the background job, and nothing in Hangfire 1.8.25 resets it: RetryCount is read and written only by the retry filter. If you requeue a job from the dashboard after its retries ran out and it fails again, it goes straight back to Failed.
The Delay Formula#
Without DelaysInSeconds, the delay before retry n is computed by DefaultDelayInSecondsByAttemptFunc as Math.Pow(attempt - 1, 4) + 15 + random.Next(30) * attempt:
delay(n) = (n - 1)^4 + 15 + r * n seconds, where r is a random integer from 0 to 29The fourth-power term takes over quickly:
| Retry | Delay before it | Elapsed since the first failure |
|---|---|---|
| 1 | 15–44 s | 15–44 s |
| 2 | 16–74 s | 31 s–2 min |
| 3 | 31–118 s | 1–4 min |
| 4 | 1.6–3.5 min | 2.6–7.5 min |
| 5 | 4.5–6.9 min | 7–14 min |
| 6 | 10.7–13.6 min | 18–28 min |
| 7 | 22–25 min | 40–53 min |
| 8 | 40–44 min | 1 h 20 min–1 h 37 min |
| 9 | 68–73 min | 2 h 28 min–2 h 50 min |
| 10 | 1 h 50 min–1 h 54 min | 4 h 18 min–4 h 45 min |
The elapsed column counts delays only; each attempt's run time comes on top. Each delay is also a minimum: a waiting retry sits in the Scheduled state, on the dashboard's Retries page, until the delayed-job scheduler enqueues it (it polls every 15 seconds by default) and a worker is free.
Which Failures Are Retried#
Any exception the job method throws is retried, and so is an exception from a server filter wrapped around the job. Hangfire unwraps its own JobPerformanceException before building the Failed state, so the retry filter sees the type your code threw (Worker.cs).
Two interruptions don't count as failures and don't use up a retry:
- Server shutdown. An
OperationCanceledExceptionraised because the server is stopping puts the job back in its queue to run after restart, as the cancellation token docs describe. - A state change while running. If the job is deleted or requeued while it's processing, Hangfire records no failure for that run.
A job that hangs never throws, so it is never retried; it stays in Processing. Why a Hangfire job gets stuck in Processing covers that case.
Where the Job Ends Up#
When the last retry fails, the job stays in the Failed state with its exception, and with LogEvents on, the filter logs an error. Failed jobs don't expire: they stay in storage until you requeue or delete them.
With OnAttemptsExceeded = AttemptsExceededAction.Delete, the job moves to Deleted instead. Since 1.8.0, the Deleted state keeps the exception (1.8.0 release notes), and like any final state it expires after the storage's JobExpirationTimeout, one day by default.
Configure AutomaticRetry Per Job or Globally#
On a Method, Class, or Interface#
using Hangfire;
[AutomaticRetry(Attempts = 3)]
public class ReportJobs
{
// Uses the class attribute: one run plus three retries.
public void BuildDailySummary()
{
Console.WriteLine("Building the daily summary");
}
// The method attribute replaces the class attribute for this method.
[AutomaticRetry(Attempts = 0, OnAttemptsExceeded = AttemptsExceededAction.Delete)]
public void EmailDailySummary()
{
Console.WriteLine("Emailing the daily summary");
}
}When several AutomaticRetry filters apply to one job, Hangfire keeps one. It sorts filters by Order, then by scope (global, then class, then method), and drops all but the last filter of each type that doesn't allow duplicates (JobFilterProviderCollection.cs). With equal Order values, a method attribute beats a class attribute, which beats the global filter. The winner replaces the others outright: any property you don't set on it takes the attribute's default, not the class or global value.
Hangfire reads attributes from the type named in the job expression. If you register RecurringJob.AddOrUpdate<IInvoiceJobs>(...), the attribute belongs on the interface method, because the implementing class is never inspected (job filter docs).
Replace the Global Default#
using Hangfire;
builder.Services.AddHangfire(config =>
{
config.UseSqlServerStorage(builder.Configuration.GetConnectionString("Hangfire"));
// Remove the built-in instance, then register the policy every job should get.
GlobalJobFilters.Filters.Remove<AutomaticRetryAttribute>();
config.UseFilter(new AutomaticRetryAttribute
{
Attempts = 3,
DelaysInSeconds = new[] { 30, 300, 1800 },
});
});
builder.Services.AddHangfireServer();UseFilter adds the instance to GlobalJobFilters.Filters, and Remove<T>() was added in 1.8.0. The documented GlobalJobFilters.Filters.Add(new AutomaticRetryAttribute { Attempts = 5 }) from Dealing with Exceptions also works, because the instance added later wins the tie, but it leaves two global instances in the collection. Removing the built-in one first keeps exactly one.
Choose the Delays#
DelaysInSeconds replaces the formula. When there are more attempts than entries, the last entry repeats, so Attempts = 5 with { 60, 300, 900 } waits 60, 300, 900, 900, and 900 seconds. A delay of 0 re-enqueues the job immediately instead of scheduling it.
For a computed back-off, set DelayInSecondsByAttemptFunc, which receives the retry number. Attribute arguments can't be delegates, so this only works on an instance you register globally, and DelaysInSeconds takes precedence when both are set:
config.UseFilter(new AutomaticRetryAttribute
{
Attempts = 5,
// 30 s, 60 s, 120 s, 240 s, 480 s
DelayInSecondsByAttemptFunc = attempt => 30 * (1 << (int)(attempt - 1)),
});Fail or Delete When Retries Run Out#
Fail keeps the job on the Failed page with its exception, where someone can requeue it after fixing the cause. Use it when a failed run still needs to happen. Delete discards the job once retries run out, for work that has no value later, such as a reminder that's pointless an hour late. The job's reason then reads "Exceeded the maximum number of retry attempts.", or "Retries were disabled for this job." when Attempts = 0.
Attempts = 0 turns off retries, not repeats. The attribute's own source notes that a job can still run more than once through re-queue on shutdown and the other compensation that gives Hangfire its at-least-once guarantee.
Retry Only Some Exceptions#
OnlyOn (added in 1.8.0) retries only the listed exception types and their subclasses. ExceptOn (added in 1.8.15) retries everything except them:
using System.Net.Http;
using Hangfire;
public class SupplierJobs
{
// Retry network trouble only. On .NET 5 and later, HttpClient reports its own
// timeout as an OperationCanceledException that wraps a TimeoutException.
[AutomaticRetry(Attempts = 4,
OnlyOn = new[] { typeof(HttpRequestException), typeof(OperationCanceledException) })]
public Task PullPricesAsync() => Task.CompletedTask;
// Retry everything except bad input, which fails the same way every time.
[AutomaticRetry(Attempts = 4,
ExceptOn = new[] { typeof(ArgumentException), typeof(FormatException) })]
public Task ImportPriceFileAsync() => Task.CompletedTask;
}The filter compares only the outer exception's type. That matters for HttpClient, whose timeout on .NET 5 and later is an OperationCanceledException with a TimeoutException inside (HttpClient.SendAsync), so OnlyOn = new[] { typeof(TimeoutException) } would not retry it.
An exception the filter skips goes straight to Failed. The filter logs nothing for it and doesn't apply OnAttemptsExceeded, so a skipped exception ends in Failed even when you configured Delete.
Turn Off Retry Logging#
LogEvents = false silences the filter's warning on each retry and its error when retries run out. Leave it on unless the job logs its own failures: Hangfire's worker doesn't log job exceptions itself, so with it off a failure shows up only in the dashboard.
Retries Can Overlap Later Recurring Runs#
Each Occurrence Gets Its Own Retry Budget#
A recurring job is a schedule that creates an ordinary background job at each occurrence, tagged with RecurringJobId and Time parameters (RecurringJobExtensions.cs). AutomaticRetry applies to each one separately with a fresh RetryCount, and the recurring scheduler creates the next occurrence without checking whether the previous one is still retrying.
With default delays and ignoring run times, retries of one failed occurrence start after the next occurrence has been created:
| Schedule | Default retries that start after the next occurrence |
|---|---|
| Every 5 minutes | Retries 5–10, sometimes 4 |
| Every 15 minutes | Retries 6–10 |
| Hourly | Retries 8–10 |
| Daily | None, unless attempts run for hours or delays are longer |
An outage can line them up exactly. When the server comes back, the delayed-job scheduler enqueues every retry whose time has passed, and the recurring scheduler, in its default Relaxed misfire mode, creates one run for the occurrences it missed. Yesterday's retry and today's run can start within seconds of each other.
Order is a second problem. If the 01:00 run of an hourly job finally succeeds on its ninth retry, sometime between 03:28 and 03:50, it finishes after the 02:00 and 03:00 runs. If it writes "latest" values, it overwrites newer data with older data.
Repeat-Safe Work and the Occurrence Time#
A retry runs the method again from the first line. Whatever the failed attempt already did, such as emails sent, rows inserted, or a charge captured, happens again unless the job checks first. Make the work safe to repeat, or set Attempts = 0 for that method. Preventing errors in scheduled automation covers idempotency and bounded retries in general.
Time windows need the same care. A job that processes "the previous hour" based on DateTime.UtcNow processes the wrong hour when a retry runs three hours late. Read the Time parameter instead, which the recurring scheduler sets to the occurrence's Unix time in seconds and which survives every retry:
using Hangfire;
using Hangfire.Server;
public class OrderSyncJob
{
public void SyncPreviousHour(PerformContext? context)
{
var occurrence = context?.GetJobParameter<long>("Time") ?? 0;
var scheduledAt = occurrence > 0
? DateTimeOffset.FromUnixTimeSeconds(occurrence)
: DateTimeOffset.UtcNow; // Enqueued directly, not by the recurring scheduler.
var windowEnd = new DateTimeOffset(
scheduledAt.Year, scheduledAt.Month, scheduledAt.Day, scheduledAt.Hour, 0, 0, TimeSpan.Zero);
var windowStart = windowEnd.AddHours(-1);
Console.WriteLine($"Syncing orders from {windowStart:u} to {windowEnd:u}");
}
}
// Hangfire substitutes the real PerformContext for the null argument at run time.
RecurringJob.AddOrUpdate<OrderSyncJob>(
"sync-orders-hourly",
job => job.SyncPreviousHour(null),
"5 * * * *");Under Relaxed misfire handling, a run created after an outage carries the time it was finally triggered, not the missed slot, so a job that must cover every window still needs its own record of what it processed.
When to Cap Attempts#
A workable rule for recurring jobs is to keep the whole retry chain shorter than the gap between occurrences, so every occurrence is settled before the next one starts:
- Hourly:
Attempts = 3withDelaysInSeconds = new[] { 60, 300, 900 }finishes within about 21 minutes of the first failure, plus run times. - Every few minutes:
Attempts = 0or1. The next occurrence is effectively the retry, and default ten-retry chains would pile up dozens of copies of the same failing work. - Daily: the default chain ends within five hours, which is often fine. If someone needs the output by a set time, size the chain to end before it.
Retries can't fix a wiring bug: an ObjectDisposedException from a captured request scope fails every attempt the same way, as explained in Hangfire dependency injection, so cap attempts for jobs whose failures are deterministic.
If two runs of a job must never execute at once, retry limits aren't enough; see Hangfire DisableConcurrentExecution. That filter takes a distributed lock before the job runs, and when the lock times out, the attempt fails and goes through AutomaticRetry like any other exception.
Detect the Final Failure With a Job Filter#
Hangfire decides a job's next state in two phases (StateMachine.cs). Election filters run first and can replace the candidate state; AutomaticRetry is one of them and turns a retryable Failed candidate into Scheduled. Apply filters run next, with the state that will actually be stored. So an applied Failed state means no automatic retry is coming:
using Hangfire.States;
using Hangfire.Storage;
using Microsoft.Extensions.Logging;
public sealed class FinalFailureFilter : IApplyStateFilter
{
private readonly ILogger<FinalFailureFilter> _logger;
public FinalFailureFilter(ILogger<FinalFailureFilter> logger) => _logger = logger;
public void OnStateApplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
{
try
{
switch (context.NewState)
{
// Retryable failures were already turned into Scheduled or Enqueued.
case FailedState failed:
LogFinalFailure(context, failed.Exception.GetType().FullName, failed.Exception);
break;
// OnAttemptsExceeded = Delete ends in Deleted, carrying the exception.
case DeletedState { ExceptionInfo: not null } deleted:
LogFinalFailure(context, deleted.ExceptionInfo.Type, exception: null);
break;
}
}
catch (Exception ex)
{
// A filter that throws breaks the state change itself.
_logger.LogWarning(ex, "Could not inspect Hangfire job {JobId}", context.BackgroundJob.Id);
}
}
public void OnStateUnapplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
{
}
private void LogFinalFailure(ApplyStateContext context, string? exceptionType, Exception? exception)
{
_logger.LogError(
exception,
"Hangfire job {JobId} (recurring job {RecurringJobId}) failed after {RetryCount} retries: {ExceptionType}",
context.BackgroundJob.Id,
context.GetJobParameter<string>("RecurringJobId", allowStale: true) ?? "none",
context.GetJobParameter<int>("RetryCount", allowStale: true),
exceptionType);
}
}Register it once, globally:
using Hangfire;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
builder.Services.AddHangfire((provider, config) => config
.UseSqlServerStorage(builder.Configuration.GetConnectionString("Hangfire"))
.UseFilter(new FinalFailureFilter(
provider.GetRequiredService<ILogger<FinalFailureFilter>>())));
builder.Services.AddHangfireServer();Route that error event to whatever already pages your team, such as an alert rule in your log platform. Unlike the retry filter's own error log, it also fires for exceptions skipped by OnlyOn or ExceptOn and for jobs that end in Deleted.
If you write this as an IElectStateFilter instead, give it an Order above 20. Election filters run in ascending Order, AutomaticRetryAttribute uses Order = 20, and a filter attribute defaults to -1, so a default-order election filter sees every failed attempt as Failed before the retry filter replaces it. Even with a higher order, a later filter can still change the candidate, which is why the apply phase is the safer place to look.
Keep the filter small. OnStateApplied runs inside the state-change transaction while Hangfire holds the job's distributed lock, so an HTTP call there delays the worker and still goes out if the transaction then fails. The worker makes up to 10 attempts at a state change that throws, running the filter each time, so deduplicate alerts on the job ID. If all of them fail, it moves the job to Failed with filters disabled, and this filter never sees it. To inspect failed jobs on demand instead, use the monitoring API covered in Hangfire monitoring.
The harder limit is that the filter runs only while Hangfire runs. A stopped server, a recurring registration removed by a deploy, or a job hung in Processing produces no state change to observe.
Decide Which Failure Should Alert Someone#
Three alert policies fire at different moments, and each suits different jobs:
| Alert on | Fires | Suits jobs where | Example |
|---|---|---|---|
| Every failed attempt | Seconds after any attempt throws | A failed attempt has side effects, or failures are rare enough to be news | A payout batch calling a payment API; any job with Attempts = 0 |
| Final failure | When Hangfire gives up: 4 h 18 min or more after the first error at defaults | Transient errors are normal and retries usually win | An hourly supplier price sync against a rate-limited API |
| Missed completion deadline | When no successful run has finished by a set time | Someone uses the output at a known time, or staleness is the real risk | A 02:00 export a partner collects at 06:00; a 5-minute queue drain |
Per-attempt signals already exist: Hangfire logs a warning for every retry, and a filter can match an applied Scheduled state whose reason starts with "Retry attempt", which is how the retry filter fills the Retries page.
Run the numbers for the last two examples. If the 02:00 export's first attempt fails at 02:01, its tenth retry starts between about 06:19 and 06:46, plus run times, so Hangfire's final verdict comes after the partner has already collected the file. Only a deadline set early enough to act on, say 05:00, warns you in time. For the 5-minute queue drain, one occurrence's retry chain outlasts more than 50 later occurrences. If a dependency is down, all of them fail together: per-attempt alerts fire every few minutes, final-failure alerts arrive hours late, and "no successful run in 30 minutes" is the alert that helps.
Most recurring jobs want two signals: a final-failure alert that carries the exception, and a schedule-based alert from outside Hangfire for runs that never report at all.
Monitor Hangfire Retries and Missed Runs With CronRadar#
CronRadar is that outside check. The CronRadar.Hangfire package reads the recurring-job registry at startup, declares each job's ID and cron schedule, reconciles the registry every 15 minutes by default, and reports start, complete, and fail for each run through a global Hangfire filter. Because CronRadar holds the expected schedule, a run that doesn't report by its scheduled time plus the grace period raises a missed-run alert, whether the server stopped or no worker is serving the queue. A start without a finish is flagged as stuck, and a recurring job that vanishes from the registry is paused and marked as no longer discovered, with its history kept. Alerts go to email, Slack, a webhook, Discord, or PagerDuty.
dotnet add package CronRadar.Hangfireusing CronRadar.Hangfire;
builder.Services.AddHangfire(config =>
config.UseSqlServerStorage(builder.Configuration.GetConnectionString("Hangfire")));
builder.Services.AddHangfireServer();
// After AddHangfire. Reads the application key from CRONRADAR_API_KEY.
builder.Services.AddCronRadarHangfire();Every recurring job is monitored unless its method has [SkipMonitor]. The Hangfire integration guide has the full setup and the current release notice.
Package status, checked October 24, 2026: NuGet's latest CronRadar.Hangfire is 0.1.0. It can report a failure before Hangfire finishes retrying, including cancellations, so an alert can arrive for a run that later succeeds on retry. Reporting only after retries are exhausted requires the pending 0.2.1 release.
What that means when you choose retry counts for monitored recurring jobs:
- On 0.1.0, alerts can follow the every-attempt policy. A
failreport alerts immediately, with no grace period, so under the default ten retries a failed attempt can alert hours before Hangfire recovers or gives up. - Short retry chains keep those alerts honest. With
Attempts = 3andDelaysInSeconds = new[] { 30, 120, 600 }, a transient error resolves one way or the other about 12.5 minutes after the first failure, plus run times, so an early alert is soon followed by a recovery or a real failure. - Jobs that often fail once and then succeed will be noisy. Fix the flaky dependency or expect those alerts until 0.2.1 ships. Catching the exception to quiet them tells Hangfire the run succeeded, so it never retries, and
[SkipMonitor]removes missed-run coverage along with the noise. - Cancellations can alert. Hangfire re-queues a job canceled by shutdown rather than counting a failed attempt, but 0.1.0 can report cancellations as failures.
Once 0.2.1 is released, failures are reported only at the end of the chain, so the chain's length becomes the alert delay: a job left at ten default retries would report 4 h 18 min or more after its first error. The short chains above suit both versions.
Frequently Asked Questions#
How many times does Hangfire retry a failed job by default?#
Ten times. The global AutomaticRetryAttribute schedules up to 10 retries, so a job that keeps failing runs 11 times, with delays adding up to about 4 hours 18 minutes to 4 hours 45 minutes plus each attempt's run time. After the last retry, the job stays in the Failed state, which never expires, until you requeue or delete it.
How do I disable retries for one Hangfire job?#
Put [AutomaticRetry(Attempts = 0)] on the job method, or on the class to cover all its methods. If the job is registered through an interface, put the attribute on the interface method, because Hangfire reads attributes from the type in the job expression. Add OnAttemptsExceeded = AttemptsExceededAction.Delete if the failure should be discarded instead of kept in Failed. Hangfire can still run the job again after a shutdown, so disabling retries doesn't make it run exactly once.
How long does Hangfire wait between retries?#
By default, (n - 1)^4 + 15 + r * n seconds before retry n, where r is random from 0 to 29: 15 to 44 seconds before the first retry and about 1 hour 50 minutes before the tenth. Set DelaysInSeconds to choose the delays yourself; the last entry repeats when there are more attempts than entries. Each delay is a minimum, since the scheduler polls for due retries every 15 seconds by default and a worker must be free.
Does AutomaticRetry apply to recurring jobs?#
Yes. Each recurring occurrence is created as an ordinary background job, so the global filter or an attribute on the method applies, and each occurrence gets its own retry budget. The recurring scheduler doesn't wait for an earlier occurrence to finish retrying, so on hourly or faster schedules, retries of an old run can execute alongside or after newer runs.
How do I get notified only when all retries fail?#
Register an IApplyStateFilter that acts when the applied state is Failed, or Deleted with an exception if you use OnAttemptsExceeded = Delete. By the time that state is applied, AutomaticRetry has already turned every retryable failure into a scheduled retry. Keep the filter fast and hand the alert off instead of calling a network service inside it. With CronRadar, reporting only final failures requires the pending CronRadar.Hangfire 0.2.1; the published 0.1.0 (checked October 24, 2026) can alert on an attempt that later succeeds on retry.
Put this guide to work
Connect your Hangfire job to monitoring
Follow the current integration guide, check the release limitations, and verify a real run and alert delivery.
NuGet currently installs CronRadar.Hangfire 0.1.0. Setup and discovery work, but this release can report failures before Hangfire finishes retrying, including cancellations. Reporting only after retries are exhausted requires the pending 0.2.1 release. Every reconciliation sets each discovered monitor's grace period to CronRadar.CronRadar.DefaultGracePeriodSeconds, 60 unless set in code, replacing dashboard edits.