Hangfire Job Stuck in Processing: Orphaned or Still Running?

A Hangfire job stuck in Processing was fetched by a worker that hasn't recorded a result yet: either your method is still running, or the process running it stopped and the job is waiting to be fetched again. The dashboard tells you which. A job whose server is gone, or silent for over a minute, is flagged as aborted and gets requeued on its own. A job on a server with a fresh heartbeat is still running your code and stops only when that code returns, so it needs a timeout inside the job.

This guide covers Hangfire 1.8 with SQL Server storage, with notes for 1.7 and Hangfire.Pro.Redis. Source links point to 1.8.25, the current release (checked October 24, 2026), and the examples assume ASP.NET Core on .NET 8 or later. For job-status queries through the monitoring API, see Hangfire monitoring.

What Processing Means in Hangfire Storage#

A worker takes a job ID from a queue, moves the job to Processing with its server ID and worker ID, runs the method, and then records Succeeded, Failed, or whatever state a filter chooses. Meanwhile the storage has to remember that the job was taken. A comment in Hangfire's worker states the guarantee: a job in Processing may or may not have been performed, so it is requeued if processing doesn't finish, to make sure it runs at least once. What matters is how a fetched job is hidden from other workers, and what makes it visible again.

SQL Server: The Sliding Invisibility Timeout#

Since 1.8.0, Hangfire.SqlServer fetches with a sliding invisibility timeout of 5 minutes by default (1.8.0 release notes). Fetching stamps the queue row with a FetchedAt time. While the job runs, a keep-alive refreshes that stamp every fifth of the timeout, so once a minute at the default (SqlServerTimeoutJob). Another worker can fetch the row only after the stamp is older than the timeout (fetch query).

Two things follow. A job on a live process is never released, however long it runs, because the keep-alive continues. A job on a dead process can be fetched again about 5 minutes after its last keep-alive.

SlidingInvisibilityTimeout arrived as an opt-in option in 1.6.14 and stayed off by default until 1.8.0. The older InvisibilityTimeout (30 minutes) is marked obsolete in SqlServerStorageOptions; the obsolete message says jobs are now requeued instantly even after an ungraceful shutdown. It doesn't govern sliding fetches. If your configuration sets SlidingInvisibilityTimeout to 30 minutes, every orphan waits that long.

Redis Storage#

Hangfire.Pro.Redis uses a different rule. Since 2.4.0, a job in Processing counts as faulted only when its server becomes inactive, either through a graceful shutdown or a heartbeat timeout, and InvisibilityTimeout no longer affects jobs on active servers (Hangfire.Pro.Redis 2.4.0). On Redis, an orphan comes back only as fast as Hangfire declares its server dead.

First, Tell an Orphaned Job From a Running One#

If the dashboard answers 401 outside localhost, fix access first (Hangfire dashboard authorization), then check three dashboard pages before you change anything:

• Processing Jobs. A warning icon next to a job means its server isn't in the active list, or its last heartbeat is older than DashboardOptions.ServerPossiblyAbortedThreshold, which defaults to 1 minute (page source).
• Job details. The same condition shows "The job was aborted", adding that it "will be retried automatically after invisibility timeout" (dashboard strings). The state history lists every Processing entry with its server.
• Servers. Each server's worker count, queues, and last heartbeat. Servers report a heartbeat every 30 seconds by default (options).

Orphaned Jobs After a Deploy, Recycle, or Container Kill#

What a Graceful Stop Does#

When the host starts shutting down, Hangfire's hosted service sends its server a stop signal. Workers stop fetching, and StopTimeout later (500 ms by default since 1.8.0) Hangfire cancels the token of every running job (BackgroundProcessingServer). A job that observes the token throws OperationCanceledException. Hangfire treats that as a shutdown rather than a failure, logs "Worker stop requested while processing background job '...'. It will be re-queued.", and puts the job back in its queue.

The dashboard keeps showing that job as Processing on the old server, because the state only changes when a worker fetches it again. After a deploy, the new instance picks it up as soon as a worker listening on that queue is free.

What an Ungraceful Stop Does#

Hangfire waits up to ShutdownTimeout, 15 seconds by default, for running jobs to return. When it gives up, it logs that the server "stopped non-gracefully" and that outstanding work "could be aborted", and it advises adding "CancellationToken support for those methods" (BackgroundServerProcess). A job still running when the process exits is orphaned, whether the exit came from a deploy, an IIS app pool recycle, an OOM kill, or a container stop that outlasted its grace period.

On SQL Server 1.8, the orphan can be fetched once its keep-alive is 5 minutes stale. Servers follow a separate clock. Hangfire removes a server whose heartbeat is older than ServerTimeout (5 minutes), and each server's watchdog checks every ServerCheckInterval (also 5 minutes) (BackgroundJobServerOptions, ServerWatchdog), so a killed server can stay listed for up to about 10 minutes. On SQL Server that delay only affects the dashboard. On Redis, the requeue waits for it.

When an Aborted Job Never Comes Back#

A released job still needs a free worker that listens on its queue. If the dead server was the only one serving that queue, nothing fetches the job until such a server starts.

A job whose history shows Processing on one server after another, with no Failed entry, is probably taking the process down with it. It never throws, so AutomaticRetry never counts an attempt, and each restart fetches it again. Find the crash in the host or container logs, and delete the job until it's fixed.

Jobs That Are Genuinely Still Running#

Hangfire has no execution time limit, and it can't stop a thread for you: Thread.Abort throws PlatformNotSupportedException on .NET Core and .NET 5 and later (Microsoft). Cancellation is cooperative, so a job stops early only if its code checks the token. The usual reasons a job runs forever:

• I/O with no timeout. HttpClient.Timeout defaults to 100 seconds, but Timeout.InfiniteTimeSpan removes the limit (HttpClient.Timeout). SqlCommand.CommandTimeout defaults to 30 seconds, and 0 means wait indefinitely (CommandTimeout).
• Child processes. Process.WaitForExit() waits indefinitely (WaitForExit).
• Lock waits. A query blocked by another transaction, or a semaphore held by an earlier stuck run.
• A task that never completes, such as a TaskCompletionSource nothing completes, or an external API that accepts the connection and never answers.

The keep-alive means a hung job is never released, and Hangfire's recurring scheduler doesn't wait for the previous run. If each new run hangs the same way, they use up the workers, Math.Min(Environment.ProcessorCount * 5, 20) by default, and everything else sits in Enqueued. [DisableConcurrentExecution(timeoutInSeconds: 60)] stops a job from overlapping itself: a second run waits up to 60 seconds for the first, then fails with a DistributedLockTimeoutException (attribute, SQL Server lock), so a pile-up shows as failures.

Put a Deadline on Every Run#

Since 1.7.0, Hangfire replaces a job method's CancellationToken parameter with a token of its own (cancellation tokens). It cancels that token when the server shuts down. It also cancels it when the job leaves Processing on that worker, for example after you delete or requeue it, and it checks for that every CancellationCheckInterval, 5 seconds by default (ServerJobCancellationToken). Link your own deadline to it and pass the result to every I/O call:

using System;
using System.IO;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;

public class PriceFeedJobs
{
    // The longest one run may spend downloading before it gives up.
    private static readonly TimeSpan DownloadDeadline = TimeSpan.FromMinutes(2);

    private const string FeedUrl = "https://feeds.example.com/prices.csv";
    private const string TargetPath = "/var/data/prices.csv";

    private readonly HttpClient _http;

    public PriceFeedJobs(HttpClient http) => _http = http;

    public async Task DownloadPricesAsync(CancellationToken cancellationToken)
    {
        // Hangfire cancels cancellationToken on shutdown and when the job is
        // deleted or requeued. The linked source adds this run's own deadline.
        using var deadline = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
        deadline.CancelAfter(DownloadDeadline);

        using var response = await _http.GetAsync(
            FeedUrl, HttpCompletionOption.ResponseHeadersRead, deadline.Token);
        response.EnsureSuccessStatusCode();

        // Write to a side file and swap it in, so a run that is interrupted
        // and then retried or requeued never leaves half a file in place.
        var partialPath = TargetPath + ".partial";
        await using (var file = File.Create(partialPath))
        {
            await response.Content.CopyToAsync(file, deadline.Token);
        }

        File.Move(partialPath, TargetPath, overwrite: true);
    }
}

What happens next depends on which token fired (CoreBackgroundJobPerformer):

• Your deadline. The OperationCanceledException counts as an ordinary failure. The job moves to Failed and AutomaticRetry schedules a retry, 10 attempts by default (exceptions and retries). Hangfire automatic retry covers tuning that policy.
• Server shutdown. The job goes back in its queue and isn't counted as a failure.
• Deleted or requeued in the dashboard. The run is abandoned, and the job keeps the state you gave it.

For a call that takes no token, Task.WaitAsync(TimeSpan, CancellationToken) (.NET 6 and later) limits how long you wait, though the operation behind the task keeps running (WaitAsync). For a child process, WaitForExitAsync(token) ends the wait and Kill(entireProcessTree: true) stops the process (WaitForExitAsync, Kill). The older IJobCancellationToken interface still works, but the docs call it obsolete and recommend CancellationToken for new code.

Give Shutdown Enough Time to Requeue Cleanly#

A deploy shouldn't orphan anything. Jobs that honor the token unwind quickly and get requeued. A job that needs a few seconds to reach a safe point gets them only if every layer above it waits that long, and the shortest limit wins:

Hangfire's wait is bounded by the host's, because the hosted service's StopAsync waits on the host's cancellation token. Plain Docker's 10 seconds is shorter than Hangfire's 15, so a slow job there is killed before Hangfire stops waiting. On .NET 6 and later, the host treats SIGTERM as a graceful stop (.NET Generic Host). Order the limits so each fits inside the next:

using Hangfire;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddHangfire(config =>
    config.UseSqlServerStorage(builder.Configuration.GetConnectionString("Hangfire")));

builder.Services.AddHangfireServer(options =>
{
    // How long running jobs get after the stop signal (default 15 seconds).
    options.ShutdownTimeout = TimeSpan.FromSeconds(20);
});

// How long the host waits for all hosted services (default 30 seconds).
// Keep it above Hangfire's ShutdownTimeout and below the platform's grace period.
builder.Services.Configure<HostOptions>(options =>
    options.ShutdownTimeout = TimeSpan.FromSeconds(25));

builder.Services.AddHttpClient<PriceFeedJobs>();

var app = builder.Build();

app.Services.GetRequiredService<IRecurringJobManager>().AddOrUpdate<PriceFeedJobs>(
    "download-prices",
    jobs => jobs.DownloadPricesAsync(CancellationToken.None),
    "*/15 * * * *");

app.Run();

These values fit Kubernetes' default 30-second grace period. Under Docker or Compose, raise stop_grace_period above 25 seconds or lower the two values. CancellationToken.None in the registration is a stand-in that Hangfire replaces at run time.

When the Job Is Stuck in Enqueued Instead#

Enqueued means no worker has taken the job yet. Check these in order:

1. No server listens on its queue. A server processes only the queues in BackgroundJobServerOptions.Queues, which defaults to default alone. A job placed in another queue, with [Queue("reports")] or the 1.8 overloads that take a queue name, waits until a server lists that queue, for example options.Queues = new[] { "reports", "default" } (configuring queues). The Servers page shows each server's queues.
2. All workers are busy. If the Processing count matches the total workers on the Servers page, the queue is waiting on capacity. Fix hung jobs before raising WorkerCount or moving slow jobs to their own queue.
3. No server is running, or it uses different storage. An IIS-hosted app that has idled out or recycled processes nothing until the next request (always running). A server with a different connection string, SchemaName, or Redis prefix never sees the job.

For recurring jobs that never even reach Enqueued, see Hangfire recurring job not running.

Recover Without Running Work Twice#

Dashboard actions change the job's state; they don't reach into a running thread. From the Processing page, Requeue and Delete act only on jobs still in Processing. From a job's details page they act on any state (routes).

A deleted job's late result is ignored, and it doesn't come back after a restart, because the worker records a result only while the job is still in Processing and picks up only Enqueued, Scheduled, or Processing jobs (Worker).

For an orphan on SQL Server, waiting is the safer choice. Requeue inserts a new queue entry while the orphan's old entry is still waiting out its timeout, and if the job is still enqueued or processing when that old entry becomes fetchable, a second worker can take it too.

Every path leads to the rule in Hangfire's best practices: make background methods reentrant, because a run can be interrupted and called again. Write output to a temporary location and swap it in, make external calls safe to repeat, and record progress so a rerun can skip finished work. Preventing errors in scheduled automation has a checklist.

Get Alerted When a Run Starts and Never Finishes#

The Processing page shows a stuck job only to someone who opens it, and if the dashboard is hosted by the same app, it goes down with that app. To be told, record each run's start and finish outside the process.

CronRadar's Hangfire integration does this for recurring jobs. Add builder.Services.AddCronRadarHangfire(); after AddHangfire(). It discovers the recurring-job registry and installs a server filter that reports start before each execution and complete or fail after it, with no monitoring code in job methods. If a run starts and neither complete nor fail arrives before the job's next run is due plus its grace period, CronRadar marks the monitor critical and alerts that the run is stuck. CronRadar.Hangfire 0.1.0 declares a 60-second grace period, the base SDK's DefaultGracePeriodSeconds, and every reconciliation resets the monitor to it, so set CronRadar.CronRadar.DefaultGracePeriodSeconds before AddCronRadarHangfire() to change it; an edit in the dashboard lasts only until the next reconciliation. Alerts go to email, Slack, webhook, Discord, or PagerDuty.

The stuck window is per job, not per run: any report from the same job, including the next run's start, resets it. It catches a run after which the job goes silent, such as a host that dies mid-run and doesn't return, or a hang that blocks later runs too. If Hangfire starts the next run on another free worker, the earlier hang shows only on the Processing page. A deadline inside the job turns that hang into a failure, and a fail report alerts right away. The published CronRadar.Hangfire 0.1.0 (checked September 20, 2026) can report a failure before Hangfire finishes retrying, including cancellations; reporting only after retries are exhausted requires the pending 0.2.1 release.

Frequently Asked Questions#

Why are Hangfire jobs stuck in Processing after a deployment?#

The old instance stopped while they were running. Jobs that honored their CancellationToken were requeued at shutdown and show as Processing only until a new server fetches them. Jobs still running when the process exited are orphaned; on SQL Server they become fetchable once the 5-minute sliding invisibility timeout passes (the Hangfire 1.8 default). If they stay longer, check that the new servers listen on those jobs' queues.

How long until Hangfire requeues an orphaned job?#

On Hangfire.SqlServer 1.8, once its last keep-alive is older than SlidingInvisibilityTimeout, 5 minutes by default. With the transactional fetch 1.7 used by default, as soon as SQL Server rolls back the dead connection's transaction. On Hangfire.Pro.Redis 2.4 and later, when the dead server is removed, at least ServerTimeout (5 minutes) after its last heartbeat. Then it still needs a free worker on its queue.

Can a Hangfire job run twice after being requeued?#

Yes. Hangfire guarantees at-least-once execution, so an orphaned run that did part of its work runs again from the start. Requeueing a job that's still running starts a second copy if the code ignores its token, and on SQL Server, requeueing an orphan before its old queue entry expires can let two workers take it. Make jobs safe to repeat.

How do I stop a Hangfire job that is running too long?#

Delete it from the dashboard. Within about 5 seconds Hangfire cancels the job's CancellationToken, and code that passes the token to its I/O stops, leaving the job Deleted. Code that ignores the token can't be stopped from outside, because .NET Core can't abort a thread, so it runs until it returns or the process restarts. Long term, give the job its own deadline.

How do I get alerted when a Hangfire job starts but never finishes?#

Record a start and a finish outside the process. CronRadar.Hangfire reports both for each recurring job and flags the run as stuck if no complete or fail arrives before the next run is due plus the grace period. The next run's start resets that window, so add a deadline inside the job too; a timed-out run reports a failure, which alerts immediately.