# Cron Job Monitoring Grace Period: How Long to Wait Before Alerting

By Dan Ochoa · Published 2026-10-25 · Updated 2026-10-31
Canonical: https://cronradar.com/blog/cron-grace-period

> How to choose a cron job monitoring grace period from run times, scheduler delays, and check-in placement, so missed runs alert fast without false alarms.

**Set a cron job's grace period just above the latest its check-in arrives on a healthy run, and below the delay you can afford before hearing about a real miss.** Both numbers come from evidence you can collect: how long the job runs, how late the scheduler starts it, and whether the job checks in when it starts or only when it finishes. A completion-only heartbeat needs a grace that covers the whole run; a start check-in needs one that covers only the start delay. The method applies to any heartbeat monitor, with CronRadar as the worked example.

## A Grace Period Trades False Alarms for Detection Time

A heartbeat monitor reads the job's schedule to know when each check-in is due; the [dead man's switch](https://cronradar.com/blog/cron-dead-mans-switch) explains the mechanism. The grace period is how long past that time it waits before calling the run missed. Interval heartbeats, such as Opsgenie's, fold the grace into the interval; [converting an Opsgenie heartbeat](https://cronradar.com/blog/opsgenie-heartbeat-migration) separates the two. Each minute of grace is a minute longer before you learn of a real miss, and a grace shorter than the job's normal lateness produces false alarms.

Two bounds settle the number:

- **The floor is evidence:** the latest a healthy run's check-in arrives after its scheduled time. Below it, you get false alarms.
- **The ceiling is cost:** how late you can learn of a miss without losing anything. For a sync feeding a live dashboard, that's minutes; for a nightly backup nobody restores from before morning, it's hours.

When misses are expensive, sit near the floor. If the floor is above the ceiling, no grace period works: move the check-in or remove the source of lateness. For a P1 job in the [CronRadar Scheduled Job Methodology](https://cronradar.com/methodology), the ceiling comes from its finish-by deadline: the alert has to leave time to rerun the job before it.

Where the floor sits depends on how often the job runs. False alarms per month are roughly the number of runs times the share that check in later than the grace. A grace at the 99th percentile of lateness gives a job that runs every 5 minutes about 86 false alarms a month (8,640 runs), and a nightly job about one every three months. Frequent jobs need a grace beyond their slowest observed run, unless the check-in moves to the start.

## Check-In Placement Decides What the Grace Must Cover

### Completion Only: The Grace Covers the Whole Run

With one heartbeat on success, a 02:30 backup that runs 40 minutes checks in at 03:10, 40 minutes after it was due. The grace has to cover start delay plus the longest run you'll accept, so a missed run is noticed only after that whole window.

In exchange, one deadline catches everything: a run that never started, hung, or died with its host fails to check in by the same time. Report `fail` on a nonzero exit as well, and failures alert immediately instead of at the deadline.

### Start and Complete: The Grace Covers Only the Start

When the job also reports at start, the deadline question becomes "did it start on time?" The grace covers start delay alone, often seconds, so a missed run is flagged soon after it was due however long the job takes, and the monitor records each run's duration. A run that starts and never finishes falls to stuck detection.

### Stuck Detection Is Per Job, Not Per Run

In CronRadar, every check-in, including `start`, moves the expected time to the job's next scheduled occurrence. If that time plus the grace period passes while the last event was a `start` with no `complete` or `fail`, the monitor goes `critical` as stuck. The window belongs to the job, not the run: the next run's `start` resets it, so a hang is caught only when the job then goes silent, for example because a lock skips the next run or a systemd timer waits for the hung service. There is no per-job maximum-runtime setting.

That has two consequences for the grace:

- **Under a skip lock, a run can last one interval plus the grace before it's flagged.** The grace becomes the overrun you tolerate; [preventing overlapping cron jobs](https://cronradar.com/blog/prevent-overlapping-cron-jobs) walks through the timeline.
- **For infrequent jobs, the window is long.** A hung nightly job is flagged after the next night's run time plus grace, and only if that run doesn't report either; a monthly one waits a month. A timeout inside the job turns a hang into a `fail`, which alerts at once. No timeout can report a run that died with its host, though; only a completion deadline catches that quickly.

|                        | Completion only, plus `fail`   | Start, plus `complete` or `fail`                                  |
| ---------------------- | ------------------------------ | ----------------------------------------------------------------- |
| Grace must cover       | Start delay and run time       | Start delay                                                       |
| Missed run flagged     | Scheduled time plus long grace | Scheduled time plus short grace                                   |
| Hung run flagged       | At the same deadline           | At the job's own timeout, or after the next occurrence plus grace |
| Run lost with its host | At the same deadline           | After the next occurrence plus grace, if nothing else reports     |
| Run durations          | Not recorded                   | Recorded                                                          |

Frequent jobs belong on start and complete. For infrequent jobs, choose: start and complete with a timeout gives durations and fast missed-start alerts, while a completion deadline catches a lost run the same day.

## Budget for Every Source of Start Delay

Under either placement, the floor includes how late the job actually starts.

### Scheduler Slack and Jitter

- **cronie's `RANDOM_DELAY`** (Fedora, RHEL, Arch) delays job starts by a random number of minutes up to its value. The random factor is chosen when crond starts and stays fixed until it restarts ([crontab(5), cronie 1.7.2](https://github.com/cronie-crond/cronie/blob/cronie-1.7.2/man/crontab.5)), so budget the full value.
- **systemd timers** fire within `AccuracySec=` of their time, one minute by default, and `RandomizedDelaySec=` (default 0) adds a random delay up to its value ([systemd.timer(5)](https://www.freedesktop.org/software/systemd/man/latest/systemd.timer.html), systemd 262). Budget both; [systemd timers](https://cronradar.com/blog/systemd-timers) covers the settings.
- **Jenkins `H`** is "a hash of the job name, not a random function", so the value is stable per job ([Pipeline syntax](https://www.jenkins.io/doc/book/pipeline/syntax/#cron-syntax)). An `H * * * *` job runs at the same minute every hour, not necessarily :00. Give the monitor that minute instead of widening the grace; Jenkins shows it under the schedule field as "Would last have run at …; would next run at …" ([Messages.properties](https://github.com/jenkinsci/jenkins/blob/master/core/src/main/resources/hudson/triggers/Messages.properties)). [Jenkins scheduled builds](https://cronradar.com/blog/jenkins-scheduled-builds) covers `H` and `TZ=` in full.
- **GitHub Actions** says the schedule event "can be delayed during periods of high loads", that high load includes the start of every hour, and that under enough load "some queued jobs may be dropped" ([schedule event](https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows#schedule), checked October 24, 2026). No bound is documented: schedule off the hour, measure your workflow's start delays, and treat a dropped run as the real miss it is.
- **GitLab pipeline schedules** are queued only when GitLab's pipeline schedule worker runs, every 5 minutes on GitLab.com and every 10 minutes by default on self-managed instances, and GitLab says batching under load can start them "several minutes to over an hour after their scheduled time" ([maximum scheduled pipeline frequency](https://docs.gitlab.com/19.5/administration/cicd/limits/#maximum-scheduled-pipeline-frequency)). [GitLab scheduled pipelines](https://cronradar.com/blog/gitlab-scheduled-pipelines) covers sizing the grace for both.
- **Kubernetes CronJobs:** the 1.37.1 controller requeues each CronJob for its next scheduled time plus 100 ms ([utils.go](https://github.com/kubernetes/kubernetes/blob/v1.37.1/pkg/controller/cronjob/utils.go)), so the Job is created close to its time. The delay comes after: a `Pending` Pod may still be waiting to be scheduled or "downloading container images over the network" ([Pod phase](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-phase)), and a check-in from inside the container includes that time. A Job the controller can't create within `startingDeadlineSeconds` of its time, for any reason, is skipped rather than delayed ([CronJob](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/)).

### Queue Wait in Hangfire and Celery

- **Hangfire 1.8** checks recurring jobs every `SchedulePollingInterval`, 15 seconds by default ([BackgroundJobServer.cs](https://github.com/HangfireIO/Hangfire/blob/v1.8.25/src/Hangfire.Core/BackgroundJobServer.cs), [DelayedJobScheduler.cs](https://github.com/HangfireIO/Hangfire/blob/v1.8.25/src/Hangfire.Core/Server/DelayedJobScheduler.cs)), then enqueues each due job to wait for a free worker; the default is five workers per processor, capped at 20 ([BackgroundJobServerOptions.cs](https://github.com/HangfireIO/Hangfire/blob/v1.8.25/src/Hangfire.Core/BackgroundJobServerOptions.cs)). Long jobs that fill the workers delay everything queued behind them.
- **Celery 5.6** beat sends the task on schedule, and a worker runs it when a process is free. Concurrency defaults to the number of CPU cores, and each process prefetches up to four messages ([configuration](https://docs.celeryq.dev/en/v5.6.3/userguide/configuration.html)); the docs recommend a prefetch multiplier of one when tasks run long ([prefetch limits](https://docs.celeryq.dev/en/v5.6.3/userguide/optimizing.html#prefetch-limits)).

A check-in made from inside the job counts queue wait as start delay. Budget for the busiest hour, or give scheduled jobs a queue and workers of their own.

### Locks That Wait

A lock with the wait policy, such as `flock -w`, a systemd timer waiting for its running service, or Kubernetes `Forbid` without `startingDeadlineSeconds`, starts the next run when the previous one ends. That run's start is late by the previous run's overrun, so budget for it, or use a skip policy and let stuck detection cover the long run.

### Daylight Saving Time

A job on local time moves an hour in UTC twice a year. CronRadar evaluates schedules in UTC, and its [schedule syntax](https://docs.cronradar.com/schedules) rule is to use the earlier of the two UTC times with a grace of at least an hour plus the job's usual duration. For the half of the year when the job runs at the later time, the monitor shows `warning` for an hour before each run. Running the scheduler in UTC avoids that and the [skipped and doubled runs](https://cronradar.com/blog/cron-daylight-saving-time) on transition nights.

### Constant Offsets Are Schedule Bugs

If every run is late by the same amount, fix the schedule, not the grace. Common causes are a crontab time sent in local time to a monitor that reads UTC, a Jenkins `H` minute, or an expression restricting both day fields, which Linux cron matches on either field and CronRadar on both. A drifting host clock does the same, since CronRadar timestamps check-ins on arrival. Widening the grace to hide an offset delays every real alert by that offset.

## Derive the Number From Measurements

1. **Measure what the check-in will see:** start delay for a start check-in, start delay plus run time for completion only. In CronRadar, each run in a monitor's run history shows its Duration and its Lateness against the expected time; with completion-only check-ins, Lateness already includes the run time. Run history lasts 30 days on Free and 90 on Grow, so log runs yourself for a longer sample:

   ```bash
   #!/bin/sh
   # /opt/backup-timed.sh: run the backup and log its duration and exit code.
   start=$(date +%s)
   /opt/backup.sh
   rc=$?
   echo "$(date -u +%Y-%m-%dT%H:%M:%SZ) $(($(date +%s) - start)) $rc" >> /var/log/backup-runs.log
   exit "$rc"
   ```

2. **Take a percentile that fits the run count,** using the false-alarm arithmetic above. This prints the spread of successful runs:

   ```bash
   awk '$3 == 0 { print $2 }' /var/log/backup-runs.log | sort -n | awk '
     function rank(p,  r) { r = int(p * NR); if (r < p * NR) r++; return r < 1 ? 1 : r }
     { d[NR] = $1 }
     END { if (NR) printf "runs=%d p50=%ds p95=%ds p99=%ds max=%ds\n",
             NR, d[rank(0.50)], d[rank(0.95)], d[rank(0.99)], d[NR] }'
   ```

3. **Add delays the sample didn't capture:** the scheduler settings above, and the extra hour for a local-time job.
4. **Add headroom** for data growth, and round up.
5. **Compare it with the ceiling.** If it's higher, move the check-in to the start or remove the lateness.

### Three Worked Examples

The numbers are illustrations; substitute your own measurements.

**A sync every 5 minutes, about 2 minutes a run.** Suppose two weeks of logs (4,032 runs) show a p99 of 2 min 50 s, a maximum of 3 min 55 s, and starts within a few seconds of the minute. Completion only would need more than the maximum, say 300 seconds, and a missed 10:05 run would be flagged after 10:10. With a start check-in, the grace only has to clear the start delay: 120 seconds, flagged after 10:07. Under a skip lock, a run can then last 7 minutes before it's stuck, well past the maximum.

**A nightly backup at 02:30 UTC that takes 10 to 40 minutes.** Completion only: the longest run plus a quarter for data growth is 50 minutes; add a minute of start delay and round up to 3,600 seconds, flagged after 03:30. With a start check-in, the floor is seconds, and the ceiling is high, because nothing anyone does differs between learning at 02:32 and at 02:45. A grace of 900 seconds covers start delays the sample never saw, at no real cost. Add a 90-minute timeout, about twice the longest run, so a hang becomes a `fail` at 04:00.

_Figure: The overdue check runs once a minute: with no check-in, each monitor is warning once 02:30 passes and turns critical at the first check that finds the run past its grace and at least two minutes late, after 02:45 with a start check-in and after 03:30 with completion only. The completion-only grace has to be that long because the heartbeat can't arrive before the 10 to 40 minute run ends._

**A monthly invoice run at 03:00 UTC on the 1st, 15 to 25 minutes.** Six recorded runs make percentiles meaningless, so take twice the longest, 50 minutes, and round up to 3,600 seconds. Stuck detection would take a month to report a run lost with its host, so use a completion deadline: a heartbeat on success and `fail` on failure. Anything short of a successful run by 04:00 on the 1st opens an incident.

## How CronRadar Applies the Grace Period

- **Setting it.** Send `gracePeriod` in seconds on the check-in that creates the monitor; without it, the monitor gets the product default of 90 seconds, and the minimum is 60. Later check-ins don't change an existing monitor's grace or schedule, so edit **Grace Period (seconds)** in the monitor's settings, from 60 to 86,400. Keep the script's value in step, because the next check-in recreates a deleted monitor from it ([API reference](https://docs.cronradar.com/api-reference)). Monitors the Hangfire and Quartz.NET extensions discover work differently: every reconciliation sets their grace period to `CronRadar.CronRadar.DefaultGracePeriodSeconds`, 60 unless set in code, replacing an edit in the dashboard.
- **Timing.** The overdue check runs once a minute. When the expected time passes without a check-in, the monitor turns `warning`. Once the run is past its grace period and at least two minutes late, it turns `critical`, its last result becomes `timeout`, and an incident opens. Any grace from 60 to 120 seconds therefore acts as two minutes; setting 120 makes the setting match what happens.
- **`warning` is a status, not an alert.** A `start` doesn't clear it, so a run whose `start` arrived after a check had marked it late stays `warning` until it reports `complete` or `fail`.
- **Failures skip the grace.** A `fail` check-in opens an incident immediately.

## A Check-In Wrapper With a Justified Grace Period

There's no CronRadar SDK for crontab and no auto-discovery, so each cron job reports with its own HTTP check-in. This is the nightly backup from the examples:

```bash
#!/bin/sh
# /opt/backup-monitored.sh
# gracePeriod=900: starts arrive within seconds of 02:30, and hearing of a
# missed start at 02:45 costs nothing. The grace doesn't cover the 10-40
# minute run: the start check-in answers "did it start?", and timeout turns
# a hang into a fail at 04:00.
# The key lives in a file only the job's account can read: cronie and Debian's cron
# copy crontab variables into the headers of the mail they send.
CRONRADAR_API_KEY=$(cat /etc/cronradar/api-key 2>/dev/null)
url="https://cron.life/api/v1/ping/nightly-backup"
run_id="nightly-backup-$(date +%s)-$$"

report() {
  event=$1
  shift
  curl --fail --silent --show-error --max-time 5 --output /dev/null \
    --request POST --get --user "$CRONRADAR_API_KEY:" \
    --data-urlencode "schedule=30 2 * * *" \
    --data-urlencode "gracePeriod=900" \
    --data-urlencode "run_id=$run_id" \
    "$@" "$url/$event" || true
}

report start
timeout -k 5m 90m /opt/backup.sh
rc=$?
if [ "$rc" -eq 0 ]; then
  report complete
else
  report fail --data-urlencode "message=backup.sh exited with code $rc"
fi
exit "$rc"
```

```text
30 2 * * * /opt/backup-monitored.sh
```

Each report gives up after five seconds and ignores errors, and the script exits with the backup's own status. GNU `timeout` exits 124 when it stops the backup at the limit ([timeout(1)](https://man7.org/linux/man-pages/man1/timeout.1.html)), so code 124 in a `fail` message means the run hung or ran far longer than usual.

For the monthly job, drop the start check-in and the timeout, and report success to the heartbeat URL itself:

```bash
#!/bin/sh
# /opt/invoices-monitored.sh
# gracePeriod=3600: twice the longest of six recorded runs (25 minutes), rounded up.
CRONRADAR_API_KEY=$(cat /etc/cronradar/api-key 2>/dev/null)
url="https://cron.life/api/v1/ping/monthly-invoices"

report() {
  curl --fail --silent --show-error --max-time 5 --output /dev/null \
    --request POST --get --user "$CRONRADAR_API_KEY:" \
    --data-urlencode "schedule=0 3 1 * *" \
    --data-urlencode "gracePeriod=3600" \
    "$@" || true
}

/opt/invoices.sh
rc=$?
if [ "$rc" -eq 0 ]; then
  report "$url"
else
  report --data-urlencode "message=invoices.sh exited with code $rc" "$url/fail"
fi
exit "$rc"
```

## Frequently Asked Questions

### What is a grace period in cron monitoring?

It's how long a monitor waits after a job's scheduled time before treating a missing check-in as a missed run. It absorbs normal lateness, such as scheduler delay and, with completion-only check-ins, the job's own run time. Inside the grace, CronRadar shows `warning`; after it, `critical`.

### How long should a grace period be?

Longer than the latest a healthy run checks in, measured at a percentile that fits how often the job runs, and shorter than the delay you can afford before hearing about a miss. With a start check-in, that's usually a couple of minutes. With a completion-only heartbeat, it's the longest run plus start delay and headroom.

### Why do I get false alarms from my cron monitor?

Usually the grace is shorter than the job's real lateness: a completion-only heartbeat on a job whose run time varies, or a scheduler or queue that adds delay. If every run is late by the same amount, the schedule is wrong, often a local time sent to a monitor that reads UTC.

### Should I send a ping at the start or end of a cron job?

For frequent jobs, both: the start check-in lets the grace cover only start delay, and the completion records success and duration. For a job that runs monthly or less often, a completion deadline (a heartbeat at the end, plus `fail` on errors) catches a run lost with its host the same day, which start-based stuck detection catches only after the next scheduled time.

### What happens if my job runs longer than the grace period?

With a completion-only heartbeat, the monitor goes `critical` and opens an incident; the late heartbeat then resolves it and sends a recovery alert. With start and complete check-ins, run time doesn't count against the grace unless the run outlasts its next scheduled time plus the grace with nothing else reporting, which marks it stuck. A timeout inside the job sets the real limit.