Applying the CronRadar Scheduled Job Methodology

This page applies the rules of the CronRadar Scheduled Job Methodology. The research each rule rests on is on the evidence page.

Rank Every Job P1, P2, or P3#

CronRadar recommends monitoring every scheduled job and ranking each one. The rank decides how much protection and how loud an alert a job gets, never whether it is monitored. Evidence: cron records nothing about its launches, and Google's SRE book says owners "can (and should!)" monitor their jobs, including "independent monitoring of the effect of cron jobs."

Our classes are named P1, P2, and P3 rather than critical, high, and low. "Critical" already names a CronRadar status and severity, a PagerDuty severity, a Control-M job attribute, and an HCL keyword. The numbering follows PagerDuty's priority convention, in which P1 is the most important; Temporal and Control-M's Service Priority also put 1 highest.

Five Questions Decide the Class#

1. Can a run be skipped? The SRE book's example: "a garbage collection cron job scheduled to run every five minutes may be able to skip one launch, but a payroll cron job scheduled to run once a month should not be skipped." A run can be skipped when the next run covers its work.
2. Can it safely run twice? Some jobs, such as "a process that sends out an email newsletter to a wide distribution, should not be launched more than once" (SRE ch. 24). Idempotent scheduled jobs covers how to make a second run harmless.
3. Does it have a finish-by deadline? A provider's cutoff, the start of a business day, or the time another job reads its output. For a job that feeds another, the deadline is the latest time its consumer can start. HCL computes this backward: "if the critical job deadline is 19:00 and the estimated duration of the critical job is 30 minutes, the critical job will not finish by the deadline unless it has started by 18:30. If the immediate predecessor of the critical job has an estimated duration of 20 minutes, it must start at latest by 18.10" (Planning critical jobs).
4. What depends on it downstream? HCL: "The critical path is the chain of dependencies, leading to the critical job, that is most at risk of causing the deadline to be missed at any given time" (Processing and monitoring critical jobs). Control-M's SLA Management job "maps out the critical path" of a workflow (SLA Management Job).
5. Can it succeed with a wrong result? An exit status of 0 says the command didn't fail, not that the output is right. Control-M treats "Job finished too quickly" as an SLA condition (SLA Management Job). CronRadar's outcome checks test up to five reported values per run. Silent success covers where these runs come from.

The Rule#

Our rule assigns the first class whose condition a job meets:

Questions 1 and 3 test the first P1 condition. Questions 2 and 5 find failures that don't look like failures, a second run and a run that exits 0 with a wrong result; either counts toward the second P1 condition when it loses money, customer-visible data, or backups. Question 2 also decides how the job must be built (see "Architecture Rules for Each Scale"). Question 4 finds the people and jobs that use the result, which is the test for P2, and gives a job that feeds another the finish-by deadline of its consumer.

In CronRadar, a monitor whose severity is blank inherits its application's default, then the workspace's, which starts as Error. A job with no severity on its monitor or application, in a workspace whose default was never changed, therefore alerts as Error, our P2. Billing, renewal, and email jobs covers the most common P1 jobs.

Escalate by Class#

CronRadar recommends one detection path for every class and a different route for each class.

Detection Is the Same for Every Class#

CronRadar applies these steps to every monitor, whatever its severity:

1. warning: the expected check-in time has passed and the monitor is within its grace period.
2. critical: the grace period has passed and the check-in is at least 2 minutes late, checked once a minute. A fail check-in makes the monitor critical at once, and so does a completed run that violates an outcome check, with the result outcomefailed.
3. First alert: an InitialFailure alert. A monitor that never reported sends NeverExecuted, and a violated outcome check sends OutcomeFailure.
4. Reminders: a ContinuedFailure alert every 24 hours while the monitor stays overdue, with no check-in arriving, and the incident is unacknowledged. Every check-in, fail included, moves the expected time to the next occurrence, so an open incident gets no reminder while the job keeps checking in on schedule: a job that reports fail or violates an outcome check on every run gets one alert, not daily reminders. Acknowledging the incident in CronRadar stops the reminders, not the monitoring.
5. Recovery: a Recovered alert when a successful run resolves the incident.

Severity does not change any of this timing. The grace default depends on how a monitor is created; Choosing a grace period covers how to size it.

Routing Differs by Class#

Our rule: only P1 routes to PagerDuty. P2 goes to Slack or email, and P3 to a webhook that opens a ticket.

These are the Page, Notify, and Ticket tiers in cron job alerting. Evidence: PagerDuty's incident scale also separates a high-urgency page (SEV-3), a low-urgency page (SEV-4), and a ticket (SEV-5), and places "Cron failure (not impacting event & notification pipeline)" at SEV-4. Our rule differs: such a job is P2 or P3 and is notified by Slack, email, or a ticket, not paged.

A ticket handler gets CronRadar's webhook, which carries the alert type, severity, owner, and the monitor's runbook, logs, and repository URLs. The monitor.still_down event repeats every 24 hours while the monitor stays overdue and unacknowledged, so the handler should update the existing ticket, not open a new one.

The PagerDuty Urgency Caveat#

CronRadar sends each monitor's severity to PagerDuty as the event severity: critical, error, or warning. A recovery is sent as info and resolves the PagerDuty alert.

PagerDuty then decides urgency:

• On a service set to dynamic notifications, critical and error each "Uses high-urgency notification rules and escalates if not acknowledged", and warning "Uses low-urgency notification rules and cannot escalate" (Dynamic Notifications).
• Otherwise the service's own setting applies, and "all services initially default to High-urgency notifications, escalate as needed" (Configurable Service Settings).

So a P2 job, which carries Error, would page at high urgency on a service set to dynamic notifications or on a service left at its default urgency. That is why only P1 routes to PagerDuty. A P1 job carries Critical and pages at high urgency on both.

What CronRadar Leaves to PagerDuty#

CronRadar has no escalation levels, escalation timers, or on-call schedules, and its reminder cadence is the same for every class. For P1, PagerDuty's escalation policy supplies the levels, the timeout (30 minutes by default), and up to nine repeats.

Architecture Rules for Each Scale#

CronRadar recommends the rules below. Each names the research it rests on.

L1 Rules: One Host#

1. Every job checks in. Evidence: cron launches are "fire-and-forget", so nothing records a run that didn't happen.
2. Overlap is handled on purpose: flock for cron, or the systemd timer's built-in skip. Evidence: Slack's scripts that "take longer than their recurrence"; systemd never starts a second instance. Preventing overlapping cron jobs has the commands.
3. Catch-up is a decision. cron makes up nothing, anacron only daily or slower jobs, and systemd only with Persistent=true. Choose per job, using question 1.
4. The API key lives in an environment file, never in the crontab line. This rule rests on CronRadar's own behavior, not on the research: CronRadar check-ins use HTTP Basic authentication, which keeps the key out of "shell history, proxy logs, or copied URLs" (CronRadar HTTP API).

A crontab line with no key on it:

# /etc/cron.d/nightly-export (host clock in UTC; CronRadar evaluates schedules in UTC)
0 2 * * * exporter /usr/bin/flock -n /run/lock/nightly-export.lock /opt/jobs/run-nightly-export.sh

The wrapper it runs:

#!/bin/sh
# /opt/jobs/run-nightly-export.sh
# The env file defines CRONRADAR_API_KEY; it is owned by the job's user, mode 0600.
# If it is missing, the check-ins fail and the job still runs.
if [ -r /etc/cronradar/nightly-export.env ]; then . /etc/cronradar/nightly-export.env; fi
ping="https://cron.life/api/v1/ping/nightly-export"
run_id="nightly-export-$(date -u +%F)"

report() {
  event=$1
  shift
  curl --fail --silent --show-error --max-time 5 --request POST \
    --user "$CRONRADAR_API_KEY:" --get "$ping/$event" \
    --data-urlencode "run_id=$run_id" "$@" >/dev/null || true
}

report start --data-urlencode "schedule=0 2 * * *"
if /opt/jobs/nightly-export; then
  report complete
else
  status=$?
  report fail --data-urlencode "message=nightly-export exited with code $status"
  exit "$status"
fi

A check-in failure never changes the job's result: each report times out after 5 seconds and is ignored if it fails, and the wrapper exits with the job's own status. With flock -n, an occurrence skipped because the previous run still holds the lock sends no check-in. CronRadar alerts only if the previous run is still running, with a start but no complete or fail, when the skipped occurrence's grace period and the 2-minute threshold pass, and it records that alert as a stuck run. If the previous run completes first, the skipped occurrence raises no alert.

L2 Rules: A Scheduler Inside the Application#

1. Exactly one instance schedules. Use the framework's own option: Laravel's onOneServer, a single Celery beat, or a clustered Quartz.NET job store. Evidence: Laravel's report generated "three times" on three servers; Celery's "duplicate tasks". Distributed cron compares the options.
2. Jobs are idempotent or keyed by run. Evidence: Hangfire says "there’s no reliable way to prevent multiple executions of the same background job" without transactions in the job itself.
3. Monitoring runs outside the application. Evidence: our L2 inference that an in-process check stops with its process. CronRadar evaluates schedules on its own servers, so an application that stops produces a missed-run alert. The CronRadar.Hangfire and CronRadar.Quartz extensions discover recurring jobs and create their monitors, so a discovered job that never runs raises NeverExecuted. CronRadar.Quartz 0.1.0 monitors only jobs on repeating simple triggers: it sends a cron trigger's six-field expression unchanged, and CronRadar rejects it until the pending 0.1.1.

L3 Rules: A Scheduler Service With One Leader#

1. One leader, and it only launches. Evidence: Google, Slack, Chronos, and Meson all hand execution to another system.
2. Each run is identified by its scheduled time. Evidence: Google's launch names, Slack's row per execution, and Kubernetes' scheduled-timestamp annotation. In CronRadar, send that identity as run_id on the start and terminal check-ins; an Idempotency-Key header makes CronRadar ignore a duplicate check-in.
3. Choose skip or at-least-once per class. A P1 job that must not run twice fails closed: when the scheduler cannot tell whether a launch happened, it skips, the missed check-in alerts, and a person completes the occurrence before its finish-by deadline. A job that must not be skipped runs at least once, with an idempotency key built from its scheduled time so a second run does nothing. Evidence: Google's preference for skipping and its note that "In case of a skipped launch, cron job owners can take action that appropriately matches the nature of the cron job"; the at-least-once designs of Maestro, Dropbox ATF, and Cloud Scheduler.

L4 Rules: A Shared Platform#

1. An inventory names an owner for every job. Evidence: Google's job owners who monitor their own jobs, Dropbox's lambda owners, and Maestro's ownership management and lineage. Scheduled job inventory shows how to collect one.
2. Start times are jittered or hashed. Evidence: Google's ? field, Netflix's midnight peak, and the jitter settings in Temporal, Cadence, systemd, and EventBridge Scheduler. A CronRadar monitor compares check-ins with the job's cron schedule, so the grace period must cover the jitter.
3. Each team has isolation and quotas. Evidence: Dropbox ATF's dedicated queues, clusters, and quotas; Meta Async's per-use-case quotas; FOQS's per-namespace rate limits; Slack's isolated queue; Cadence's rate limits.
4. Retries are bounded and split by error type. Evidence: Maestro's 2, 10, and 0 defaults; Dropbox's infinite retry loops; the caps in EventBridge Scheduler and Cloud Scheduler. A CronRadar fail check-in opens an incident at once, so a job that retries reports fail after its last attempt. Retrying failed cron jobs shows a wrapper.
5. The platform team owns the scheduler; job owners own their jobs and alerts. Evidence: Google's "Cron job owners can (and should!) monitor their cron jobs" and Dropbox's "lambda owners own all aspects". In CronRadar, a monitor with no owner at any level names the workspace in its alerts.
6. Data dependencies use event triggers, not clock offsets. Evidence: Maestro: "the daily workflow should process the data when the data partition is ready, not always at midnight"; Chronos "supports the definition of jobs triggered by the completion of other jobs". ETL job monitoring covers ordering steps by data.

The Per-Job Contract#

CronRadar recommends one contract card per job, kept with the job's code and in the inventory. The card records the overlap, missed-run, finish-by deadline, and retry behaviors, which the research shows schedulers handle differently, so no default decides them silently.

CronRadar monitors also store an environment and logs and repository URLs. Scheduled job runbooks has a runbook template.

An example card:

job: payroll-export
owner: payroll-oncall
priority: P1 # CronRadar severity: Critical
schedule: '0 6 25 * *' # UTC
overlap: skip # Kubernetes Forbid, Temporal Skip
missed_run: once # systemd Persistent=true
finish_by: '12:00 UTC on the 25th' # the payroll provider's cutoff
retries: # bounded, by error type
  platform: 3
  user: 0
  timeout: 0
run_id: 'payroll-export-2026-10' # job plus scheduled month
check_ins: [start, complete, fail]
outcome_checks:
  - 'employees_missing = 0'
grace_seconds: 1800
destinations: [pagerduty, slack]
runbook_url: 'https://git.example.com/payroll/runbooks/payroll-export.md'

Review the Ranking#

CronRadar recommends three reviews:

1. Reconcile the inventory against monitors on a schedule. A job with no monitor is unprotected, and a monitor with no job is noise. Evidence: cron launches are "fire-and-forget", and Google's job owners "can (and should!)" monitor their jobs; the reconciliation tools below are CronRadar's own. CronRadar's GET /sync/status lists an application's monitors, and a sync in reconcile mode pauses monitors whose jobs disappeared from the scheduler. Scheduled job inventory shows the comparison.
2. Re-rank a job when what depends on it changes. A new consumer with a deadline can move a P3 job to P1. Evidence: HCL "dynamically recalculates the critical start times within the critical network" while the plan runs; Maestro's lineage shows "who depends on whom".
3. Review after every P1 incident. Change the class, severity, destinations, and grace period together. Evidence: PagerDuty's guidance on severity during an incident: "just assume the highest and review during a postmortem" (Severity Levels).

What CronRadar Covers and What It Does Not#

Package versions are those published when CronRadar last checked them, on September 20, 2026.

Dependency sources: Airflow tasks "have upstream and downstream dependencies set between them" (Tasks); in Cadence, "users can define parent-child dependencies" (Announcing Cadence 1.0); Control-M and HCL Workload Automation map critical paths, as cited under question 4.

Frequently Asked Questions#

How do I decide whether a scheduled job is P1, P2, or P3?#

A job is P1 if a run cannot be skipped and has a finish-by deadline, or if a failure loses money, customer-visible data, or backups. It is P2 if a skipped or late run degrades something people or downstream jobs use the same day. Everything else is P3. In CronRadar, P1 maps to Critical severity, P2 to Error, and P3 to Warning.

Which scheduled job failures should page someone?#

Only P1 jobs. CronRadar recommends PagerDuty plus the owning team's channel for P1, Slack or email for P2, and a webhook that opens a ticket for P3. On a PagerDuty service with dynamic notifications, both the critical and error severities page at high urgency, so a P2 job sent there would page like a P1.

Is it better for a scheduled job to skip a run or run twice?#

The sources disagree. Google's SRE book prefers skipping because a skipped launch is easier to recover from than a double launch. Netflix Maestro, Dropbox ATF, Google Cloud Scheduler, and EventBridge Scheduler deliver at least once; Maestro deduplicates triggers, and Dropbox ATF and Cloud Scheduler require idempotent work. CronRadar recommends deciding per job: a job that must not run twice skips when in doubt and alerts, and a job that must not be skipped runs at least once with an idempotency key.

What does a deadline mean in a job scheduler?#

It depends on the tool. A start-by deadline, such as Kubernetes startingDeadlineSeconds, is the latest a late run may start. A finish-by deadline, such as HCL Workload Automation's deadline or Control-M's Complete By, is when the result must exist. A maximum run time, such as Kubernetes activeDeadlineSeconds or HCL maxdur, limits how long a run may take.

Does CronRadar escalate alerts or manage on-call?#

No. CronRadar sends a first alert, a reminder every 24 hours while the monitor stays overdue and the incident is unacknowledged, and a recovery alert, the same for every severity. Escalation levels, escalation timeouts, and on-call schedules belong in PagerDuty, which CronRadar sends alerts to with the monitor's severity.