Datadog Cron Job Monitoring: Options, Gaps, and Dedicated Monitors
Contents
To monitor cron jobs in Datadog, have each run send a signal (a DogStatsD metric, a service check, a log line, or an event) and create a monitor that alerts when that signal reports a failure or stops arriving. Datadog's list of monitor types has no heartbeat or cron type; its own guide for cron jobs uses metric, log, and event monitors evaluated once a day, week, or month. Otherwise a monitor evaluates a window of time, so the alert follows the time since the last signal, not the cron expression. Kubernetes CronJobs are the exception: Datadog's Kubernetes integration flags a CronJob whose next scheduled run is in the past. Checked October 25, 2026, against Datadog's documentation, the datadog Python package 0.55.0, and DogStatsD in Agent v6 and later.
What Datadog Can Watch#
| Signal from the job | Datadog monitor | Missed run | Failed run | Limits |
|---|---|---|---|---|
| A count incremented on success | Metric | The window's sum falls below 1 | Only as a missing success | Custom metrics; window up to one month |
| A service check, OK or CRITICAL | Service check | No check run for no_data_timeframe minutes |
The first CRITICAL status | Messages up to 500 characters, dropped on OK |
| A log line the job already writes | Log | Matching count below 1 |
A count of error lines | Indexed Standard Tier logs only; window up to 2 days |
| An event sent by the job | Event | Count below a threshold | Matching failure events | Window up to 48 hours |
| Nothing: a Kubernetes CronJob's objects | Integration | kubernetes_state.cronjob.on_schedule_check |
kubernetes_state.cronjob.complete |
Kubernetes only; Cluster Agent v1.12+ |
Three other products touch scheduled work. Data Observability: Jobs Monitoring alerts "when a job fails, or is running beyond its completion time" on Databricks, Airflow, dbt, AWS Glue, Azure Data Factory, and Spark. Airflow DAG schedule monitoring covers why a DAG misses its schedule and what Airflow itself can alert on. A CI Pipeline monitor alerts on failed pipelines that CI Visibility collects, including ones your CI provider runs on a schedule. A Live Process monitor, included in the Enterprise plan, can confirm the cron daemon runs, but Datadog warns that monitors on processes living under 20 seconds "may be flaky," so it can't watch the jobs.
Report Each Run With DogStatsD#
DogStatsD, bundled with the Datadog Agent, listens on UDP port 8125: "Because it uses UDP, your application can send metrics to DogStatsD and resume its work without waiting for a response" (DogStatsD). This wrapper runs any command, then sends a success count, the run's duration as a distribution, and a service check carrying OK or CRITICAL, and calls flush, as the submission docs advise before a program exits:
#!/usr/bin/env python3
"""dd-cron: run a command, then report its result to the local Datadog Agent.
Usage: dd-cron <job-name> <command> [args...]
Written for the datadog package 0.55.0 (pip install datadog).
"""
import subprocess
import sys
import time
from datadog import initialize, statsd
if len(sys.argv) < 3:
sys.exit("usage: dd-cron <job-name> <command> [args...]")
job, command = sys.argv[1], sys.argv[2:]
tags = [f"job:{job}", "env:prod"]
started = time.monotonic()
try:
rc = subprocess.call(command)
except OSError as exc: # the command could not be started
print(f"dd-cron: {exc}", file=sys.stderr)
rc = 127
if rc < 0: # killed by a signal: report it the way a shell would
rc = 128 - rc
elapsed = time.monotonic() - started
try:
initialize(statsd_host="127.0.0.1", statsd_port=8125)
statsd.distribution("cron.job.duration", elapsed, tags=tags)
if rc == 0:
statsd.increment("cron.job.success", tags=tags)
statsd.service_check(
"cron.job.status",
statsd.OK if rc == 0 else statsd.CRITICAL,
tags=tags,
message=None if rc == 0 else f"{job} exited with code {rc}",
)
statsd.flush()
except Exception as exc: # monitoring must never change the job's result
print(f"dd-cron: DogStatsD report failed: {exc}", file=sys.stderr)
sys.exit(rc)The crontab entry is 30 2 * * * /usr/local/bin/dd-cron nightly-backup /opt/backup.sh. Reporting happens after the job, so it can't delay or fail it, and the script exits with the job's own code. If the host's Agent is stopped, the UDP report is lost, and the monitors below can't tell that from a run that never happened. Hosts without an Agent can post service checks to /api/v1/check_run instead.
Alert When Successes Stop Arriving#
A metric monitor turns the success count into a dead man's switch. This is the request body for Datadog's create-monitor API:
{
"name": "nightly-backup has not succeeded in 26 hours",
"type": "query alert",
"query": "sum(last_26h):sum:cron.job.success{job:nightly-backup,env:prod}.as_count() < 1",
"message": "No successful nightly-backup run in 26 hours. @[email protected]",
"options": {
"thresholds": { "critical": 1 },
"on_missing_data": "default",
"require_full_window": false
}
}With sum aggregation, the default missing-data behavior evaluates an empty window as 0 (On Missing Data guide), so a window with no success trips < 1. The API reference recommends require_full_window: false for sparse metrics, "otherwise some evaluations are skipped." Monitors with a window between 24 and 48 hours evaluate every 10 minutes (evaluation frequency).
Older monitors and Terraform configs use notify_no_data and no_data_timeframe, once labeled "Notify if data is missing." For metric monitors, Datadog's guide says to replace both with on_missing_data, which uses the monitor's own window. Its recipe for a tag that stops reporting, an impossible threshold plus the missing-data notification, is the same dead man's switch.
One caveat applies to all of them: "The monitor must be able to evaluate data before alerting on missing data" (monitor configuration). A job that was never installed, or reports under a misspelled tag, may never alert, so check that the monitor shows OK after the first real run.
A Rolling Window Measures Time, Not the Schedule#
The 26-hour window covers a 24-hour interval plus two hours of slack. If Monday's 02:30 run succeeds at 02:40 UTC and Tuesday's never runs, the window empties at 04:40 Tuesday and the alert follows within one 10-minute evaluation. Shrink the slack and a slow night pages someone; widen it and a missed run waits longer.
Irregular schedules don't fit any single window. A job on 0 6 * * 1-5 breaks a 26-hour window every Saturday morning, and a 74-hour window reports a missed Tuesday run on Thursday. A recurring downtime with an RRULE can mute weekends, but downtimes "do not prevent monitor states transitions," and a monitor still alerting when the downtime expires notifies then, so the downtime must outlast Monday's run. Monitor cron jobs with Prometheus shows the same problem with PromQL age rules.
Custom Schedules Cover Daily and Longer Jobs#
Custom schedules are Datadog's schedule-aware answer. A metric, log, or event monitor evaluates at a set time: daily, on chosen weekdays, on a day of the month, or by RRULE. "Each evaluation is independent from the previous one and sends a notification when the result is not OK." Datadog's examples include checking on the first of each month "whether the cron job generating customer invoices has successfully run." The API's recurrences take a time zone.
The limits, from the same guide: "Evaluation frequencies must be a day or longer," so hourly and every-15-minutes jobs stay on rolling windows. A custom schedule can't be removed once added; you recreate the monitor. And the API reference labels the custom_schedule field "in private beta," so confirm your account has it before planning around it.
Catch Failures Directly With a Service Check Monitor#
The wrapper's cron.job.status check carries each run's result, so a service check monitor can alert on the first CRITICAL instead of waiting for a window to empty:
{
"name": "nightly-backup failed or stopped reporting",
"type": "service check",
"query": "\"cron.job.status\".over(\"job:nightly-backup\").by(\"job\").last(2).count_by_status()",
"message": "nightly-backup failed or stopped reporting. @[email protected]",
"options": {
"thresholds": { "critical": 1, "ok": 1 },
"notify_no_data": true,
"no_data_timeframe": 1560
}
}The thresholds count consecutive statuses: one CRITICAL alerts, one OK resolves, and last(2) must be at least the largest threshold (query syntax). The API reference doesn't list on_missing_data for service check monitors, so they keep the older pair: no_data_timeframe is "the number of minutes before a monitor notifies after data stops reporting," here 26 hours, and defaults to 24 hours for service checks. Grouping by job rather than host means a job moved to a new host doesn't leave a silent old-host group behind. Messages are limited to 500 characters and "dropped for checks with an OK status" (service checks API).
Logs and Events Need No New Metric#
If the job's output already reaches Datadog Log Management, a log monitor can count a line such as backup complete. Datadog's log monitor docs say to "set the condition to below 1" to be notified when matching logs stop. The monitor evaluates only indexed Standard Tier logs over at most 2 days. Grouped, a below condition fires for a group only if that group has logs below the threshold, or if no group has any. A job that goes silent while the monitor's other jobs still log doesn't trip it, so give each critical job its own ungrouped monitor.
Event monitors work the same way on events your job sends through DogStatsD or the API, with a window of up to 48 hours (query syntax).
Kubernetes CronJobs: The Integration Tracks the Schedule#
On Kubernetes, the Kubernetes State Core check, which ships in the Datadog Cluster Agent (v1.12 or later) and is turned on in its configuration, reports CronJob state with no code in the job:
kubernetes_state.cronjob.on_schedule_check, a service check documented as "Alert if the cronjob's next schedule is in the past."kubernetes_state.cronjob.complete: "Whether the last job of the cronjob is failed or not."kubernetes_state.cronjob.duration_since_last_successfulandkubernetes_state.cronjob.spec_suspend, for age and suspended CronJobs.kubernetes_state.job.duration: "Time elapsed between the start and completion time of the job or the current time if the job is still running."
That last metric can flag a hung Job, something a report sent when the job ends never can. Kubernetes State Core isn't among the integrations Datadog lists as able to emit custom metrics. These checks see Kubernetes objects, not the work: a container that exits 0 after writing nothing looks successful. Kubernetes CronJob monitoring covers the failure modes behind these signals.
Duration, Hangs, and What They Cost#
The wrapper's cron.job.duration distribution is aggregated server-side, and once percentile aggregations are enabled on it, a metric monitor can alert on a percentile such as p95 over its window (distributions). Duration arrives only when the run ends, so a hung job sends nothing; outside Kubernetes, the success-count monitor catches it only once its window empties. Put a time limit inside the job, as cron job duration tracking shows, so a hang becomes a failure.
Custom metrics carry the cost. Datadog identifies one as "a combination of a metric name and tag values (including the host tag)," bills the hourly count of distinct series averaged over the month, and says usage "is not impacted by data point submission frequency" (custom metrics billing, checked October 25, 2026). A distribution counts five per tag combination, ten with percentiles, so the wrapper creates six per job and host: one for the success count, five for duration. Pro accounts get 100 indexed custom metrics per host in their allotment, Enterprise 200, and usage beyond that is billed at the rate in your contract. Log monitors add the cost of indexing the logs they count.
Datadog or a Dedicated Cron Monitor#
| Datadog, built as above | Dedicated cron monitor (CronRadar) | |
|---|---|---|
| When a run is expected | A rolling window per monitor, or a custom schedule evaluated daily or less often; on Kubernetes, each CronJob's next run | The job's five-field cron expression in UTC, plus a grace period |
| Failed run | First CRITICAL service check, or a failure count or log match | A fail check-in alerts immediately |
| Hung run | Silent until a window empties; on Kubernetes, kubernetes_state.job.duration keeps growing |
A start with no finish turns critical after the next expected run plus grace |
| Duration | Distributions, percentiles, dashboards, anomaly detection | Last successful duration; alerts only through a reported outcome |
| Path from job to alert | The host's Agent, the log pipeline, or the API | An HTTPS check-in from the job to https://cron.life |
| What you pay for | Custom metrics by tag combination; indexed logs | Active monitors by schedule class |
| Everything else | Infrastructure, APM, logs, and dashboards in one place | Nothing else: no metrics, logs, or uptime checks |
Datadog is the better fit when it's already on every host and someone owns it. Job signals sit next to the host's CPU, logs, and traces; Kubernetes CronJobs get schedule-aware checks with no code; Airflow, Databricks, and dbt pipelines have Jobs Monitoring; duration history feeds dashboards and anomaly detection; and daily or longer jobs fit custom schedules, time zones included.
A dedicated monitor fits when many jobs run on irregular or sub-daily schedules and each would need a hand-tuned window, when you want an explicit start, complete, and fail lifecycle with stuck detection, or when the alert path shouldn't depend on an Agent or log pipeline on the same host. Push vs pull monitoring covers why a report that never arrives needs an expectation to be noticed.
Running CronRadar Next to Datadog#
CronRadar is one dedicated option. A job checks in to https://cron.life with a heartbeat, or with start, complete, and fail. CronRadar reads the five-field cron schedule in UTC, and every check-in moves the expected time to the next scheduled occurrence, so after Friday's run of 0 6 * * 1-5 it expects Monday at 06:00 and the weekend needs no rule. Its overdue check runs once a minute: a late monitor is warning once the expected time passes and critical once it is past the grace period (90 seconds by default, 60 minimum) and at least two minutes late. A fail alerts immediately. A complete sent over HTTP can carry up to five numeric or boolean outcomes checked against expectations you configure, such as duration_seconds <= 900, the only way to alert on a slow run. Alerts go to email, Slack, webhook, Discord, or PagerDuty, with at most one reminder every 24 hours while an incident stays open and a recovery message when a successful run resolves it.
There's no installable CronRadar SDK for Python or shell today and no auto-discovery for crontab, so each job reports with its own HTTP check-in. This wrapper sends check-ins around dd-cron, so one crontab entry feeds both. Each check-in gives up after five seconds and ignores errors, and the job's exit code passes through:
#!/bin/sh
# /usr/local/bin/cron-run: CronRadar check-ins around dd-cron.
# Usage: cron-run <job-name> '<schedule in UTC>' <command> [args...]
name=$1
schedule=$2
shift 2
run_id="$name-$(date +%s)-$$"
# The application's API key, alone in a file only the job's account can read
CRONRADAR_API_KEY=$(cat /etc/cronradar/api-key) || CRONRADAR_API_KEY=
report() {
event=$1
shift
curl --fail --silent --show-error --max-time 5 --output /dev/null \
--request POST --get --user "$CRONRADAR_API_KEY:" \
--data-urlencode "schedule=$schedule" \
--data-urlencode "gracePeriod=900" \
--data-urlencode "run_id=$run_id" \
"$@" "https://cron.life/api/v1/ping/$name/$event" || true
}
report start
/usr/local/bin/dd-cron "$name" "$@"
rc=$?
if [ "$rc" -eq 0 ]; then
report complete
else
report fail --data-urlencode "message=$name exited with code $rc"
fi
exit "$rc"The entry is 30 2 * * * /usr/local/bin/cron-run nightly-backup '30 2 * * *' /opt/backup.sh on a UTC server; elsewhere, pass the UTC time. The schedule and grace period apply only when a check-in creates the monitor; change them later in the monitor's settings. Sending them on every run recreates a deleted monitor on the next one.
Its limits: schedules are UTC only, and when both day fields are restricted CronRadar expects a run only on dates matching both, while Linux cron runs on either. Stuck detection is per job, not per run: the next run's start resets it, and there is no maximum-runtime setting. A monitor created by its first check-in can't flag a job that never runs; one created ahead of time in the dashboard stays pending and sends a never-executed alert once its first expected run plus grace passes. It collects no metrics streams, so trends stay in Datadog.
Pricing: Free covers up to 10 active monitors with every alert channel and no card, and keeps 30 days of run history and 90 of alerts. On Grow, the first 10 active monitors stay free; each further monitor is billed monthly by schedule class, $2 Fast (gap under 5 minutes), $1 Frequent (5 minutes to under an hour), or $0.50 Standard (hourly or less often), in arrears on the month's peak, with no tiers, minimum, or monitor limit, and an optional monthly budget that holds new monitors paused. Grow keeps 90 days of run history and 180 of alerts. Business is for custom requirements. Runs, retries, and API calls are never billed. Cron monitoring tools compared covers other hosted services.
Frequently Asked Questions#
Can Datadog monitor cron jobs?#
Yes, through general-purpose monitors: the job sends a metric, service check, log line, or event on each run, and a matching monitor alerts on a failure or on the signal going missing. Datadog's list of monitor types has no dedicated heartbeat or cron type. Kubernetes CronJobs need no job code, because the Kubernetes State Core check reports their state.
How do I alert in Datadog when a cron job doesn't run?#
Increment a count on every successful run and create a metric monitor such as sum(last_26h):sum:cron.job.success{job:nightly-backup}.as_count() < 1 with the default missing-data behavior, which evaluates an empty window as zero. The window must cover the job's interval plus slack. For a daily or less frequent job, a custom schedule can instead evaluate at a set time after the run is due.
Does Datadog understand cron schedules?#
Not as cron expressions. Monitors evaluate a window of time, or, with custom schedules, run at times you set by day, weekday, day of month, or RRULE, no more often than daily. On Kubernetes, the kubernetes_state.cronjob.on_schedule_check service check alerts when a CronJob's next schedule is in the past.
How do I track cron job duration in Datadog?#
Send the run's elapsed time as a DogStatsD distribution, as the wrapper above does, then graph it, or enable percentiles and alert on one. Each tag combination counts as five custom metrics, ten with percentiles. On Kubernetes, kubernetes_state.job.duration measures each Job and keeps growing while one runs.
Should I use Datadog or a dedicated cron monitor?#
If Datadog already runs everywhere and your jobs are on Kubernetes, daily or longer schedules, or supported data platforms, its monitors may be all you need, and they keep job signals beside everything else. Add a dedicated cron monitor when many jobs run on irregular or sub-daily schedules, when you want start, complete, and fail tracking with stuck detection, or when the alert shouldn't depend on an Agent on the same host. The wrapper above shows the two running side by side.
Put this guide to work
Know when the next run is missing
Connect one scheduled job, confirm a real completion, then verify the alert destination. Keep your existing scheduler in charge of running the work.