# Monitor Cron Jobs With Prometheus: Textfile, Pushgateway, Alerts

By Dan Ochoa · Published 2026-10-25 · Updated 2026-10-25
Canonical: https://cronradar.com/blog/monitor-cron-jobs-prometheus

> Monitor cron jobs with Prometheus: last-success metrics via the textfile collector or Pushgateway, alert rules for late and missing runs, and their limits.

**To monitor cron jobs with Prometheus, have every run record a last-success timestamp, through node_exporter's textfile collector on the job's host or the Pushgateway for jobs tied to no host, and alert when `time()` minus that timestamp exceeds what the job's schedule allows.** Add the run's duration and exit code, a rule for series that disappear, and an always-firing Watchdog alert for the monitoring stack itself. Prometheus never learns the cron schedule, so each threshold is a fixed age you choose by hand; on Kubernetes, kube-state-metrics fills part of that gap. Versions checked October 25, 2026: Prometheus 3.15, node_exporter 1.12, Pushgateway 1.11, kube-state-metrics 2.20, Alertmanager 0.34.

[Cron monitoring vs uptime monitoring](https://cronradar.com/blog/cron-monitoring-vs-uptime-monitoring) maps which tools catch which scheduled-job failures; this guide goes deep on the Prometheus side.

## Record a Last-Success Timestamp, Plus Duration and Exit Code

Prometheus's instrumentation guide is direct: "The key metric of a batch job is the last time it succeeded." It also suggests the overall runtime and the last completion time, all as gauges ([Prometheus](https://prometheus.io/docs/practices/instrumentation/#batch-jobs)). For a cron job, three gauges:

| Metric                                    | Value                                   | Written         |
| ----------------------------------------- | --------------------------------------- | --------------- |
| `cron_job_last_success_timestamp_seconds` | Unix time of the last run that exited 0 | Only on success |
| `cron_job_last_duration_seconds`          | Wall-clock seconds of the last run      | Every run       |
| `cron_job_last_exit_code`                 | Exit status of the last run             | Every run       |

A cron job exits after each run, so there's nothing to scrape in between; a gauge left behind by the last run is what Prometheus reads. Record times in seconds, Prometheus's base unit ([naming](https://prometheus.io/docs/practices/naming/#base-units)), so the age is `time()` minus the gauge. Name the job in a label of your own, `cron_job` here: Prometheus attaches `job` and `instance` to scraped series and renames clashing labels to `exported_job` and `exported_instance` ([scrape_config](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config)).

## On the Job's Host, Write a Textfile for node_exporter

For batch jobs tied to a machine, Prometheus recommends node_exporter's textfile collector over the Pushgateway ([Prometheus](https://prometheus.io/docs/practices/pushing/)). On every scrape, node_exporter 1.12 parses each file ending in `.prom` in the directories named by `--collector.textfile.directory` ([textfile.go](https://github.com/prometheus/node_exporter/blob/v1.12.1/collector/textfile.go)). Upstream, the flag has no default. Debian's `prometheus-node-exporter` package (1.9.0 in Debian 13) sets it to `/var/lib/prometheus/node-exporter`, where files must be readable by the `prometheus` user ([README.textfile](https://sources.debian.org/src/prometheus-node-exporter/1.9.0-1/debian/README.textfile/)).

Four rules for the files:

- **Text exposition format:** `\n` line endings, and `\`, `"`, and newlines escaped in label values ([exposition formats](https://prometheus.io/docs/instrumenting/exposition_formats/)). Job names of letters, digits, and dashes need no escaping.
- **No timestamps.** A sample with its own timestamp makes node_exporter 1.12 skip the whole file.
- **Write, then rename.** A scrape can arrive mid-write, which is why [node_exporter's README](https://github.com/prometheus/node_exporter/blob/v1.12.1/README.md#textfile-collector) writes a temporary file and moves it into place. rename(2) replaces the target atomically only within one filesystem ([rename(2)](https://man7.org/linux/man-pages/man2/rename.2.html)), so keep the temporary file in the same directory.
- **Permissions.** The job's account needs write access to the directory, node_exporter's account read access to the file. `mktemp` creates files only their owner can read ([GNU coreutils](https://www.gnu.org/software/coreutils/manual/html_node/mktemp-invocation.html)), so set the mode before renaming.

A file that fails to parse is dropped and `node_textfile_scrape_error` becomes 1, which is worth its own alert. This wrapper runs any command and writes the three gauges:

```bash
#!/bin/sh
# /usr/local/bin/cron-metrics: run a command, record its result for node_exporter.
# Usage: cron-metrics <job-name> <command> [args...]
dir=/var/lib/prometheus/node-exporter
name=$1
shift

# Write a temporary name the collector ignores, then rename it into place.
write_prom() {
  tmp="$dir/.$1.prom.$$"
  cat >"$tmp" && chmod 0644 "$tmp" && mv -f "$tmp" "$dir/$1.prom" || rm -f "$tmp"
}

start=$(date +%s)
"$@"
rc=$?
end=$(date +%s)

write_prom "$name" <<EOF
# TYPE cron_job_last_duration_seconds gauge
cron_job_last_duration_seconds{cron_job="$name"} $((end - start))
# TYPE cron_job_last_exit_code gauge
cron_job_last_exit_code{cron_job="$name"} $rc
EOF

# Only a successful run moves the success timestamp.
if [ "$rc" -eq 0 ]; then
  write_prom "$name-success" <<EOF
# TYPE cron_job_last_success_timestamp_seconds gauge
cron_job_last_success_timestamp_seconds{cron_job="$name"} $end
EOF
fi

exit "$rc"
```

The crontab entry becomes `30 2 * * * /usr/local/bin/cron-metrics nightly-backup /opt/backup.sh`. The success timestamp has its own file, so a failed run leaves it alone. A failed metrics write only prints an error, and the wrapper exits with the job's own status. If the wrapper itself is killed or the host reboots mid-run, nothing is written; the age rule catches that later. [Shell script best practices](https://cronradar.com/blog/cron-shell-script-best-practices) covers exit codes, traps, and time limits for the job itself.

## Off-Host Jobs Push to the Pushgateway, With Care

Prometheus "only recommend[s] using the Pushgateway in certain limited cases," usually "capturing the outcome of a service-level batch job," one not tied to a machine ([When to use the Pushgateway](https://prometheus.io/docs/practices/pushing/)). The same page lists the costs: a single point of failure and bottleneck, no per-target `up` metric, and pushed series kept until you delete them. That last property suits a last-success timestamp: a job that stops running leaves its old timestamp behind, and the age rule fires.

```bash
#!/bin/sh
# /usr/local/bin/cron-push: run a command, push its result to the Pushgateway.
# Usage: cron-push <job-name> <command> [args...]
gateway=http://pushgateway.internal:9091
name=$1
shift

start=$(date +%s)
"$@"
rc=$?
end=$(date +%s)

{
  echo "# TYPE cron_job_last_duration_seconds gauge"
  echo "cron_job_last_duration_seconds $((end - start))"
  echo "# TYPE cron_job_last_exit_code gauge"
  echo "cron_job_last_exit_code $rc"
  if [ "$rc" -eq 0 ]; then
    echo "# TYPE cron_job_last_success_timestamp_seconds gauge"
    echo "cron_job_last_success_timestamp_seconds $end"
  fi
} | curl --fail --silent --show-error --max-time 5 --output /dev/null \
  --data-binary @- "$gateway/metrics/job/cron/cron_job/$name" || true

exit "$rc"
```

What matters, per the [Pushgateway 1.11 README](https://github.com/prometheus/pushgateway/blob/v1.11.4/README.md):

- **POST, not PUT.** `curl --data-binary` sends a POST, which replaces only the metrics named in the push. PUT replaces the whole group, so a failed run pushed with PUT would delete the success timestamp. Python's `push_to_gateway` sends PUT and `pushadd_to_gateway` POST, both with a 30-second default timeout worth lowering ([client_python 0.26](https://github.com/prometheus/client_python/blob/v0.26.0/prometheus_client/exposition.py)); Go's batch-job example calls `Add`, not `Push`, for this reason ([client_golang 1.25](https://github.com/prometheus/client_golang/blob/v1.25.0/prometheus/push/example_add_from_gatherer_test.go)).
- **The URL path is the grouping key.** `job="cron"` (always first) and `cron_job="nightly-billing"` identify the group and overwrite those labels in the body. Leave `instance` out, so moving the job to another host doesn't strand an old group.
- **Set `honor_labels: true`** on the scrape job, or the pushed `job` becomes `exported_job`.
- **Persistence is off by default.** Without `--persistence.file`, a restart loses everything; with it, a push can wait up to `--persistence.interval`, 5 minutes by default, before reaching disk ([main.go](https://github.com/prometheus/pushgateway/blob/v1.11.4/main.go)).
- **Delete retired groups**, `curl -X DELETE http://pushgateway.internal:9091/metrics/job/cron/cron_job/nightly-billing`, or the retired job's age alert never stops.
- **Pushed timestamps are rejected.** Each group gets `push_time_seconds`, its last successful push, which doubles as a last-run signal.

```yaml
scrape_configs:
  - job_name: pushgateway
    honor_labels: true
    static_configs:
      - targets: ['pushgateway.internal:9091']
```

## Alert on Age, Failure, and Missing Series

Both wrappers produce the same metric names, so one rules file serves both. Load it through `rule_files` in `prometheus.yml`:

```yaml
groups:
  - name: cron-jobs
    rules:
      # One per job: 30 2 * * *, finishes by 03:10 UTC.
      - alert: CronJobNotSucceeding
        expr: time() - cron_job_last_success_timestamp_seconds{cron_job="nightly-backup"} > 26 * 3600
        keep_firing_for: 10m
        labels:
          severity: page
        annotations:
          summary: '{{ $labels.cron_job }} last succeeded {{ $value | humanizeDuration }} ago'

      - alert: CronJobFailed
        expr: cron_job_last_exit_code != 0
        labels:
          severity: page
        annotations:
          summary: '{{ $labels.cron_job }} on {{ $labels.instance }} exited with code {{ $value }}'

      # Seen in the past week, gone now: file removed, exporter or host down, gateway wiped.
      - alert: CronJobMetricsVanished
        expr: |
          max_over_time(cron_job_last_success_timestamp_seconds[7d])
            unless cron_job_last_success_timestamp_seconds
        for: 15m
        labels:
          severity: page

      # A job that must exist, including one that has never succeeded.
      - alert: CronJobSuccessMissing
        expr: absent_over_time(cron_job_last_success_timestamp_seconds{cron_job="nightly-backup"}[1h])
        labels:
          severity: page
```

**The age threshold** is the schedule's interval plus the longest normal run plus slack. For `30 2 * * *` finishing by 03:10, 26 hours alerts by about 05:10 on the morning of a missed run. Prometheus's alerting guide suggests "at least enough time for 2 full runs of the batch job," so one failure doesn't page: 10 hours for a job that runs every 4 hours and takes 1 ([alerting practices](https://prometheus.io/docs/practices/alerting/#batch-jobs)). Use one run when a single miss matters; [choosing a grace period](https://cronradar.com/blog/cron-grace-period) shows how to size the slack from measured run times. To skip one rule per job, have the wrapper also write a `cron_job_max_age_seconds` gauge and alert on `time() - cron_job_last_success_timestamp_seconds > cron_job_max_age_seconds`.

**`for` and `keep_firing_for`.** The threshold already holds the tolerance, so `for` on the age rule only adds delay. But a failed scrape makes series stale at once ("A failed scrape is the same as an empty scrape," [scrape.go](https://github.com/prometheus/prometheus/blob/v3.15.1/scrape/scrape.go)), stale series return nothing ([staleness](https://prometheus.io/docs/prometheus/latest/querying/basics/#staleness)), and the alert would resolve and refire. `keep_firing_for: 10m` rides out the gap ([alerting rules](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/)). Put `for` on the presence rules, where a restart shouldn't page.

**Missing series are the quiet failure.** When the success series is gone (file deleted, host or exporter down, Pushgateway restarted without persistence), the age expression returns nothing, and nothing fires. The vanished rule needs no job names but stops firing seven days later, and it never sees a job that never reported. For jobs that must exist, name them: `absent_over_time()` returns 1 when a selector had no samples in its window, and `absent()` with a `for` clause does the same ([functions](https://prometheus.io/docs/prometheus/latest/querying/functions/#absent)).

**The failure rule** fires on the first evaluation after a scrape sees a non-zero exit code and stays firing until a successful run writes 0.

### A Fixed Age Can't Follow a Cron Schedule

A job on `0 6 * * 1-5`, weekdays at 06:00 UTC, breaks a 25-hour threshold every Saturday, and a 73-hour one reports a missed Tuesday run on Thursday. PromQL's `day_of_week()` and `hour()` return the evaluation time's weekday and hour in UTC ([functions](https://prometheus.io/docs/prometheus/latest/querying/functions/#day_of_week)), so you can write the schedule in by hand:

```yaml
- alert: CronJobNotSucceeding
  # 0 6 * * 1-5, finishes by 06:30 UTC. Quiet Saturday, Sunday, and Monday before 07:00.
  expr: |
    (time() - cron_job_last_success_timestamp_seconds{cron_job="weekday-report"} > 25 * 3600)
      unless on() (day_of_week() == 6 or day_of_week() == 0 or (day_of_week() == 1 and hour() < 7))
  keep_firing_for: 10m
  labels:
    severity: page
```

It works, and it shows the cost: the crontab line rewritten in PromQL, once per irregular schedule, and still approximate. A missed Friday run alerts Friday morning, then resolves at midnight UTC as if fixed. A job on the 1st of each month needs a threshold above 31 days, so a missed March 1 run after a 28-day February goes unreported until March 4. And both functions use UTC, so a schedule that follows daylight saving drifts against the rule twice a year.

## Kubernetes CronJobs: kube-state-metrics Knows the Schedule

On Kubernetes, the exporter reads the CronJob object, schedule included. kube-state-metrics 2.20 computes `kube_cronjob_next_schedule_time` from the schedule and `timeZone` as the next run after the last scheduled one, and omits it while the CronJob is suspended ([cronjob.go](https://github.com/kubernetes/kube-state-metrics/blob/v2.20.0/internal/store/cronjob.go)). With the stable `kube_cronjob_status_last_successful_time` ([CronJob metrics](https://github.com/kubernetes/kube-state-metrics/blob/v2.20.0/docs/metrics/workload/cronjob-metrics.md)), that gives schedule-aware rules:

```yaml
- alert: KubeCronJobRunNotStarted
  expr: time() - kube_cronjob_next_schedule_time > 600
  labels:
    severity: page
  annotations:
    summary: '{{ $labels.namespace }}/{{ $labels.cronjob }}: no Job yet for the run due {{ $value | humanizeDuration }} ago'

- alert: KubeCronJobLatestRunNotSucceeded
  # The latest scheduled run has not succeeded within an hour of its scheduled time.
  expr: |
    kube_cronjob_status_last_successful_time < kube_cronjob_status_last_schedule_time
      and time() - kube_cronjob_status_last_schedule_time > 3600
  labels:
    severity: page
```

For failed runs, alert on `kube_job_failed{condition="true"}`, not `kube_job_status_failed`, which counts failed Pods, including ones a retry replaced, until the Job fails for a known reason ([job.go](https://github.com/kubernetes/kube-state-metrics/blob/v2.20.0/internal/store/job.go)). Deleted objects drop off the metrics endpoint ([kube-state-metrics](https://github.com/kubernetes/kube-state-metrics/blob/v2.20.0/README.md)), so a deleted CronJob goes quiet like a deleted textfile. [Kubernetes CronJob monitoring](https://cronradar.com/blog/kubernetes-cronjob-monitoring) covers events, history limits, and run-time alerts.

## Watch the Watcher With an Always-Firing Alert

Every rule above runs inside the stack it watches. If Prometheus stops, nothing is evaluated; if Alertmanager stops or loses its route out, alerts go nowhere. Prometheus's guide asks for alerts on its servers, Alertmanagers, and Pushgateways, preferably as one end-to-end test ([metamonitoring](https://prometheus.io/docs/practices/alerting/#metamonitoring)). The usual form is kube-prometheus 0.19's Watchdog, whose expression is `vector(1)`, so it always fires ([general.libsonnet](https://github.com/prometheus-operator/kube-prometheus/blob/v0.19.0/jsonnet/kube-prometheus/components/mixin/alerts/general.libsonnet)). Alertmanager repeats it to an external service that alerts when it stops arriving ([Watchdog runbook](https://runbooks.prometheus-operator.dev/runbooks/general/watchdog/)): a [dead man's switch](https://cronradar.com/blog/cron-dead-mans-switch) for your monitoring.

```yaml
# Prometheus rule
- alert: Watchdog
  expr: vector(1)
  labels:
    severity: none
```

```yaml
# alertmanager.yml: merge this route and receiver into your existing config
route:
  routes:
    - matchers:
        - alertname="Watchdog"
      receiver: external-heartbeat
      group_wait: 0s
      group_interval: 1m
      repeat_interval: 1m

receivers:
  - name: external-heartbeat
    webhook_configs:
      - url_file: /etc/alertmanager/heartbeat-url
        send_resolved: false
```

Alertmanager checks `repeat_interval` only at each `group_interval`, rounding up to a multiple ([route](https://prometheus.io/docs/alerting/latest/configuration/#route)), so this sends one notification a minute. `url_file` keeps the receiving URL, often a secret, out of the config ([webhook_config](https://prometheus.io/docs/alerting/latest/configuration/#webhook_config)). The Watchdog covers Prometheus down, Alertmanager down, a broken route, and lost egress. It doesn't cover a single exporter or Pushgateway, which the vanished-series and `up == 0` rules handle, or a selector typo that matches nothing and never fires; `promtool test rules` catches that ([unit testing rules](https://prometheus.io/docs/prometheus/latest/configuration/unit_testing_rules/)).

## What the DIY Setup Costs Compared With a Hosted Heartbeat Monitor

|                        | Prometheus, built as above                                                  | Hosted heartbeat monitor                               |
| ---------------------- | --------------------------------------------------------------------------- | ------------------------------------------------------ |
| New service            | None beyond node_exporter; a Pushgateway for off-host jobs                  | Another vendor and account                             |
| Where the data lives   | Your TSDB, queryable next to every other metric                             | The vendor's service                                   |
| Missed-run expectation | A fixed age per job; weekday and monthly schedules need hand-written PromQL | The cron expression itself, if the service accepts one |
| Per-job setup          | A wrapper call, an age rule, and an `absent` rule for jobs that must exist  | A wrapper call carrying the schedule and grace period  |
| Alert depends on       | Exporter or gateway, Prometheus, Alertmanager, and egress, plus a Watchdog  | The vendor being up, not your hosts or network         |
| Trends and dashboards  | Full history, graphs, any PromQL                                            | Whatever the service records and keeps                 |

Prometheus wins when it's already running and someone owns it: no new dependency, data in your stack, duration trends beside every other graph. It's weakest at the expectation itself, wherever cron is irregular, and every alert path runs through infrastructure that can fail along with the job. A hang is slow to surface either way unless the job enforces its own time limit.

### Running CronRadar Alongside Prometheus

CronRadar is one hosted option. A job checks in to `https://cron.life` with a heartbeat, or with start, complete, and fail. CronRadar reads the five-field cron schedule in UTC, expects each check-in at the next scheduled occurrence, and waits a grace period (90 seconds by default, 60 minimum) before a late run goes from `warning` to `critical`. After Friday's run of `0 6 * * 1-5`, the next expected run is Monday at 06:00, so the weekend needs no rule. A `fail` alerts immediately, a start with no finish is flagged as stuck, and a `complete` sent over HTTP can carry up to five numeric or boolean outcomes; a missing or violated expectation opens an incident. Alerts go to email, Slack, webhook, Discord, or PagerDuty.

Its limits: schedules are UTC only, and when both day fields are restricted it expects a run only on dates matching both, while Linux cron runs on either. Stuck detection is per job, not per run: it fires only once the job's next expected run plus grace passes with no other check-in, the next run's `start` resets it, and there's no maximum-runtime setting. There's no CronRadar SDK or auto-discovery for shell scripts or crontab; each job reports with its own HTTP check-in. It collects no metrics streams, so dashboards and long-term trends stay in Prometheus. [Cron monitoring tools compared](https://cronradar.com/comparisons/best-cron-monitoring-tools) covers other hosted services.

Both can run from one crontab entry. This wrapper sends CronRadar check-ins around the `cron-metrics` script above; each check-in gives up after five seconds and ignores errors, and the job's exit code passes through:

```bash
#!/bin/sh
# /usr/local/bin/cron-run: CronRadar check-ins around cron-metrics.
# Usage: cron-run <job-name> '<schedule in UTC>' <command> [args...]
name=$1
schedule=$2
shift 2
run_id="$name-$(date +%s)-$$"
# The application's API key, alone in a file only the job's account can read
CRONRADAR_API_KEY=$(cat /etc/cronradar/api-key) || CRONRADAR_API_KEY=

report() {
  event=$1
  shift
  curl --fail --silent --show-error --max-time 5 --output /dev/null \
    --request POST --get --user "$CRONRADAR_API_KEY:" \
    --data-urlencode "schedule=$schedule" \
    --data-urlencode "gracePeriod=900" \
    --data-urlencode "run_id=$run_id" \
    "$@" "https://cron.life/api/v1/ping/$name/$event" || true
}

report start
/usr/local/bin/cron-metrics "$name" "$@"
rc=$?
if [ "$rc" -eq 0 ]; then
  report complete
else
  report fail --data-urlencode "message=$name exited with code $rc"
fi
exit "$rc"
```

The entry is `30 2 * * * /usr/local/bin/cron-run nightly-backup '30 2 * * *' /opt/backup.sh` on a UTC server; elsewhere, pass the UTC time. The schedule and grace period apply only when a check-in creates the monitor; change them later in the monitor's settings. Sending them every time recreates a deleted monitor on the next run.

## Frequently Asked Questions

### Can Prometheus monitor cron jobs?

Yes, indirectly. Each run leaves a last-success timestamp in a textfile or the Pushgateway, and a rule fires when it gets too old. Prometheus doesn't know the cron schedule, so you pick each threshold; on Kubernetes, kube-state-metrics exports the next scheduled time.

### Should I use Pushgateway for cron jobs?

Only for service-level jobs not tied to one machine; for jobs on a scraped host, Prometheus recommends the textfile collector. With the Pushgateway, push with POST so a failed run keeps the last success, set `honor_labels: true`, enable persistence, and delete retired groups.

### How do I alert when a cron job hasn't run in Prometheus?

Alert on `time() - cron_job_last_success_timestamp_seconds > threshold`, with the threshold covering one schedule interval, the job's longest run, and some slack. Add an `absent()` or vanished-series rule, because the age expression can't fire once its series is gone.

### What is the node_exporter textfile collector?

A node_exporter collector that exposes metrics from `.prom` files in the directory set by `--collector.textfile.directory`, alongside the host's other metrics. Batch jobs write results there in the Prometheus text format, to a temporary file that is then renamed into place.

### Who monitors Prometheus itself?

Something outside it. An always-firing Watchdog alert, repeated by Alertmanager to an external service that alerts when the notifications stop, covers Prometheus, Alertmanager, and the path out of your network with one signal.