Cron Job Hanging? Find the Cause, Add Timeouts, Catch Stuck Runs

A cron job hangs when one of its processes is blocked on something that never answers, such as a network call, a lock, a file server, or a full disk, and cron never stops it because cron has no maximum-runtime setting. Find the stuck process, see what it is waiting on, and give that call its own timeout. Then put an outer limit on the whole run, such as GNU timeout, systemd's TimeoutStartSec=, or Kubernetes activeDeadlineSeconds, make sure the limit reaches every process the job started, and report the result so a hang becomes an alert instead of a pile-up.

cronie and Debian's cron start the next run beside the stuck one; BusyBox crond won't start that line again until it exits (what is a cron job compares them). Commands assume Linux with procps-ng, util-linux, and GNU coreutils 9.12, the current release (checked October 24, 2026).

Find the Stuck Process and Everything It Started#

pgrep -af nightly-report
ps -eo pid,ppid,pgid,sid,stat,etime,wchan:32,args --forest

pgrep -f matches the whole command line and -a prints it (pgrep(1)). In the tree, etime is how long each process has run and wchan names the kernel function it sleeps in (ps(1)).

Each cron run gets a process group of its own. cronie 1.7.2 and Debian's cron 3.0pl1-212 call setsid() just before running the job's shell, which makes that shell the leader of a new session and process group (cronie, Debian); BusyBox crond 1.37.0 calls setpgrp() (crond.c). On cronie, the PID in a job's CROND[4242]: (root) CMD (...) log line is that shell, so ps -s 4242 --forest -o pid,pgid,stat,etime,wchan:32,args shows exactly one run.

Then read the STAT column:

• S, interruptible sleep: waiting for an event. The next section finds which.
• D, uninterruptible sleep, usually I/O: a disk or file server. The process won't act on SIGTERM until the call returns, so the fix is the device or mount, not a shorter timeout.
• T, stopped by a job-control signal: see the prompt case under the culprits.
• R for minutes: computing or looping, not waiting. A stack dump shows where.

If the script has exited but its CROND process remains with no CMDEND log line, a background child still holds the job's output; cronie logs CMDEND only once every writer has closed that pipe (do_command.c). Start such children with >/dev/null 2>&1.

See What It's Waiting On#

Run these as root. With Yama's ptrace_scope set to 1, an ordinary user can attach only to its own descendants (Yama).

cat /proc/4317/wchan; echo
cat /proc/4317/stack
strace -f -yy -p 4317
ls -l /proc/4317/fd
ss -tnp | grep 'pid=4317,'
lsof -p 4317

• /proc/PID/wchan names the kernel function the process sleeps in (proc_pid_wchan(5)). /proc/PID/stack shows the whole kernel call chain; it needs a kernel built with CONFIG_STACKTRACE and is restricted to root (proc_pid_stack(5), Linux 6.12 source). NFS or block-layer functions there put the wait in the kernel, not in your code.
• strace -p attaches and prints the system call the process is blocked in, left open until it returns; Ctrl-C detaches and leaves the process running. -f -p attaches every thread, not just the one whose ID is the PID, and -yy decodes descriptors, including socket addresses (strace(1)). read or poll on a socket is waiting for a peer, futex for another thread, and wait4 for a child, so trace the child.
• /proc/PID/fd shows sockets as socket:[inode] (proc_pid_fd(5)). ss -tnp matches TCP sockets to processes and shows the remote address (ss(8)), and lsof -p lists every open file and connection (lsof(8)).

Ask the Runtime for a Stack Trace#

The system call tells you what the process waits for; a language-level dump tells you which line of your code asked.

The Usual Culprits#

• A network call with no timeout. strace sits in read or poll and ss shows an established connection; several common clients wait forever by default (table below). If the server vanished without closing the connection, TCP keepalive notices only on sockets that enable it, and Linux's defaults wait 7,200 idle seconds and then send nine probes 75 seconds apart, over two hours in all (tcp(7)).
• A lock. In PostgreSQL, the job's session shows wait_event_type Lock in pg_stat_activity, and pg_blocking_pids(pid) names the sessions in its way (pg_stat_activity, pg_blocking_pids). A flock without -n or -w also waits indefinitely; preventing overlapping cron jobs covers lock choices.
• NFS. On a hard mount, the default, NFS requests are retried indefinitely (nfs(5)); the process sits in D.
• A full disk the writer waits on. MySQL 8.4 doesn't fail a MyISAM or binary-log write when the disk fills; it rechecks for space once a minute (MySQL), so the job's INSERT just waits.
• A prompt. Under cronie, stdin is a pipe that cron closes after writing any % input (do_command.c), so a prompt usually reads end-of-file and fails. When you test a wrapper script from a terminal, though, timeout runs the command outside the terminal's foreground process group, so a command that reads the terminal is stopped (T) until the deadline; --foreground exists for that case (timeout.c).

Put a Timeout on the Call That Hung#

An outer limit kills the whole run. A timeout on the call fails just that call, with an error your code can log, retry, or report. Defaults vary:

Because the requests read timeout counts the gap between bytes, a server that trickles data can still keep the call running. PostgreSQL 18's documentation advises against setting statement_timeout or lock_timeout in postgresql.conf, since that affects all sessions (client settings). Give the job its own role instead:

ALTER ROLE nightly_report SET statement_timeout = '20min';
ALTER ROLE nightly_report SET lock_timeout = '30s';
ALTER ROLE nightly_report SET idle_in_transaction_session_timeout = '5min';

The values apply from the role's next login (ALTER ROLE). idle_in_transaction_session_timeout ends a session that sits idle inside an open transaction, so a client that hangs mid-transaction stops holding its locks against everyone else.

Wrap the Whole Run in GNU timeout#

0 6 * * * timeout -k 2m 45m /opt/reports/build-report.sh

At 45 minutes, timeout sends SIGTERM; if the command is still running 2 minutes later, it sends SIGKILL. Durations take s, m, h, or d, and 0 disables a limit. -s SIGNAL changes the first signal, -v reports each signal on stderr, and --preserve-status (-p since coreutils 9.6) returns the command's own status even after a timeout (timeout.c, NEWS).

timeout Kills a Process Group, Not a Process Tree#

Unless --foreground is set, GNU timeout moves itself into a new process group before starting the command. On expiry it signals the command, then the whole group, then sends SIGCONT so stopped members act on the signal (timeout.c). Children and grandchildren that stay in the group die with the command. Three kinds survive:

• Children under --foreground. Only the command is signaled; the help text says "children of COMMAND will not be timed out."
• Processes that leave the group, such as a daemon or helper that calls setsid() and becomes the leader of a new session and group (setsid(2)).
• Processes in D state, which act on the signal only when their I/O returns.

To stop a run by hand, signal its group, not one PID; kill 4317 reaches one process while its children carry on. A negative PID after -- names a process group (kill(1)):

kill -TERM -- -4242   # every process in process group 4242
pkill -TERM -s 4242   # every process in session 4242

On cronie and Debian, where the job's shell leads its own session, pkill -s also reaches the group timeout created (pgrep(1)). Neither reaches a process that called setsid(). A control group does, because a child inherits its parent's cgroup across fork() and execve() (cgroups(7)) whatever its session, so a job that starts daemonizing helpers is safer as a systemd service.

BusyBox timeout Signals One PID#

On Alpine and other BusyBox images, timeout is BusyBox's (1.37.0 in Alpine 3.24). It execs the command in its own place and leaves a background watcher that signals that one PID, then sends SIGKILL after -k (timeout.c). There is no group kill, so children survive, and no 124: a command killed by SIGTERM reports 143 (128 + 15). apk add coreutils installs GNU timeout (coreutils 9.11 in Alpine 3.24).

Limits in systemd, Kubernetes, and GitHub Actions#

systemd#

A Type=oneshot service has no start timeout by default, and RuntimeMaxSec= doesn't apply to oneshot services, so the limit is TimeoutStartSec= (systemd.service(5)):

[Service]
Type=oneshot
ExecStart=/opt/reports/build-report.sh
TimeoutStartSec=45min
TimeoutStopSec=2min

When the limit passes, systemd 262 sends KillSignal= (SIGTERM) to every process in the unit's control group under the default KillMode=control-group, then SIGKILL to whatever remains after TimeoutStopSec=, 90 seconds by default upstream (systemd.kill(5), systemd-system.conf(5)). That includes children that called setsid(). systemd timers covers the timer side.

Kubernetes#

A CronJob's job template has two different deadlines, and the Job docs warn to set the field "at the proper level":

Use the Job-level field as the run's limit; the pod-level one limits each attempt.

Termination sends SIGTERM (many runtimes use the image's STOPSIGNAL instead) to process 1 in each container, then SIGKILL after terminationGracePeriodSeconds, 30 by default (pod termination). When a shell script is PID 1, its children never get the TERM, and the script ignores it unless it traps it, because a namespace's init process receives only signals it has a handler for (pid_namespaces(7)). The KILL then ends every process in the container. Kubernetes CronJob monitoring covers the rest of the spec.

GitHub Actions#

A job's timeout-minutes defaults to 360, and a step can have its own (workflow syntax). On cancellation, the runner sends SIGINT to the step's shell, SIGTERM 7.5 seconds later, and kills the process tree 2.5 seconds after that (cancellation; checked October 24, 2026), so cleanup must handle INT and fit in seconds.

Let the Job Clean Up Before the KILL#

The first signal is the job's chance to delete partial output, release what it holds, and exit. A shell trap does it, with one catch: a trap set for a signal that arrives while a foreground command runs waits until that command finishes (POSIX.1-2024 §2.12). GNU timeout and systemd signal every process, so the foreground child gets the signal too, exits, and the trap runs. BusyBox timeout, Kubernetes, and a plain kill PID signal only the shell, so the child keeps working and the trap waits. Run the work in the background and wait for it: a trapped signal makes wait return at once.

#!/bin/sh
# /opt/reports/build-report.sh
workdir=$(mktemp -d /srv/reports/.build.XXXXXX) || exit 1
child=

stop() {
  echo "build-report: stopping on signal" >&2
  if [ -n "$child" ]; then
    kill -TERM "$child" 2>/dev/null
    wait "$child"
  fi
  rm -rf "$workdir"
  exit 143
}
trap stop INT TERM

/opt/reports/bin/render --out "$workdir" &
child=$!
wait "$child"
rc=$?
if [ "$rc" -eq 0 ]; then
  mv "$workdir/report.pdf" /srv/reports/daily.pdf || rc=1
fi
rm -rf "$workdir"
exit "$rc"

The trap forwards TERM even when INT arrives, because the same POSIX section has a non-interactive shell start background commands with SIGINT ignored. Building in a directory on the same file system makes the final mv a rename, so readers never see half a report.

For the grace before KILL (-k, TimeoutStopSec=, terminationGracePeriodSeconds), take the cleanup's worst case and add margin. Keep cleanup local and bounded: a cleanup step that calls the service that just hung needs its own timeout.

Get Alerted When a Run Hangs#

A timeout ends the hang; an outside monitor tells someone. For CronRadar there is no SDK for crontab and no auto-discovery, so each cron job reports with its own HTTP check-in. This wrapper reports the start, runs the job under timeout, and turns its exit status into the failure message:

#!/bin/sh
# /opt/reports/build-report-monitored.sh
url="https://cron.life/api/v1/ping/daily-report"
run_id="daily-report-$(date +%s)-$$"

report() {
  event=$1
  shift
  curl --fail --silent --show-error --max-time 5 --output /dev/null \
    --request POST --get --user "$CRONRADAR_API_KEY:" \
    --data-urlencode "schedule=0 6 * * *" \
    --data-urlencode "gracePeriod=600" \
    --data-urlencode "run_id=$run_id" \
    "$@" "$url/$event" || true
}

report start
timeout -k 2m 45m /opt/reports/build-report.sh
rc=$?
case "$rc" in
  0) report complete ;;
  124) report fail --data-urlencode "message=build-report.sh hit its 45-minute timeout and stopped on SIGTERM" ;;
  137) report fail --data-urlencode "message=build-report.sh was killed by SIGKILL (timeout -k or the OOM killer)" ;;
  *) report fail --data-urlencode "message=build-report.sh exited with code $rc" ;;
esac
exit "$rc"

Run it from the crontab as 0 6 * * * /opt/reports/build-report-monitored.sh, with CRONRADAR_API_KEY in its environment. Each report gives up after five seconds and ignores errors, and the script exits with the job's own status. A fail check-in alerts immediately, without waiting for a grace period. The first run creates the monitor from schedule, which CronRadar reads in UTC; changing schedule or gracePeriod in the script later doesn't update an existing monitor, so edit those in the monitor's settings.

Stuck detection is the backstop for a run that never reports at all, such as one whose host goes down mid-run. It needs the start check-in. Every check-in, including start, moves the expected time to the job's next scheduled occurrence. If that time plus the grace period passes while the last event is a start with no complete or fail, the monitor goes critical as stuck. The window belongs to the job, not the run: the next run's start resets it, so a hang is caught only when the job then goes silent, as it does under a skip lock, a systemd timer waiting on its service, or BusyBox crond. On cronie or Debian's cron, the next run starts beside the hung one and reports normally, so the hang is never flagged. There is no per-job maximum-runtime setting either, which is why the limit belongs in the job, where timeout turns a hang into a fail at 45 minutes. Choosing a grace period covers sizing the window, and Hangfire jobs stuck in Processing applies the same rules to Hangfire.

Frequently Asked Questions#

How do I find out why my cron job is hanging?#

Find the run with pgrep -af or ps --forest and check its state. Then see what it waits on: strace -p PID gives the system call, ss -tnp or lsof -p the connection or file behind it, and py-spy, jcmd, or dotnet-stack the line of code. Common causes are network calls without timeouts, lock waits, and NFS.

How do I set a timeout for a cron job?#

Prefix the crontab command with GNU timeout: timeout -k 2m 45m /opt/reports/build-report.sh sends SIGTERM at 45 minutes and SIGKILL two minutes later. Under systemd, set TimeoutStartSec= on the oneshot service; in Kubernetes, set activeDeadlineSeconds on the Job. Also give each network and database call its own timeout.

Why doesn't timeout kill my script's child processes?#

GNU timeout signals its own process group, so a child escapes when --foreground is set or when the child starts a new session with setsid(), as daemons do. BusyBox timeout signals only the command's PID. Stop a run by hand with kill -TERM -- -PGID, or run the job as a systemd service, which kills its whole control group.

What exit code does timeout return?#

GNU timeout exits 124 when the first signal stopped the command and 137 when SIGKILL ended it; 125 to 127 mean timeout failed or couldn't run the command, and anything else is the command's own status. BusyBox timeout has no timeout status, so install coreutils on Alpine before a wrapper tests for 124.

How do I get alerted when a cron job runs too long?#

Run the job under timeout so an overrun becomes a failure, and report each run's start and result to an external monitor. In CronRadar, a fail check-in alerts at once. With no per-job maximum runtime, a start with no result is flagged as stuck only once the job's next scheduled time plus its grace period passes with nothing else reporting.