← Blog  ·  October 2026

Cron Job Monitoring Best Practices: How to Know When Your Scheduled Tasks Fail

The nightly backup job last ran 11 days ago. Nobody noticed. The backup you needed this morning doesn't exist.

Cron jobs are among the most relied-upon and least-observed parts of production infrastructure. A web request that fails generates a 500 error, an exception in logs, and often a user complaint. A cron job that fails generates nothing. It simply stops producing results, and the gap isn't noticed until someone tries to use those results.

This guide covers seven concrete practices for making cron jobs observable in production — with examples for standard crontab, GitHub Actions, and other schedulers.

Practice 1: Always redirect output to a log file

By default, crontab on Linux delivers stdout and stderr to the local mail spool. Nobody reads the mail spool on production servers. The fastest thing you can do right now is redirect output to a real log file:

# Before: output goes nowhere visible
0 3 * * * /opt/jobs/sync.py

# After: stdout and stderr go to a log file with timestamps
0 3 * * * /opt/jobs/sync.py >> /var/log/nightly-sync.log 2>&1

Rotate logs to prevent unbounded growth:

# /etc/logrotate.d/nightly-sync
/var/log/nightly-sync.log {
  daily
  rotate 14
  compress
  missingok
  notifempty
}

This doesn't alert you — but it gives you something to inspect when things go wrong, and the timestamps let you immediately see whether a run happened and what it produced.

Practice 2: Check exit codes explicitly

Cron runs whatever command you give it and ignores the exit code unless you build exit code awareness into your wrapper. Make it explicit:

#!/bin/bash
# run-sync.sh — wrapper that checks exit code

/opt/jobs/sync.py
EXIT_CODE=$?

if [ $EXIT_CODE -ne 0 ]; then
  echo "sync.py failed with exit code $EXIT_CODE at $(date)" \
    | mail -s "CRON FAILURE: nightly-sync" ops@example.com
  exit $EXIT_CODE
fi

echo "sync.py completed successfully at $(date)"

Then in crontab:

0 3 * * * /opt/jobs/run-sync.sh >> /var/log/nightly-sync.log 2>&1

This catches failures in the job itself. It does not catch the job never running — for that, you need a heartbeat.

Practice 3: Add a heartbeat ping (dead man's switch)

A heartbeat monitoring pattern inverts the detection model: instead of alerting on failure, the job signals success — and the monitoring service alerts when the signal stops arriving. This single technique catches three failure modes at once: job fails, job never starts, and job is killed before completing.

# Crontab: ping on success only (the && ensures the ping only fires on exit 0)
0 3 * * * /opt/jobs/sync.py && curl -s "https://cron.watch/ping/your-job-id" >/dev/null 2>&1

For jobs with complex logic where you want to ping from inside the script:

#!/bin/bash
set -e  # exit on any error

# ... job logic here ...

# Only reached if no errors occurred above
curl -s "https://cron.watch/ping/your-job-id" >/dev/null 2>&1
echo "Job completed and heartbeat sent at $(date)"

The monitoring service checks for the ping within the expected window. If your job runs at 3am daily, configure a 26-hour grace window — the service alerts if no ping arrives within 26 hours of the last one.

Practice 4: Configure grace periods to reduce false positives

Not all cron delays are failures. A job that normally takes 2 minutes occasionally takes 15 minutes when the server is under load, or when the dataset is larger than usual. Without a grace period, you'll get alerts for normal variation.

A grace period is the amount of extra time beyond the expected run window before the monitoring service fires an alert. For a job that runs every hour and usually takes 5 minutes, a reasonable grace period is 10–15 minutes. For a nightly job, 2–3 hours is typically enough buffer to avoid false positives from server maintenance windows.

Grace period tuning Start with a larger grace period and tighten it over time as you observe actual job duration variance. A grace period that's too short creates alert fatigue. A grace period that's too long delays legitimate failure alerts. Most teams land at 20–30% of the expected job duration as a reasonable default.

Practice 5: Build retry logic for transient failures

Many cron job failures are transient: a database is briefly unavailable, an external API returns a 503, a network timeout. A job that retries once or twice before failing permanently generates far fewer spurious alerts:

#!/bin/bash
# run-with-retry.sh — retry up to 3 times with 60-second backoff

MAX_ATTEMPTS=3
ATTEMPT=0
SLEEP_BETWEEN=60

while [ $ATTEMPT -lt $MAX_ATTEMPTS ]; do
  ATTEMPT=$((ATTEMPT + 1))
  echo "Attempt $ATTEMPT of $MAX_ATTEMPTS at $(date)"

  if /opt/jobs/sync.py; then
    echo "Success on attempt $ATTEMPT"
    curl -s "https://cron.watch/ping/your-job-id" >/dev/null 2>&1
    exit 0
  fi

  echo "Attempt $ATTEMPT failed"
  if [ $ATTEMPT -lt $MAX_ATTEMPTS ]; then
    echo "Waiting ${SLEEP_BETWEEN}s before retry..."
    sleep $SLEEP_BETWEEN
  fi
done

echo "All $MAX_ATTEMPTS attempts failed — alerting"
echo "Sync job failed after $MAX_ATTEMPTS attempts" \
  | mail -s "CRON FAILURE: nightly-sync (all retries exhausted)" ops@example.com
exit 1

Don't add retries blindly. For jobs with side effects (writing to a database, sending emails, charging cards), retrying on failure can create duplicate actions. Make sure the underlying operation is idempotent before adding automatic retries.

Practice 6: Alert on the right channels with the right urgency

Email is the default alert destination for cron failures, but it's often the wrong one. Email inboxes are noisy; a failure alert at 3am may not be seen until 9am. Match alert channel to job criticality:

For GitHub Actions, configure branch protection to require the scheduled workflow to pass before merges, and set up Slack integration via the GitHub Actions Slack app for workflow failures.

Practice 7: Maintain a monitoring dashboard

Individual alerts tell you when something is wrong. A dashboard tells you the overall health of your scheduled job infrastructure at a glance. At minimum, the dashboard should show: job name, schedule, last run time, last run status, and whether the next run is expected within the grace window.

This matters most when investigating an incident. When a data discrepancy is discovered on Monday morning, the first question is: "did all the weekend jobs run successfully?" Without a dashboard, answering that requires digging through log files on multiple servers. With a dashboard, it takes ten seconds.

For self-hosted cron jobs, a simple approach is a status table generated from log file timestamps and stored in a web-accessible location. For managed schedulers (Heroku, Render, GitHub Actions), the platform UI serves this function — but only shows its own jobs, not cross-platform visibility.

Putting it together

The minimum viable monitoring stack for a production cron job is practices 1, 3, and 6: structured logging, a heartbeat ping, and alerts that reach someone who can act. The other practices reduce alert fatigue and improve investigation speed, but the three-part core is what prevents silent failures.

The one failure mode that catches teams most often is not a noisy crash — it's the job that exits 0, produces no errors, and simply stops generating the output it was supposed to produce. For that failure mode, a heartbeat monitor is the only detection mechanism that works reliably.

📟 Cron job monitoring without the setup overhead

CronWatch implements heartbeat monitoring, grace periods, escalating alerts, and a cross-platform dashboard out of the box. Add one line to any cron job to get started — no agent to install, no server to maintain.

Join the CronWatch waitlist → Parse a cron expression

← All articles  ·  CronWatch  ·  Cron parser tool