Celery Beat is the scheduler at the heart of most Python and Django applications that need periodic tasks. It handles everything from sending daily digests to processing queued data to expiring old records. When it's working, it's invisible. When it stops — and it does stop — the silence is the problem.
Unlike a crashed web server, a stopped Celery Beat process produces no error page, no 500 response, no alert. The tasks simply don't run. This article covers why Beat fails silently, the most common failure modes, and three monitoring approaches you can implement today.
Celery Beat is a separate process from your Celery workers. Its job is to push tasks onto the queue at the right time. If Beat crashes, the workers keep running — they're just waiting for tasks that never arrive. From the worker's perspective, everything is fine. No error surfaces anywhere in your observability stack.
The default architecture gives you nothing to alert on. Beat writes to a schedule file (celerybeat-schedule) and logs task dispatch to stdout, but only if you're watching. There's no built-in dead-man detection, no "last ran at" metric exposed to Prometheus, no webhook on missed schedules.
The result: your codebase believes the scheduled tasks are running because the schedule is configured correctly. The scheduler disagrees — it's not running at all.
The most common cause. Beat runs as a separate process (celery -A myapp beat) and needs its own process supervisor entry. A deployment that restarts workers but not Beat, an OOM kill, or a missing Restart=always in the systemd unit leaves Beat silently dead. Workers keep running. Queue stays empty for periodic tasks.
Beat dispatches tasks by publishing to the message broker. If Redis or RabbitMQ goes down — even briefly — Beat may fail to enqueue the task and log a connection error that nobody is watching. Depending on your Celery version and broker settings, Beat may recover automatically or it may require a restart. Either way, you lose the runs during the outage window, and there's no automatic catch-up for missed schedules.
Beat can dispatch a task successfully while zero workers are available to consume it. The task sits in the queue, expires (if you've set task_soft_time_limit), and the next instance gets dispatched before the previous one finishes. This leads to silent data inconsistencies — your task "ran" from Beat's perspective, but the work never completed.
Celery Beat respects CELERY_TIMEZONE, but DST transitions can cause tasks to fire an hour early, an hour late, or twice in a row. If your schedule uses crontab(hour=2, minute=0) and your server switches from UTC-5 to UTC-4, the 2am window either doesn't exist or occurs at an unexpected wall-clock time. The logs will show the task ran, but at the wrong moment.
The most reliable pattern: add a dedicated heartbeat task to Beat's schedule and have it ping a monitoring URL every minute. If the monitoring service doesn't see a ping within the expected window, it fires an alert. This catches three failure modes at once — Beat not running, Beat running but failing to dispatch, and the heartbeat task itself failing.
# celery.py
from celery import Celery
from celery.schedules import crontab
app = Celery('myapp')
app.conf.beat_schedule = {
# Your existing tasks...
'nightly-report': {
'task': 'myapp.tasks.generate_report',
'schedule': crontab(hour=2, minute=0),
},
# Add this heartbeat task:
'heartbeat': {
'task': 'myapp.tasks.send_heartbeat',
'schedule': 60.0, # every 60 seconds
},
}
app.conf.timezone = 'UTC'# tasks.py
from celery import shared_task
@shared_task
def send_heartbeat():
import requests
try:
requests.get(
'https://your-monitor/heartbeat/TOKEN',
timeout=5
)
except Exception:
pass # Don't let heartbeat failures affect other tasksThis pattern works with both django-celery-beat (database-backed schedules) and the standard Beat scheduler. For django-celery-beat, add the heartbeat via the Django admin or a data migration rather than hardcoding it in beat_schedule.
Rather than a dedicated heartbeat task, have your most-critical existing task write a timestamp to Redis (or any fast store) at the end of each successful run. A separate lightweight checker — a cron job, Lambda, or health-check endpoint — reads that timestamp and alerts if it's stale.
# tasks.py
from celery import shared_task
from django.core.cache import cache
from django.utils import timezone
@shared_task
def generate_report():
# ... your task logic ...
# Write last-success timestamp
cache.set('generate_report:last_success', timezone.now().isoformat(), timeout=None)
# health_checks.py (called by a monitoring endpoint or separate cron)
from django.core.cache import cache
from django.utils import timezone
from datetime import timedelta
def celery_beat_is_healthy():
last = cache.get('generate_report:last_success')
if not last:
return False
last_run = timezone.datetime.fromisoformat(last)
expected_interval = timedelta(hours=24)
return (timezone.now() - last_run) < expected_interval * 1.5The advantage here is that it tests actual task completion, not just dispatch. Beat could be running and enqueueing tasks that immediately fail on the worker — the heartbeat approach wouldn't catch that, but a last-success timestamp would.
Flower is the standard Celery monitoring UI. It gives you a real-time view of workers, tasks, and queues. For Beat monitoring specifically, it's most useful for investigating failures rather than preventing them — Flower doesn't natively fire alerts when a scheduled task misses its window.
To get alerting from Flower, pair it with a tool that scrapes Flower's REST API and alerts on task failures or worker disconnections. Flower exposes task history at /api/tasks, which you can poll from a simple script:
import requests
from datetime import datetime, timedelta
# Check for recent failures of a specific task
resp = requests.get('http://localhost:5555/api/tasks?limit=50')
tasks = resp.json()
for task_id, task in tasks.items():
if task['name'] == 'myapp.tasks.generate_report':
if task['state'] == 'FAILURE':
# Send alert
print(f"Task failed: {task['traceback']}")Flower works well for Celery 4 and Celery 5 applications. In containerized environments, run it as a sidecar with celery -A myapp flower --port=5555.
Procfile, systemd unit, or container spec has Beat with Restart=always (or equivalent). It must restart independently of workers.CELERY_TIMEZONE = 'UTC' — eliminates an entire class of DST-related missed runs. Override to local time only if your schedule is explicitly wall-clock dependent.--loglevel=info and ship logs to a central store. Even without alerting, searchable logs let you reconstruct when Beat last dispatched a given task.CronWatch monitors Celery Beat tasks, cron jobs, GitHub Actions schedules, and AWS EventBridge — across every platform you use. Get alerted in Slack or email within minutes of a missed run or task failure. Works with django-celery-beat, Celery 4, and Celery 5.
Join the CronWatch waitlist → Parse a cron expression