Been running a handful of scheduled tasks on two VPS instances for about a year now — nightly DB dumps, cache warmers, a couple of data sync scripts. Had everything piped to log files and thought I was covered.
Then last month one of the sync scripts started hanging mid-execution. It never crashed, never threw an error, just... sat there. The log showed the start timestamp but no finish line. Took me four days to notice because I wasn't checking logs daily (who does).
After that I looked into what people use to catch this kind of thing.
What I tried:
Healthchecks.io — solid, does the job. You curl a URL at the end of your cron and if the ping doesn't arrive within the expected window, you get an alert. Simple concept.
I also tested WatchCron which works on the same principle but felt a bit snappier to set up for multiple jobs. Has a dashboard that shows timing patterns across runs, which helped me spot that one of my backup jobs was gradually taking longer each week — would've missed that with just pass/fail alerts.
The pattern that works for me now:
#!/bin/bash
# at the end of each cron script
curl -fsS -m 10 --retry 3 $MONITOR_URL > /dev/null
If the script hangs or exits early, the ping never fires, and I get a Slack notification within minutes.
One thing I changed — I stopped redirecting cron output to /dev/null. Feels obvious in hindsight but I see it everywhere in tutorials. If your task does fail, you want that output in the mail spool or a log, not gone.
What's your setup for catching silent failures? Curious if anyone's doing something beyond the "ping on success" model — like tracking execution duration or exit codes.
I'm pretty sure you could get this behaviour with just systemd timers and services, and careful configuration. You get logging for free also this way.