6

Been running a handful of scheduled tasks on two VPS instances for about a year now — nightly DB dumps, cache warmers, a couple of data sync scripts. Had everything piped to log files and thought I was covered.

Then last month one of the sync scripts started hanging mid-execution. It never crashed, never threw an error, just... sat there. The log showed the start timestamp but no finish line. Took me four days to notice because I wasn't checking logs daily (who does).

After that I looked into what people use to catch this kind of thing.

What I tried:

Healthchecks.io — solid, does the job. You curl a URL at the end of your cron and if the ping doesn't arrive within the expected window, you get an alert. Simple concept.

I also tested WatchCron which works on the same principle but felt a bit snappier to set up for multiple jobs. Has a dashboard that shows timing patterns across runs, which helped me spot that one of my backup jobs was gradually taking longer each week — would've missed that with just pass/fail alerts.

The pattern that works for me now:

#!/bin/bash
# at the end of each cron script
curl -fsS -m 10 --retry 3 $MONITOR_URL > /dev/null

If the script hangs or exits early, the ping never fires, and I get a Slack notification within minutes.

One thing I changed — I stopped redirecting cron output to /dev/null. Feels obvious in hindsight but I see it everywhere in tutorials. If your task does fail, you want that output in the mail spool or a log, not gone.

What's your setup for catching silent failures? Curious if anyone's doing something beyond the "ping on success" model — like tracking execution duration or exit codes.

you are viewing a single comment's thread
view the rest of the comments
[-] IanTwenty@piefed.social 4 points 3 hours ago

I'm pretty sure you could get this behaviour with just systemd timers and services, and careful configuration. You get logging for free also this way.

this post was submitted on 11 Aug 2026
6 points (100.0% liked)

Programming

28036 readers
261 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



founded 3 years ago
MODERATORS