Scheduler Latency
A measure of HTTP-based job scheduler latency (how "on time" various job schedulers fire).
A measure of how "on time" various HTTP job schedulers are. An HTTP job scheduler calls some HTTP-based API on the schedule of your choosing. For this benchmark, we configured various schedulers to fire requests every 60 minutes at this website, smplmark.org, and then captured when the request was received to calculate how "on time" the scheduler was.
Results
Skew (ms) per subject over 1,220 measurements, ranked by average.
| Subject | Avg | Median | Min | Max | P95 | P99 | Count |
|---|---|---|---|---|---|---|---|
| GitHub Actions | 1,894,313 | 2,052,542 | 47,881 | 3,586,288 | 3,505,191 | 3,564,835 | 58 |
| Cloudflare Workers | 37,909 | 33,187 | 33,111 | 52,867 | 51,400 | 51,533 | 131 |
| AWS EventBridge | 26,468 | 26,474 | 26,273 | 26,649 | 26,605 | 26,646 | 131 |
| cron-job | 7,514 | 7,507 | 6,328 | 9,529 | 8,436 | 9,146 | 131 |
| Google Cloud Scheduler | 5,331 | 5,067 | 4,850 | 13,538 | 6,584 | 8,555 | 131 |
| EasyCron | 3,900 | 3,842 | 3,270 | 5,378 | 4,487 | 4,827 | 114 |
| QStash | 1,915 | 1,520 | 297 | 6,386 | 3,259 | 5,165 | 131 |
| Smpl Jobs | 1,507 | 1,408 | 575 | 2,579 | 2,360 | 2,555 | 131 |
| Runhooks | 859 | 754 | 466 | 11,647 | 1,052 | 2,473 | 131 |
| Posthook | 682 | 684 | 207 | 2,545 | 1,218 | 2,077 | 131 |
Metrics
- Skew (
skew) (ms) — computed on read. The number of milliseconds past the "top of the hour". Computed with JsonLogic:{"%":[{"var":"created_at"},3600000]}
Subjects (10)
- Smpl Jobs
- GitHub Actions
- Posthook
- cron-job
- EasyCron
- Google Cloud Scheduler
- AWS EventBridge
- Runhooks
- QStash
- Cloudflare Workers
Methodology
Each scheduler is configured to POST a single HTTP request to
https://app.smplmark.org/api/v1/measurements every 60 minutes, at the top of the hour. smplmark.org records the date/time each request arrives. The derived metric, skew, is the number of milliseconds past the top of the hour (the arrival timestamp mod 3,600,000 ms). The lower the skew, the more "on time" the scheduler. The measurement API records every request that arrives; nothing is deduplicated, filtered, or hand-edited.
The reference clock. Arrival time is the timestamp smplmark.org assigns the moment a request reaches the measurement API. smplmark runs on Cloudflare, so we inherit Cloudflare's clocks, which are kept in sync via NTP; we don't independently verify them. If smplmark's clock drifts, every subject's skew will shift by the same amount — the ordering of the leaderboard will survive, but the absolute numbers assume the receiver's clock is right. Each scheduler's own idea of when the hour starts is not corrected for: firing when your clock says it's time is part of what's being measured.
The wrap. Skew is computed modulo the hour, so a scheduler that fires 60.1 minutes late will be interpreted as only 6 seconds late while a request arriving early will be interpreted as nearly an hour late. In practice, none of the schedulers fire early and the only scheduler that's more than a minute late is GitHub Actions, which misses by wild margins, not by seconds.
Geography. The receiver operates in US-East, so schedulers firing from US-West or Europe pay a network tax — a few milliseconds — that US-East schedulers do not.
Configuration. Every subject runs on its free tier — or close to it; a couple of them bill about a dollar a month — configured for one hourly job with the vendor's default settings.
Published by smplkit.com.