Migrating production infrastructure? Get up to ₹10,000 in migration credits. Apply now
voroaBlog
Background Workers

APScheduler Guide: Python Cron Jobs Tested for Missed Runs

· · 6 min read

Python scheduled jobs tested for duplicate runs, overlapping jobs and missed runs after a restart

APScheduler is one of the most popular ways to run scheduled jobs inside a Python app: a report every morning, a cleanup every hour, a sync every five minutes. It works with Flask, Django and FastAPI, and the basics take a few lines.

This guide covers those basics, then what we saw when we tested restarts, overlap and two processes. One common setting quietly throws away every missed run.

On this page

Advertisement

A cron job in five lines

bash
pip install "apscheduler<4"
python
from apscheduler.schedulers.blocking import BlockingScheduler

sched = BlockingScheduler(timezone="Asia/Kolkata")

@sched.scheduled_job("cron", hour=9, minute=0)
def daily_digest():
    send_digest()          # your code

sched.start()

That runs daily_digest at 9:00 am India time every day. Version 3.11 is the current stable release. Version 4 is still in alpha with a different API, so pin <4 for now.

Schedulers, triggers and job stores

The library has three parts you choose between:

Part Common choices Use
Scheduler BlockingScheduler, BackgroundScheduler, AsyncIOScheduler Blocking for a script that only schedules; Background inside a web app; AsyncIO with FastAPI
Trigger cron, interval, date Calendar times, fixed gaps, or one run at a set time
Job store Memory (default), SQLAlchemy, Redis Whether jobs survive a restart

A cron trigger takes the same fields as a cron line, by name: minute="*/5" for every five minutes, day_of_week="mon-fri", hour=9 for weekday mornings. Or build one from a crontab string with CronTrigger.from_crontab("0 9 * * 1-5").

Test 1: two processes, every job twice

Setup for all tests: version 3.11.3 on Python 3.14, a cron trigger firing every second so results appear quickly, and Redis 8.10.3 to count runs. Measured 7 November 2026 on a laptop.

A web app usually runs as several worker processes. Gunicorn with four workers loads your app four times (without --preload). If the scheduler starts at import time, there are now four schedulers.

Setup (20 seconds) Runs Seconds with more than one run
One process 20 0
Two processes 40 20 of 20

Diagram: one process runs each scheduled time once; two processes run it twice; Gunicorn with four workers would start four schedulers

Every job ran twice. With four Gunicorn workers it would run four times. Two fixes work:

  • Run the scheduler in one dedicated process, not inside every web worker.
  • Or take a lock before the work, for example a Redis key set with NX, so only one process wins each run.

Test 2: overlap is blocked by default

A job that takes 2.5 seconds, scheduled every second. This is where it differs from many libraries.

Setup (15 seconds) Runs Most running at once
Default (max_instances=1) 5 1
max_instances=3 15 3

By default, the scheduler will not start a job while the previous run is still going. It skips that run and logs a warning that the maximum number of running instances was reached.

That is usually the safe choice. Raise max_instances only if the job is safe to run in parallel.

Test 3: what a restart does to missed runs

We stopped the scheduler for about 5 seconds, as a deploy would, and started it again. Then we tried the settings people usually recommend.

Job store and settings Runs (20 s) Missed runs
Memory store (default) 14 Lost
SQLite store, add_job(..., replace_existing=True) 14 Lost
SQLite store, job kept, coalesce=False 20 Replayed: 7 runs at once on restart
SQLite store, job kept, coalesce=True 14 No burst; consistent with one merged catch-up run

The memory store forgets everything on restart, which most people expect.

The second row is the trap. A persistent store is meant to remember missed runs. But the common pattern add_job(..., id="x", replace_existing=True) at start-up overwrites the stored job with a fresh one, so its missed runs are thrown away. It looked identical to the memory store.

Missed runs only came back when the code left the stored job alone:

python
import time
from apscheduler.schedulers.background import BackgroundScheduler
from apscheduler.jobstores.sqlalchemy import SQLAlchemyJobStore

sched = BackgroundScheduler(
    jobstores={"default": SQLAlchemyJobStore(url="sqlite:///jobs.sqlite")},
    timezone="Asia/Kolkata",
)
sched.start(paused=True)                 # load stored jobs without running them yet
if sched.get_job("digest") is None:      # first start only
    sched.add_job(daily_digest, "cron", hour=9, id="digest",
                  misfire_grace_time=3600, coalesce=True)
sched.resume()                           # a stored job keeps its missed runs
while True:
    time.sleep(60)                       # keep the process alive

We ran this pattern through three restarts in a row: no error, and the stored job carried on each time. The paused start needs BackgroundScheduler; with BlockingScheduler, start() never returns.

Then choose what a catch-up means. With coalesce=False, every missed run fires at once on restart; we saw a burst of 7.

With coalesce=True, the documentation says they collapse into one run, which is right for most reports and syncs. On a job firing every second our test could not tell that merged run apart from a normal one, but there was no burst. misfire_grace_time sets how late a run may be and still count.

Timing: runs fired a median of 8 to 15 ms after the scheduled second with the memory store, and 33 to 50 ms with the SQLite store.

APScheduler, schedule or system cron?

This library schedule library System cron or a platform cron job
Cron-style expressions Yes No, readable intervals only Yes
Survives restarts With a persistent store No Yes, runs from outside your app
Duplicate runs with many web workers Yes, unless you isolate it Yes No
Overlap protection Built in Manual Manual
Best for Schedules inside a long-running app Small scripts Jobs that must run on time regardless of the app

For queued work rather than timed work, a job queue fits better. Our Celery and Redis guide covers the Python option, including Celery beat for periodic tasks.

Run Python scheduled jobs on Voroa

Two ways, depending on your code. If your job is a script that runs and ends, create a Cron Job from your repository with a command such as python digest.py and a schedule. Schedules run in India Standard Time.

If the scheduler lives inside a long-running process, deploy that process as a background worker, a single instance kept running, so the scheduler is never duplicated by web traffic.

Either way, keep jobs safe to run twice. Setup is in scheduling a task on Voroa and worker or cron job. The Node.js version of these tests found the same duplicate-run problem.

FAQ

What is it used for? Running Python functions on a schedule inside your app: at set times (cron), at fixed intervals, or once at a given moment.

Why does my scheduled job run twice? The scheduler started in more than one process, often one per Gunicorn or Uvicorn worker. Run it in a single process or use a lock.

Does it run missed jobs after a restart? Only with a persistent job store, and only if you do not replace the stored job at start-up. In our test, replace_existing=True threw the missed runs away.

What does coalesce do? With coalesce=True, several missed runs of the same job collapse into one when the scheduler catches up. With False, each missed run fires separately.

How do I stop overlapping runs? You usually do not need to: max_instances defaults to 1, so it skips a run while the last one is still going.

Should I use version 4? Not yet for production. It is still an alpha with a different API; 3.11 is the stable line.

Tested 7 November 2026 with version 3.11.3 on Python 3.14. Found something out of date? Write to [email protected].

Advertisement