Skip to content
hynekPublic

About

Fault-tolerant background threads for Python

Resources

Code of conduct

Contributing

Security policy

Stars

30 stars

Watchers

0 watching

Forks

Repository files navigation

bgt

Fault-tolerant background threads for Python

Documentation at ReadTheDocs License: MIT PyPI version No AI slop inside.

POV: you want a framework-agnostic way to reliably run a plain or async function or method in the background, continuously or with pauses between work units.

bgt comes to the rescue with:

  • A service that runs your code in a loop and optionally waits between work units. It wakes up on a fixed time interval, or as soon as something wakes it.

  • A supervisor that runs that loop in a background thread. If your code crashes, the supervisor restarts the loop after an exponential backoff. Write crash-only code, bgt takes care of the rest.

  • Thorough instrumentation via structlog and Prometheus.

  • Framework and platform independence.

bgt is not a job queue like Celery or RQ. Common use cases include:

  • Periodic cleanup duties for expired caches or sessions.
  • Flushing buffered metrics or events.
  • Processing a continuous stream of data in bounded batches.

Here's a service that runs a work unit every two seconds and survives its own crashes:

import bgt


def do_work() -> bool:
    ...  # one bounded work unit; if it crashes, it doesn't crash the loop

    return False  # False = wait for next wakeup; True = run again immediately


with bgt.SupervisedService.start(
    bgt.as_work_factory(do_work),
    name="example",
    wakeup=bgt.IntervalOnlyWakeup(),
    interval=2,
):
    ...  # do_work runs in the background until we leave this block

Return True from your work unit to be run again immediately. This keeps work units short, which makes shutdowns prompt. For async work units such as async def do_work() use bgt.as_async_work_factory().

Check out our step-by-step tutorial to get an instant feel for the features!

Installation

The package is available on PyPI under the bgt name:

$ uv pip install bgt

Documentation

Full documentation lives at https://bgt.hynek.me/.

Ecosystem

There are currently two packages built on top of bgt:

  • pgbg adds PostgreSQL LISTEN / NOTIFY-driven wakeups and leader election with automatic failover on top. If your services should wake up on database events, or only one process at a time should do the work, check it out!

  • django-bgt adapts bgt for Django. It provides a service decorator for your work functions, using the Django ORM to orchestrate leader election and automatic failover, while remaining database agnostic.

Credits

bgt is written by Hynek Schlawack and distributed under the terms of the MIT license.

The development is kindly supported by my employer Variomedia AG and all my fabulous GitHub Sponsors.

About

Fault-tolerant background threads for Python

Resources

Code of conduct

Contributing

Security policy

Stars

30 stars

Watchers

0 watching

Forks

Releases

Sponsor this project

Contributors

Languages