Microsoft open-sources pg_durable for fault-tolerant SQL workflows inside Postgres
No more stitching cron jobs and status tables — pg_durable checkpoints every step so you never lose progress.
Deep Dive
pg_durable is an open-source PostgreSQL extension for durable execution, letting you define long-running SQL workflows that survive crashes, restarts, or failed steps by checkpointing each operation. No extra services like Airflow or Temporal are needed. It's ideal for vector embedding pipelines, scheduled maintenance, and fan-out aggregation — available now on Azure HorizonDB or any Postgres instance with the extension installed.
Key Points
- pg_durable runs as a PostgreSQL extension with no external dependencies like Redis, Temporal, or Airflow.
- Workflows are defined in SQL using operators such as `~>` (sequential) and `|=>` (parallel), with automatic checkpointing after each step.
- Ideal for vector embedding pipelines, fan-out aggregation, and scheduled runbooks — resumes from the last checkpoint after crashes or restarts.
Why It Matters
Durable execution inside Postgres eliminates external orchestrators and manual retry logic, simplifying data pipelines for any team.