Skip to main content

Apache Airflow

Open-source workflow scheduling platform for data pipelines, scheduled tasks and research workflow automation.

Main category
Data engineering
Cost
See the official site for cost
Licence type
Open source
Platforms
Linux / macOS / Windows
Visit Apache Airflow Quick start Code repository

Version-watched: add it to favourites and confirmed new releases will show on your favourites page (stored in this browser only).

Details

Apache Airflow is an open-source workflow scheduling and orchestration platform that describes task dependencies as DAGs to run data pipelines, scheduled jobs, batch processes and automated analysis workflows.

Use cases: data engineering, workflow scheduling, scheduled tasks, research workflow automation, ETL

Note: Airflow suits more complex workflows; for small scripted tasks, start with Snakemake, Prefect or the system task scheduler.