Apache Airflow
Open-source workflow scheduling platform for data pipelines, scheduled tasks and research workflow automation.
- Main category
- Data engineering
- Cost
- See the official site for cost
- Licence type
- Open source
- Platforms
- Linux / macOS / Windows
Visit Apache Airflow Quick start Code repository
Version-watched: add it to favourites and confirmed new releases will show on your favourites page (stored in this browser only).
Details
Apache Airflow is an open-source workflow scheduling and orchestration platform that describes task dependencies as DAGs to run data pipelines, scheduled jobs, batch processes and automated analysis workflows.
Use cases: data engineering, workflow scheduling, scheduled tasks, research workflow automation, ETL
Note: Airflow suits more complex workflows; for small scripted tasks, start with Snakemake, Prefect or the system task scheduler.