airflow-workflows
Apache Airflow DAG design, operators, and scheduling best practices.
Best use case
airflow-workflows is best used when you need a repeatable AI agent workflow instead of a one-off prompt.
Apache Airflow DAG design, operators, and scheduling best practices.
Teams using airflow-workflows should expect a more consistent output, faster repeated execution, less prompt rewriting.
When to use this skill
- You want a reusable workflow that can be run more than once with consistent structure.
When not to use this skill
- You only need a quick one-off answer and do not need a reusable workflow.
- You cannot install or maintain the underlying files, dependencies, or repository context.
Installation
Claude Code / Cursor / Codex
Manual Installation
- Download SKILL.md from GitHub
- Place it in
.claude/skills/airflow-workflows/SKILL.mdinside your project - Restart your AI agent — it will auto-discover the skill
How airflow-workflows Compares
| Feature / Agent | airflow-workflows | Standard Approach |
|---|---|---|
| Platform Support | Not specified | Limited / Varies |
| Context Awareness | High | Baseline |
| Installation Complexity | Unknown | N/A |
Frequently Asked Questions
What does this skill do?
Apache Airflow DAG design, operators, and scheduling best practices.
Where can I find the source code?
You can find the source code on GitHub using the link provided at the top of the page.
SKILL.md Source
# Airflow Workflows
## DAG Structure
```python
from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.providers.common.sql.operators.sql import SQLExecuteQueryOperator
from datetime import datetime, timedelta
default_args = {
'owner': 'data-team',
'depends_on_past': False,
'email_on_failure': True,
'retries': 3,
'retry_delay': timedelta(minutes=5),
}
with DAG(
'daily_etl',
default_args=default_args,
description='Daily ETL pipeline',
schedule_interval='0 6 * * *', # 6 AM daily
start_date=datetime(2024, 1, 1),
catchup=False,
tags=['etl', 'daily'],
) as dag:
extract = PythonOperator(
task_id='extract_data',
python_callable=extract_function,
)
transform = SQLExecuteQueryOperator(
task_id='transform_data',
conn_id='warehouse',
sql='sql/transform.sql',
)
load = PythonOperator(
task_id='load_data',
python_callable=load_function,
)
extract >> transform >> load
```
## Common Operators
| Operator | Use Case |
|----------|----------|
| `PythonOperator` | Custom Python code |
| `BashOperator` | Shell commands |
| `SQLExecuteQueryOperator` | Database queries |
| `S3ToSnowflakeOperator` | Cloud data transfers |
| `DbtCloudRunJobOperator` | dbt Cloud jobs |
## Best Practices
1. **Idempotent tasks** - Safe to re-run
2. **Small tasks** - Easy to debug, retry
3. **XCom sparingly** - Only small data
4. **Templating** - Use `{{ ds }}` for dates
5. **Sensors wisely** - Avoid blocking workers
## Task Dependencies
```python
# Linear
task1 >> task2 >> task3
# Parallel
[task1, task2] >> task3
# Complex
task1 >> [task2, task3]
[task2, task3] >> task4
```
## Dynamic DAGs
```python
for table in ['users', 'orders', 'products']:
task = PythonOperator(
task_id=f'process_{table}',
python_callable=process_table,
op_kwargs={'table': table},
)
```Related Skills
apache-airflow-orchestration
Complete guide for Apache Airflow orchestration including DAGs, operators, sensors, XComs, task dependencies, dynamic workflows, and production deployment
airflow-expert
Expert-level Apache Airflow orchestration, DAGs, operators, sensors, XComs, task dependencies, and scheduling
airflow-etl
Generate Apache Airflow ETL pipelines for government websites and document sources. Explores websites to find downloadable documents, verifies commercial use licenses, and creates complete Airflow DAG assets with daily scheduling. Use when user wants to create ETL pipelines, scrape government documents, or automate document collection workflows.
airflow-dag
Create Apache Airflow DAGs for construction data pipelines. Orchestrate ETL, validation, and reporting workflows.
airflow-dag-patterns
Build production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use when creating data pipelines, orchestrating workflows, or scheduling batch jobs.
airflow-3x-migration
Comprehensive guide and patterns for migrating Apache Airflow 2.x workflows to Airflow 3.x, covering import changes, deprecated features, and new paradigms like Asset scheduling and TaskFlow API.
ahu-airflow
Fan Selection & Airflow Analysis Agent
adaptive-workflows
Self-learning workflow system that tracks what works best for your use cases. Records experiment results, suggests optimizations, creates custom templates, and builds a personal knowledge base. Use to learn from experience and optimize your LLM workflows over time.
workflows-expert
Activate when requests involve workflow execution, CI/CD pipelines, git automation, or multi-step task orchestration. This skill provides workflows-mcp MCP server integration with tag-based workflow discovery, DAG-based execution, and variable syntax expertise. Trigger on phrases like "run workflow", "execute workflow", "orchestrate tasks", "automate CI/CD", or "workflow information".
interactor-workflows
Build state-machine based automation with human-in-the-loop support through Interactor. Use when implementing approval flows, multi-step processes, automated pipelines, or any workflow requiring user input at specific stages.
integration-workflows
Cross-MCP workflows that coordinate multiple systems (Linear, GitHub, n8n, Slack) for end-to-end automation. Captures patterns that span tool boundaries.
git-pr-workflows-pr-enhance
You are a PR optimization expert specializing in creating high-quality pull requests that facilitate efficient code reviews. Generate comprehensive PR descriptions, automate review processes, and ensu