airflow-dag-patterns

Build production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use when creating data pipelines, orchestrating workflows, or scheduling batch jobs.

Best use case

airflow-dag-patterns is best used when you need a repeatable AI agent workflow instead of a one-off prompt.

Build production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use when creating data pipelines, orchestrating workflows, or scheduling batch jobs.

Teams using airflow-dag-patterns should expect a more consistent output, faster repeated execution, less prompt rewriting.

When to use this skill

  • You want a reusable workflow that can be run more than once with consistent structure.

When not to use this skill

  • You only need a quick one-off answer and do not need a reusable workflow.
  • You cannot install or maintain the underlying files, dependencies, or repository context.

Installation

Claude Code / Cursor / Codex

$curl -o ~/.claude/skills/airflow-dag-patterns/SKILL.md --create-dirs "https://raw.githubusercontent.com/ratnesh-maurya/cursor-claude-personas/main/data-engineer/.claude/skills/airflow-dag-patterns/SKILL.md"

Manual Installation

  1. Download SKILL.md from GitHub
  2. Place it in .claude/skills/airflow-dag-patterns/SKILL.md inside your project
  3. Restart your AI agent — it will auto-discover the skill

How airflow-dag-patterns Compares

Feature / Agentairflow-dag-patternsStandard Approach
Platform SupportNot specifiedLimited / Varies
Context Awareness High Baseline
Installation ComplexityUnknownN/A

Frequently Asked Questions

What does this skill do?

Build production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use when creating data pipelines, orchestrating workflows, or scheduling batch jobs.

Where can I find the source code?

You can find the source code on GitHub using the link provided at the top of the page.

SKILL.md Source

# Apache Airflow DAG Patterns

Production-ready patterns for Apache Airflow including DAG design, operators, sensors, testing, and deployment strategies.

## Use this skill when

- Creating data pipeline orchestration with Airflow
- Designing DAG structures and dependencies
- Implementing custom operators and sensors
- Testing Airflow DAGs locally
- Setting up Airflow in production
- Debugging failed DAG runs

## Do not use this skill when

- You only need a simple cron job or shell script
- Airflow is not part of the tooling stack
- The task is unrelated to workflow orchestration

## Instructions

1. Identify data sources, schedules, and dependencies.
2. Design idempotent tasks with clear ownership and retries.
3. Implement DAGs with observability and alerting hooks.
4. Validate in staging and document operational runbooks.

Refer to `resources/implementation-playbook.md` for detailed patterns, checklists, and templates.

## Safety

- Avoid changing production DAG schedules without approval.
- Test backfills and retries carefully to prevent data duplication.

## Resources

- `resources/implementation-playbook.md` for detailed patterns, checklists, and templates.

Related Skills

wcag-audit-patterns

5
from ratnesh-maurya/cursor-claude-personas

Conduct WCAG 2.2 accessibility audits with automated testing, manual verification, and remediation guidance. Use when auditing websites for accessibility, fixing WCAG violations, or implementing ac...

similarity-search-patterns

5
from ratnesh-maurya/cursor-claude-personas

Implement efficient similarity search with vector databases. Use when building semantic search, implementing nearest neighbor queries, or optimizing retrieval performance.

projection-patterns

5
from ratnesh-maurya/cursor-claude-personas

Build read models and projections from event streams. Use when implementing CQRS read sides, building materialized views, or optimizing query performance in event-sourced systems.

ddd-tactical-patterns

5
from ratnesh-maurya/cursor-claude-personas

Apply DDD tactical patterns in code using entities, value objects, aggregates, repositories, and domain events with explicit invariants.

rust-async-patterns

5
from ratnesh-maurya/cursor-claude-personas

Master Rust async programming with Tokio, async traits, error handling, and concurrent patterns. Use when building async Rust applications, implementing concurrent systems, or debugging async code.

go-concurrency-patterns

5
from ratnesh-maurya/cursor-claude-personas

Master Go concurrency with goroutines, channels, sync primitives, and context. Use when building concurrent Go applications, implementing worker pools, or debugging race conditions.

cc-skill-backend-patterns

5
from ratnesh-maurya/cursor-claude-personas

Backend architecture patterns, API design, database optimization, and server-side best practices for Node.js, Express, and Next.js API routes.

react-ui-patterns

5
from ratnesh-maurya/cursor-claude-personas

Modern React UI patterns for loading states, error handling, and data fetching. Use when building UI components, handling async data, or managing UI states.

cc-skill-frontend-patterns

5
from ratnesh-maurya/cursor-claude-personas

Frontend development patterns for React, Next.js, state management, performance optimization, and UI best practices.

angular-ui-patterns

5
from ratnesh-maurya/cursor-claude-personas

Modern Angular UI patterns for loading states, error handling, and data display. Use when building UI components, handling async data, or managing component states.

elixir-otp-patterns

5
from ratnesh-maurya/cursor-claude-personas

OTP architecture patterns for Elixir: GenServer, Supervisor, DynamicSupervisor, Registry, Application. Use when designing fault-tolerant process architectures, supervision trees, or stateful services.

ecto-database-patterns

5
from ratnesh-maurya/cursor-claude-personas

Ecto database patterns: changesets, composable queries, Multi transactions, migrations, multi-tenancy. Use when designing database interactions in Elixir applications.