data-processing
Process JSON with jq and YAML/TOML with yq. Filter, transform, query structured data efficiently. Triggers on: parse JSON, extract from YAML, query config, Docker Compose, K8s manifests, GitHub Actions workflows, package.json, filter data.
Best use case
data-processing is best used when you need a repeatable AI agent workflow instead of a one-off prompt.
Process JSON with jq and YAML/TOML with yq. Filter, transform, query structured data efficiently. Triggers on: parse JSON, extract from YAML, query config, Docker Compose, K8s manifests, GitHub Actions workflows, package.json, filter data.
Teams using data-processing should expect a more consistent output, faster repeated execution, less prompt rewriting.
When to use this skill
- You want a reusable workflow that can be run more than once with consistent structure.
When not to use this skill
- You only need a quick one-off answer and do not need a reusable workflow.
- You cannot install or maintain the underlying files, dependencies, or repository context.
Installation
Claude Code / Cursor / Codex
Manual Installation
- Download SKILL.md from GitHub
- Place it in
.claude/skills/data-processing/SKILL.mdinside your project - Restart your AI agent — it will auto-discover the skill
How data-processing Compares
| Feature / Agent | data-processing | Standard Approach |
|---|---|---|
| Platform Support | Not specified | Limited / Varies |
| Context Awareness | High | Baseline |
| Installation Complexity | Unknown | N/A |
Frequently Asked Questions
What does this skill do?
Process JSON with jq and YAML/TOML with yq. Filter, transform, query structured data efficiently. Triggers on: parse JSON, extract from YAML, query config, Docker Compose, K8s manifests, GitHub Actions workflows, package.json, filter data.
Where can I find the source code?
You can find the source code on GitHub using the link provided at the top of the page.
SKILL.md Source
# Data Processing
Query, filter, and transform structured data (JSON, YAML, TOML) efficiently from the command line.
## Tools
| Tool | Command | Use For |
|------|---------|---------|
| jq | `jq '.key' file.json` | JSON processing |
| yq | `yq '.key' file.yaml` | YAML/TOML processing |
## jq Essentials
```bash
# Extract single field
jq '.name' package.json
# Extract nested field
jq '.scripts.build' package.json
# Extract from array
jq '.dependencies[0]' package.json
# Extract multiple fields
jq '{name, version}' package.json
# Navigate deeply nested
jq '.data.users[0].profile.email' response.json
# Filter by condition
jq '.users[] | select(.active == true)' data.json
# Transform each element
jq '.users | map({id, name})' data.json
# Count elements
jq '.users | length' data.json
# Raw string output
jq -r '.name' package.json
```
## yq Essentials
```bash
# Extract field
yq '.name' config.yaml
# Extract nested
yq '.services.web.image' docker-compose.yml
# List all keys
yq 'keys' config.yaml
# List all service names (Docker Compose)
yq '.services | keys' docker-compose.yml
# Get container images (K8s)
yq '.spec.template.spec.containers[].image' deployment.yaml
# Update value (in-place)
yq -i '.version = "2.0.0"' config.yaml
# TOML to JSON
yq -p toml -o json '.' config.toml
```
## Quick Reference
| Task | jq | yq |
|------|----|----|
| Get field | `jq '.key'` | `yq '.key'` |
| Array element | `jq '.[0]'` | `yq '.[0]'` |
| Filter array | `jq '.[] \| select(.x)'` | `yq '.[] \| select(.x)'` |
| Transform | `jq 'map(.x)'` | `yq 'map(.x)'` |
| Count | `jq 'length'` | `yq 'length'` |
| Keys | `jq 'keys'` | `yq 'keys'` |
| Pretty print | `jq '.'` | `yq '.'` |
| Compact | `jq -c` | `yq -o json -I0` |
| Raw output | `jq -r` | `yq -r` |
| In-place edit | - | `yq -i` |
## When to Use
- Reading package.json dependencies
- Parsing Docker Compose configurations
- Analyzing Kubernetes manifests
- Processing GitHub Actions workflows
- Extracting data from API responses
- Filtering large JSON datasets
- Config file manipulation
- Data format conversion
## Additional Resources
For complete pattern libraries, load:
- `./references/jq-patterns.md` - Arrays, filtering, transformation, aggregation, output formatting
- `./references/yq-patterns.md` - Docker Compose, K8s, GitHub Actions, TOML, YAML modification
- `./references/config-files.md` - package.json, tsconfig, eslint/prettier patternsRelated Skills
College Football Data (CFB)
Before writing queries, consult `references/api-reference.md` for endpoints, conference IDs, team IDs, and data shapes.
College Basketball Data (CBB)
Before writing queries, consult `references/api-reference.md` for endpoints, conference IDs, team IDs, and data shapes.
validating-database-integrity
Process use when you need to ensure database integrity through comprehensive data validation. This skill validates data types, ranges, formats, referential integrity, and business rules. Trigger with phrases like "validate database data", "implement data validation rules", "enforce data integrity constraints", or "validate data formats".
forecasting-time-series-data
This skill enables Claude to forecast future values based on historical time series data. It analyzes time-dependent data to identify trends, seasonality, and other patterns. Use this skill when the user asks to predict future values of a time series, analyze trends in data over time, or requires insights into time-dependent data. Trigger terms include "forecast," "predict," "time series analysis," "future values," and requests involving temporal data.
generating-test-data
This skill enables Claude to generate realistic test data for software development. It uses the test-data-generator plugin to create users, products, orders, and custom schemas for comprehensive testing. Use this skill when you need to populate databases, simulate user behavior, or create fixtures for automated tests. Trigger phrases include "generate test data", "create fake users", "populate database", "generate product data", "create test orders", or "generate data based on schema". This skill is especially useful for populating testing environments or creating sample data for demonstrations.
test-data-builder
Test Data Builder - Auto-activating skill for Test Automation. Triggers on: test data builder, test data builder Part of the Test Automation skill category.
splitting-datasets
Process split datasets into training, validation, and testing sets for ML model development. Use when requesting "split dataset", "train-test split", or "data partitioning". Trigger with relevant phrases based on skill purpose.
scanning-database-security
Process use when you need to work with security and compliance. This skill provides security scanning and vulnerability detection with comprehensive guidance and automation. Trigger with phrases like "scan for vulnerabilities", "implement security controls", or "audit security".
processing-computer-vision-tasks
Process images using object detection, classification, and segmentation. Use when requesting "analyze image", "object detection", "image classification", or "computer vision". Trigger with relevant phrases based on skill purpose.
processing-api-batches
Optimize bulk API requests with batching, throttling, and parallel execution. Use when processing bulk API operations efficiently. Trigger with phrases like "process bulk requests", "batch API calls", or "handle batch operations".
preprocessing-data-with-automated-pipelines
Process automate data cleaning, transformation, and validation for ML tasks. Use when requesting "preprocess data", "clean data", "ETL pipeline", or "data transformation". Trigger with relevant phrases based on skill purpose.
optimizing-database-connection-pooling
Process use when you need to work with connection management. This skill provides connection pooling and management with comprehensive guidance and automation. Trigger with phrases like "manage connections", "configure pooling", or "optimize connection usage".