ml-experiment-tracker

Plan reproducible ML experiment runs with explicit parameters, metrics, and artifacts. Use before model training to standardize tracking-ready experiment definitions.

3,891 stars

byopenclaw

Complexity: easy

View on GitHub Installation ↓

About this skill

The ML Experiment Tracker skill is designed to assist users, particularly ML engineers and data scientists, in creating structured and reproducible plans for their machine learning experiments. It provides a guided workflow to define crucial experiment components such as datasets, target tasks, model families, and the parameter search space. By standardizing these definitions, the skill ensures consistency in how experiments are set up and subsequently logged in experiment tracking systems. This is vital for maintaining clear research and development records, enabling easier comparison between different runs, and fostering effective collaboration within ML teams. Before any model training commences, this skill prompts users to define metrics and acceptance thresholds, ensuring that clear success criteria are established upfront. It culminates in producing a detailed run plan, complete with versioning and artifact expectations. This plan can then be exported for execution with various tracking tools, significantly enhancing experimental rigor, clarity, and overall reproducibility.

Best use case

The primary use case for the ML Experiment Tracker is to standardize the planning phase of machine learning projects, ensuring reproducibility and consistent logging. It is most beneficial for ML engineers, data scientists, and research teams who need to manage multiple experiment runs, systematically compare model performance, and ensure that all experimental details (parameters, metrics, artifacts) are explicitly defined and trackable from the outset.

Plan reproducible ML experiment runs with explicit parameters, metrics, and artifacts. Use before model training to standardize tracking-ready experiment definitions.

A structured, machine-readable ML experiment run plan detailing datasets, parameters, metrics, artifacts, and versioning for consistent logging and execution.

Practical example

Example input

Plan a new experiment for a multi-class text classification model. Use the AG News dataset, target F1-score (macro), and evaluate fastText and TF-IDF + Logistic Regression models. Include a hyperparameter search space for learning rates [0.01, 0.001] and regularization strength [0.1, 1.0].

Example output

```json
{
  "experiment_name": "text-classification-v2",
  "dataset": "AG News",
  "task": "multi-class text classification",
  "model_family": [
    "fastText",
    "TF-IDF_LogisticRegression"
  ],
  "parameters_search_space": {
    "learning_rate": [
      0.01,
      0.001
    ],
    "regularization_strength": [
      0.1,
      1.0
    ]
  },
  "metrics": [
    {
      "name": "f1_score_macro",
      "threshold": 0.85,
      "direction": "maximize"
    }
  ],
  "artifacts_to_track": [
    "model_weights",
    "training_logs",
    "confusion_matrix.png"
  ],
  "version": "2.0.0"
}
```

When to use this skill

Before starting a new ML model training project.
When reproducibility of ML experiments is a high priority.
To standardize experiment definitions across a team or organization.
When integrating with experiment tracking systems like MLflow, Weights & Biases, etc.

When not to use this skill

For tasks unrelated to ML experiment planning or tracking.
When quick, ad-hoc model training without formal tracking is sufficient.
If you already have a mature and standardized experiment planning process fully automated.

Installation

Claude Code / Cursor / Codex

$curl -o ~/.claude/skills/ml-experiment-tracker/SKILL.md --create-dirs "https://raw.githubusercontent.com/openclaw/skills/main/skills/0x-professor/ml-experiment-tracker/SKILL.md"

Manual Installation

Download SKILL.md from GitHub
Place it in .claude/skills/ml-experiment-tracker/SKILL.md inside your project
Restart your AI agent — it will auto-discover the skill

How ml-experiment-tracker Compares

Feature / Agent	ml-experiment-tracker	Standard Approach
Platform Support	Not specified	Limited / Varies
Context Awareness	High	Baseline
Installation Complexity	easy	N/A

Frequently Asked Questions

What does this skill do?

Plan reproducible ML experiment runs with explicit parameters, metrics, and artifacts. Use before model training to standardize tracking-ready experiment definitions.

How difficult is it to install?

The installation complexity is rated as easy. You can find the installation instructions above.

Where can I find the source code?

You can find the source code on GitHub using the link provided at the top of the page.

Related Guides

Best AI Skills for Claude

Explore the best AI skills for Claude and Claude Code across coding, research, workflow automation, documentation, and agent operations.

ChatGPT vs Claude for Agent Skills

Compare ChatGPT and Claude for AI agent skills across coding, writing, research, and reusable workflow execution.

Best AI Skills for ChatGPT

Find the best AI skills to adapt into ChatGPT workflows for research, writing, summarization, planning, and repeatable assistant tasks.

SKILL.md Source

# ML Experiment Tracker

## Overview

Generate structured experiment plans that can be logged consistently in experiment tracking systems.

## Workflow

1. Define dataset, target task, model family, and parameter search space.
2. Define metrics and acceptance thresholds before training.
3. Produce run plan with version and artifact expectations.
4. Export the run plan for execution in tracking tools.

## Use Bundled Resources

- Run `scripts/build_experiment_plan.py` to generate consistent run plans.
- Read `references/tracking-guide.md` for reproducibility checklist.

## Guardrails

- Keep inputs explicit and machine-readable.
- Always include metrics and baseline criteria.

Data & Research