ai-agent-observability-evaluation
Use when measuring, evaluating, replaying, evidencing, or tracking success for AI agent tasks, steps, traces, and outcomes.
Best use case
ai-agent-observability-evaluation is best used when you need a repeatable AI agent workflow instead of a one-off prompt.
Use when measuring, evaluating, replaying, evidencing, or tracking success for AI agent tasks, steps, traces, and outcomes.
Teams using ai-agent-observability-evaluation should expect a more consistent output, faster repeated execution, less prompt rewriting.
When to use this skill
- You want a reusable workflow that can be run more than once with consistent structure.
When not to use this skill
- You only need a quick one-off answer and do not need a reusable workflow.
- You cannot install or maintain the underlying files, dependencies, or repository context.
Installation
Claude Code / Cursor / Codex
Manual Installation
- Download SKILL.md from GitHub
- Place it in
.claude/skills/ai-agent-observability-evaluation/SKILL.mdinside your project - Restart your AI agent — it will auto-discover the skill
How ai-agent-observability-evaluation Compares
| Feature / Agent | ai-agent-observability-evaluation | Standard Approach |
|---|---|---|
| Platform Support | Not specified | Limited / Varies |
| Context Awareness | High | Baseline |
| Installation Complexity | Unknown | N/A |
Frequently Asked Questions
What does this skill do?
Use when measuring, evaluating, replaying, evidencing, or tracking success for AI agent tasks, steps, traces, and outcomes.
Where can I find the source code?
You can find the source code on GitHub using the link provided at the top of the page.
SKILL.md Source
# AI Agent Observability Evaluation Acknowledgement: Shared by Peter Bamuhigire, techguypeter.com, +256 784 464178. <!-- dual-compat-start --> ## Use When - Design agent evals, task-success metrics, traces, replay, evidence capture, and regression gates. - Instrument agent runs with events, step logs, artifacts, and customer-visible status. - Turn agent execution evidence into QA, audit, incident, or product-quality signals. ## Do Not Use When - The work is not AI-specific or agentic-AI-specific. - A narrower retained AI parent skill fits the request better. ## Required Inputs - Product, tenant, user, data, risk, and operational context relevant to the AI workflow. - Target artifact: design, implementation plan, audit, test strategy, UX flow, commercial policy, or runbook. - Constraints from security, privacy, reliability, billing, support, and compliance stakeholders when relevant. ## Workflow 1. Read this SKILL.md first. 2. Load [references/routing.md](references/routing.md) to select the absorbed child reference that matches the task. 3. Load only the selected child reference files needed for the current request. 4. Produce execution-oriented output with assumptions, risks, evidence, and next actions where relevant. ## Quality Standards - Keep routing explicit: name which reference files were used when the work depends on absorbed material. - Preserve tenant isolation, auditability, cost controls, safety gates, and operational evidence when they matter. - Prefer concrete contracts, checklists, tables, schemas, runbooks, and decision records over broad summaries. ## Anti-Patterns - Loading every absorbed reference by default. - Treating AI-specific billing, compliance, safety, or UX concerns as generic SaaS work without checking AI failure modes. - Hiding retired skill names; old slugs must remain discoverable through [references/routing.md](references/routing.md). ## Outputs - A concrete deliverable matched to the request: architecture, implementation plan, audit, policy, runbook, UX flow, test strategy, or operating model. - The selected consolidated reference files and any assumptions, risks, evidence requirements, or follow-up actions that affect execution. ## References - [references/routing.md](references/routing.md) maps retired child skill slugs to their consolidated reference folders. ## Consolidated Child References - Load [references/routing.md](references/routing.md) to map retired AI child skill slugs to their reference modules. <!-- dual-compat-end -->
Related Skills
observability-monitoring
Use when designing or reviewing logs, metrics, traces, alerts, SLOs, dashboards, audit events, or production telemetry for web apps, APIs, SaaS platforms, mobile backends, and AI systems. Covers instrumentation strategy, diagnosis-first telemetry, alert quality, and operational visibility.
ai-observability-and-debugging
Use when building the observability stack for AI features in a multi-tenant SaaS — prompt/response tracing, semantic logging, replay tooling, "show me why this answer", per-stage latency/cost breakdown, ticket→trace tie-back, and dashboards that answer the operational questions (which tenant, which feature, which prompt version, which model).
ai-evaluation
Use when setting up quality assurance for AI features — defining evaluation criteria, measuring output quality, using AI-as-judge, monitoring production AI, detecting drift, and building user feedback loops
web-app-security-audit
Use when auditing a PHP/JavaScript/HTML web application for security vulnerabilities. Covers configuration, authentication, authorization, input validation, XSS, API security, HTTP headers, and dependency scanning. Produces a severity-rated audit...
vibe-security-skill
Use when designing or reviewing security for a web application, API, or multi-tenant SaaS — produces threat model, abuse case list, auth/authz matrix, and secret handling plan; covers OWASP Top 10 2025 and the AI-code-generation blind spots. Neighbours — api-design-first owns auth model fields, deployment-release-engineering owns secret rotation choreography, ai-security and llm-security own model-specific threats.
network-security
Use when designing, hardening, or auditing network-layer security for self-managed Debian/Ubuntu SaaS infrastructure — firewalls (nftables/UFW), WAF (ModSecurity + OWASP CRS), VPN (WireGuard, OpenVPN, IPsec), TLS/PKI ops, IDS/IPS (Suricata, Fail2ban), zero-trust, SSH hardening, DDoS mitigation, DNS security. Complements web-app-security-audit (app layer) and cicd-devsecops (secrets/CI).
linux-security-hardening
Use when hardening a Debian/Ubuntu server — user/group/sudo hardening, file permission audits, PAM password policy + MFA, AppArmor mandatory access control, auditd system call logging, kernel sysctl hardening, file integrity monitoring (AIDE), rootkit detection (rkhunter/chkrootkit), unattended security patching, GRUB + UEFI + LUKS boot security, and CIS benchmark compliance.
dpia-generator
Generate a Data Protection Impact Assessment (DPIA), Uganda DPPA 2019-compliant. Use when producing or reviewing a data protection impact assessment, a privacy impact assessment, when uganda-dppa-compliance flags [DPIA-REQUIRED], or when processing large-scale or sensitive personal data for a new feature.
code-safety-scanner
Scan any codebase for 14 critical safety issues across security vulnerabilities, server stability (500 errors), and payment misconfigurations. Use when auditing code before deployment, reviewing AI-generated code for production readiness, or...
world-class-engineering
Use when designing, building, reviewing, or upgrading production software systems that must be secure, performant, maintainable, scalable, and user-centered. Apply before writing specs, code, architecture, APIs, databases, mobile apps, SaaS platforms, or ERP systems.
update-Codex-documentation
Update project documentation files (README.md, PROJECT_BRIEF.md, TECH_STACK.md, ARCHITECTURE.md, docs/API.md, docs/DATABASE.md, AGENTS.md, docs/plans/NEXT_FEATURES.md) when significant changes occur. MANDATORY at end of each work session to...
skill-writing
Use when creating or upgrading skills in this repository. Covers repository-specific frontmatter rules, progressive disclosure, reference-file strategy, validation, and the quality bar required for production-grade engineering skills.