claude-player

# ClaudePlayer

An AI agent that plays Game Boy games autonomously using Claude's vision capabilities and the PyBoy emulator.

## Overview

ClaudePlayer connects Claude to a Game Boy emulator. Each turn, Claude receives a screenshot of the current game frame, reasons about what to do, and sends button inputs back to the emulator. It maintains a structured memory system to track game progress (items, NPCs, locations, quests, stats).

## Prerequisites

- Python 3.10+
- Pipenv
- An Anthropic API key
- A Game Boy ROM file (`.gb`)

## Setup

```bash
# Install dependencies
pipenv install

# Create .env with your API key
echo "ANTHROPIC_API_KEY=your_key_here" > .env

# Place a Game Boy ROM in the project directory
```

## Configuration

Edit `config.json`:

| Key | Description | Default |
|-----|-------------|---------|
| `ROM_PATH` | Path to Game Boy ROM | Required |
| `STATE_PATH` | Saved emulator state file | `null` |
| `EMULATION_MODE` | `"turn_based"` or `"continuous"` | `"turn_based"` |
| `MODEL_DEFAULTS.MODEL` | Claude model to use | `"claude-sonnet-4-5-20250929"` |
| `MODEL_DEFAULTS.THINKING` | Enable extended thinking | `true` |
| `RATE_LIMITS.RPM_THRESHOLD` | Requests per minute limit | Configured in file |
| `RATE_LIMITS.TPM_THRESHOLD` | Tokens per minute limit | Configured in file |
| `REDIS_LOGS` | Optional Redis logging config | `null` |

## Running

```bash
pipenv shell

# Default config
python play.py

# Custom config
python play.py --config my_config.json

# Create a saved emulator state
python emu_setup.py
```

## Project Structure

| Path | Purpose |
|------|---------|
| `play.py` | Launcher script |
| `claude_player/main.py` | CLI entry point with arg parsing |
| `claude_player/agent/game_agent.py` | Main orchestrator: emulator init, game loop, coordination |
| `claude_player/interface/claude_interface.py` | Claude API communication and rate limiting |
| `claude_player/state/game_state.py` | Game state tracking (memory, goals, history) |
| `claude_player/tools/tool_setup.py` | Tool definitions (send_inputs, memory ops, etc.) |
| `claude_player/tools/tool_registry.py` | Tool registry system |
| `claude_player/config/config_loader.py` | Config file parsing |
| `claude_player/config/config_class.py` | Configuration data class |
| `claude_player/utils/game_utils.py` | Button input parsing, screenshot capture |
| `claude_player/utils/memory_reader.py` | Game memory/stats reader |
| `claude_player/agent/summary_generator.py` | Periodic game progress summarization |
| `config.json` | Runtime configuration |

## Available Tools (for the agent during gameplay)

| Tool | Description |
|------|-------------|
| `send_inputs` | Send button sequences to the emulator |
| `set_game` | Identify the current game |
| `set_current_goal` | Update the gameplay objective |
| `add_to_memory` | Store items, NPCs, locations, quests, mechanics, stats |
| `remove_from_memory` | Remove a memory entry |
| `update_memory_item` | Update an existing memory entry |
| `toggle_thinking` | Toggle extended thinking mode on/off |

## How It Works

1. PyBoy emulator loads the ROM and renders a frame
2. The frame is captured as a screenshot and sent to Claude
3. Claude analyzes the screen, reasons about the game state, and calls tools
4. `send_inputs` presses buttons on the emulator (A, B, Up, Down, Left, Right, Start, Select)
5. In turn-based mode, the emulator only advances when the agent acts
6. The agent maintains structured memory to track long-term game progress
7. Periodic summaries compress the conversation history to stay within context limits