building-chatgpt-apps

# ChatGPT Apps SDK Development Guide

## Overview

Create ChatGPT Apps with interactive widgets that render rich UI inside ChatGPT conversations. Apps combine MCP servers (providing tools) with embedded HTML widgets that communicate via the `window.openai` API.

---

## window.openai API Reference

Widgets communicate with ChatGPT through these APIs:

### sendFollowUpMessage (Recommended for Actions)

Send a follow-up prompt to ChatGPT on behalf of the user:

```javascript
// Trigger a follow-up conversation
if (window.openai?.sendFollowUpMessage) {
  await window.openai.sendFollowUpMessage({
    prompt: 'Summarize this chapter for me'
  });
}
```

**Use for**: Action buttons that suggest next steps (summarize, explain, etc.)

### toolOutput

Send structured data back from widget interactions:

```javascript
// Send data back to ChatGPT
if (window.openai?.toolOutput) {
  window.openai.toolOutput({
    action: 'chapter_selected',
    chapter: 1,
    title: 'Introduction'
  });
}
```

**Use for**: Selections, form submissions, user choices that feed into tool responses.

### callTool

Call another MCP tool from within a widget:

```javascript
// Call a tool directly
if (window.openai?.callTool) {
  await window.openai.callTool({
    name: 'read-chapter',
    arguments: { chapter: 2 }
  });
}
```

**Use for**: Navigation between content, chaining tool calls.

---

## Critical: Button Interactivity Limitations

**Important Discovery**: Widget buttons may render as **static UI elements** rather than interactive JavaScript buttons. ChatGPT renders widgets in a sandboxed iframe where some click handlers don't fire reliably.

### What Works
- `sendFollowUpMessage` - Reliably triggers follow-up prompts
- Simple onclick handlers for `toolOutput` calls
- CSS hover effects and visual feedback

### What May Not Work
- Complex interactive JavaScript (selection APIs, etc.)
- Multiple chained tool calls from buttons
- `window.getSelection()` for text selection features

### Recommended Pattern: Suggestion Buttons

Instead of complex interactions, use simple buttons that suggest prompts:

```html
<div class="action-buttons">
  <button class="btn btn-primary" id="summarizeBtn">
    📝 Summarize Chapter
  </button>
  <button class="btn btn-primary" id="explainBtn">
    💡 Explain Key Concepts
  </button>
</div>

<script>
document.getElementById('summarizeBtn')?.addEventListener('click', async () => {
  if (window.openai?.sendFollowUpMessage) {
    await window.openai.sendFollowUpMessage({
      prompt: 'Summarize this chapter for me'
    });
  }
});

document.getElementById('explainBtn')?.addEventListener('click', async () => {
  if (window.openai?.sendFollowUpMessage) {
    await window.openai.sendFollowUpMessage({
      prompt: 'Explain the key concepts from this chapter'
    });
  }
});
</script>
```

---

## Architecture Summary

```
┌─────────────────────────────────────────────────────────────────┐
│                        ChatGPT UI                                │
│  ┌─────────────────────────────────────────────────────────────┐│
│  │                    Widget (iframe)                          ││
│  │   HTML + CSS + JS                                          ││
│  │   Calls: window.openai.toolOutput({action: "...", ...})    ││
│  └─────────────────────────────────────────────────────────────┘│
│                              │                                   │
│                              ▼                                   │
│                     ChatGPT Backend                              │
│                              │                                   │
│                              ▼                                   │
│              MCP Server (FastMCP + HTTP)                         │
│              - Tools: open-book, read-chapter, etc.              │
│              - Resources: widget HTML (text/html+skybridge)      │
│              - Response includes: _meta["openai.com/widget"]     │
└─────────────────────────────────────────────────────────────────┘
```

---

## Quick Start

1. **Create MCP server** with FastMCP and widget resources
2. **Define widget HTML** that uses `window.openai.toolOutput`
3. **Add response metadata** with `_meta["openai.com/widget"]`
4. **Expose via ngrok** for ChatGPT access
5. **Register in ChatGPT** Developer Mode settings

---

## Widget HTML Requirements

### Basic Widget Template

```html
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>My Widget</title>
  <style>
    * { margin: 0; padding: 0; box-sizing: border-box; }
    body {
      font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif;
      background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
      min-height: 100vh;
      padding: 24px;
      color: white;
    }
    .container { max-width: 600px; margin: 0 auto; }
    .card {
      background: rgba(255,255,255,0.95);
      color: #333;
      padding: 24px;
      border-radius: 16px;
      box-shadow: 0 10px 40px rgba(0,0,0,0.2);
    }
    .btn {
      background: #667eea;
      color: white;
      border: none;
      padding: 12px 24px;
      border-radius: 8px;
      cursor: pointer;
      font-size: 16px;
    }
    .btn:hover { background: #5a6fd6; }
  </style>
</head>
<body>
  <div class="container">
    <div class="card">
      <h1>Widget Title</h1>
      <p>Widget content here</p>
      <button class="btn" onclick="handleAction()">Click Me</button>
    </div>
  </div>
  <script>
    function handleAction() {
      // Communicate back to ChatGPT
      if (window.openai && window.openai.toolOutput) {
        window.openai.toolOutput({
          action: "button_clicked",
          data: { timestamp: Date.now() }
        });
      }
    }
  </script>
</body>
</html>
```

### Key Widget Rules

1. **Always check `window.openai.toolOutput`** before calling
2. **Use inline styles** - external CSS may not load reliably
3. **Keep widgets self-contained** - all HTML/CSS/JS in one file
4. **Test with actual ChatGPT** - browser preview won't have `window.openai`

---

## MCP Server Setup (FastMCP Python)

### Project Structure

```
my_chatgpt_app/
├── main.py              # FastMCP server with widgets
├── requirements.txt     # Dependencies
└── .env                 # Environment variables
```

### requirements.txt

```
mcp[cli]>=1.9.2
uvicorn>=0.32.0
httpx>=0.28.0
python-dotenv>=1.0.0
```

### main.py Template

```python
import mcp.types as types
from mcp.server.fastmcp import FastMCP

# Widget MIME type for ChatGPT
MIME_TYPE = "text/html+skybridge"

# Define your widget HTML
MY_WIDGET = '''<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <style>
    body { font-family: sans-serif; padding: 20px; }
    .container { max-width: 500px; margin: 0 auto; }
  </style>
</head>
<body>
  <div class="container">
    <h1>Hello from Widget!</h1>
    <p>This content renders inside ChatGPT.</p>
  </div>
</body>
</html>'''

# Widget registry
WIDGETS = {
    "main-widget": {
        "uri": "ui://widget/main.html",
        "html": MY_WIDGET,
        "title": "My Widget",
    },
}

# Create FastMCP server
mcp = FastMCP("My ChatGPT App")


@mcp.resource(
    uri="ui://widget/{widget_name}.html",
    name="Widget Resource",
    mime_type=MIME_TYPE
)
def widget_resource(widget_name: str) -> str:
    """Serve widget HTML."""
    widget_key = f"{widget_name}"
    if widget_key in WIDGETS:
        return WIDGETS[widget_key]["html"]
    return WIDGETS["main-widget"]["html"]


def _embedded_widget_resource(widget_id: str) -> types.EmbeddedResource:
    """Create embedded widget resource for tool response."""
    widget = WIDGETS[widget_id]
    return types.EmbeddedResource(
        type="resource",
        resource=types.TextResourceContents(
            uri=widget["uri"],
            mimeType=MIME_TYPE,
            text=widget["html"],
            title=widget["title"],
        ),
    )


def listing_meta() -> dict:
    """Tool metadata for ChatGPT tool listing."""
    return {
        "openai.com/widget": {
            "uri": WIDGETS["main-widget"]["uri"],
            "title": WIDGETS["main-widget"]["title"]
        }
    }


def response_meta() -> dict:
    """Response metadata with embedded widget."""
    return {
        "openai.com/widget": _embedded_widget_resource("main-widget")
    }


@mcp.tool(
    annotations={
        "title": "My Tool",
        "readOnlyHint": True,
        "openWorldHint": False,
    },
    _meta=listing_meta(),
)
def my_tool() -> types.CallToolResult:
    """Description of what this tool does."""
    return types.CallToolResult(
        content=[
            types.TextContent(
                type="text",
                text="Tool executed successfully!"
            )
        ],
        structuredContent={
            "status": "success",
            "message": "Data for the widget"
        },
        _meta=response_meta(),
    )


if __name__ == "__main__":
    import uvicorn
    print("Starting MCP Server on http://localhost:8001")
    print("Connect via: https://your-tunnel.ngrok-free.app/mcp")
    uvicorn.run(
        "main:mcp.app",
        host="0.0.0.0",
        port=8001,
        reload=True
    )
```

---

## Response Metadata Format

### Critical: `_meta["openai.com/widget"]`

Tool responses MUST include widget metadata:

```python
types.CallToolResult(
    content=[types.TextContent(type="text", text="...")],
    structuredContent={"key": "value"},  # Data for widget
    _meta={
        "openai.com/widget": types.EmbeddedResource(
            type="resource",
            resource=types.TextResourceContents(
                uri="ui://widget/my-widget.html",
                mimeType="text/html+skybridge",
                text=WIDGET_HTML,
                title="My Widget",
            ),
        )
    },
)
```

### structuredContent

Data passed to the widget. The widget can access this via `window.openai` APIs.

---

## Development Setup

### 1. Start Local Server

```bash
cd my_chatgpt_app
python main.py
# Server runs on http://localhost:8001
```

### 2. Start ngrok Tunnel

```bash
ngrok http 8001
# Get URL like: https://abc123.ngrok-free.app
```

### 3. Register in ChatGPT

1. Go to https://chatgpt.com/apps
2. Click Settings (gear icon)
3. Enable **Developer mode**
4. Click **Create app**
5. Fill in:
   - **Name**: Your App Name
   - **MCP Server URL**: `https://abc123.ngrok-free.app/mcp`
   - **Authentication**: No Auth (for development)
6. Check "I understand and want to continue"
7. Click **Create**

### 4. Test the App

1. Start a new chat in ChatGPT
2. Type `@` to see available apps
3. Select your app
4. Ask it to use your tool

---

## Common Issues and Solutions

### Widget Shows "Loading..." Forever

**Cause**: Widget HTML not being delivered correctly.

**Solution**:
1. Check server logs for `CallToolRequest` processing
2. Verify `_meta["openai.com/widget"]` in response
3. Ensure MIME type is `text/html+skybridge`

### Cached Widget Not Updating

**Cause**: ChatGPT caches widgets aggressively.

**Solution**:
1. Delete the app in Settings > Apps
2. Kill server and ngrok
3. Start fresh ngrok tunnel (new URL)
4. Create new app with new URL
5. Test in new conversation

### Widget JavaScript Errors

**Cause**: `window.openai` not available.

**Solution**: Always check before calling:
```javascript
if (window.openai && window.openai.toolOutput) {
  window.openai.toolOutput({...});
}
```

### Tool Not Showing in @mentions

**Cause**: MCP server not connected or tools not registered.

**Solution**:
1. Check server is running and accessible via ngrok URL
2. Verify ngrok tunnel is active: `curl https://your-url.ngrok-free.app/mcp`
3. Check server logs for `ListToolsRequest`

---

## Verification

Run: `python3 scripts/verify.py`

Expected: `✓ building-chatgpt-apps skill ready`

## If Verification Fails

1. Run diagnostic: Check references/ folder exists
2. Check: All reference files present
3. **Stop and report** if still failing

---

## References

- [Complete Template](references/complete_template.md) - Ready-to-use server + widget template
- [Widget Patterns](references/widget_patterns.md) - HTML/CSS/JS widget examples
- [Response Structure](references/response_structure.md) - Metadata format details
- [Debugging Guide](references/debugging.md) - Troubleshooting common issues