phone-agent
Run a real-time AI phone agent using Twilio, Deepgram, and ElevenLabs. Handles incoming calls, transcribes audio, generates responses via LLM, and speaks back via streaming TTS. Use when user wants to: (1) Test voice AI capabilities, (2) Handle phone calls programmatically, (3) Build a conversational voice bot.
Best use case
phone-agent is best used when you need a repeatable AI agent workflow instead of a one-off prompt.
Run a real-time AI phone agent using Twilio, Deepgram, and ElevenLabs. Handles incoming calls, transcribes audio, generates responses via LLM, and speaks back via streaming TTS. Use when user wants to: (1) Test voice AI capabilities, (2) Handle phone calls programmatically, (3) Build a conversational voice bot.
Teams using phone-agent should expect a more consistent output, faster repeated execution, less prompt rewriting.
When to use this skill
- You want a reusable workflow that can be run more than once with consistent structure.
When not to use this skill
- You only need a quick one-off answer and do not need a reusable workflow.
- You cannot install or maintain the underlying files, dependencies, or repository context.
Installation
Claude Code / Cursor / Codex
Manual Installation
- Download SKILL.md from GitHub
- Place it in
.claude/skills/phone-agent/SKILL.mdinside your project - Restart your AI agent — it will auto-discover the skill
How phone-agent Compares
| Feature / Agent | phone-agent | Standard Approach |
|---|---|---|
| Platform Support | Not specified | Limited / Varies |
| Context Awareness | High | Baseline |
| Installation Complexity | Unknown | N/A |
Frequently Asked Questions
What does this skill do?
Run a real-time AI phone agent using Twilio, Deepgram, and ElevenLabs. Handles incoming calls, transcribes audio, generates responses via LLM, and speaks back via streaming TTS. Use when user wants to: (1) Test voice AI capabilities, (2) Handle phone calls programmatically, (3) Build a conversational voice bot.
Where can I find the source code?
You can find the source code on GitHub using the link provided at the top of the page.
SKILL.md Source
# Phone Agent Skill
Runs a local FastAPI server that acts as a real-time voice bridge.
## Architecture
```
Twilio (Phone) <--> WebSocket (Audio) <--> [Local Server] <--> Deepgram (STT)
|
+--> OpenAI (LLM)
+--> ElevenLabs (TTS)
```
## Prerequisites
1. **Twilio Account**: Phone number + TwiML App.
2. **Deepgram API Key**: For fast speech-to-text.
3. **OpenAI API Key**: For the conversation logic.
4. **ElevenLabs API Key**: For realistic text-to-speech.
5. **Ngrok** (or similar): To expose your local port 8080 to Twilio.
## Setup
1. **Install Dependencies**:
```bash
pip install -r scripts/requirements.txt
```
2. **Set Environment Variables** (in `~/.moltbot/.env`, `~/.clawdbot/.env`, or export):
```bash
export DEEPGRAM_API_KEY="your_key"
export OPENAI_API_KEY="your_key"
export ELEVENLABS_API_KEY="your_key"
export TWILIO_ACCOUNT_SID="your_sid"
export TWILIO_AUTH_TOKEN="your_token"
export PORT=8080
```
3. **Start the Server**:
```bash
python3 scripts/server.py
```
4. **Expose to Internet**:
```bash
ngrok http 8080
```
5. **Configure Twilio**:
- Go to your Phone Number settings.
- Set "Voice & Fax" -> "A Call Comes In" to **Webhook**.
- URL: `https://<your-ngrok-url>.ngrok.io/incoming`
- Method: `POST`
## Usage
Call your Twilio number. The agent should answer, transcribe your speech, think, and reply in a natural voice.
## Customization
- **System Prompt**: Edit `SYSTEM_PROMPT` in `scripts/server.py` to change the persona.
- **Voice**: Change `ELEVENLABS_VOICE_ID` to use different voices.
- **Model**: Switch `gpt-4o-mini` to `gpt-4` for smarter (but slower) responses.Related Skills
apipick-telegram-phone-check
Check if a phone number is registered on Telegram using the apipick Telegram Checker API.
phone-calls
Make AI-powered phone calls via Bland AI - book restaurants, make appointments, inquire about services. The AI calls on your behalf and reports back with transcripts.
clawphone
Encrypted Clawdbot-to-Clawdbot messaging. Send messages to friends' Clawdbots with end-to-end encryption.
elevenlabs-phone-reminder-lite
Build AI phone call reminders with ElevenLabs Conversational AI + Twilio. Free starter guide.
phone-voice
Connect ElevenLabs Agents to your OpenClaw via phone with Twilio. Includes caller ID auth, voice PIN security, call screening, memory injection, and cost tracking.
phone-call
Make autonomous phone calls with AI voice using Twilio, Deepgram, and ElevenLabs
paylock
Non-custodial SOL escrow for AI agent deals.
agent-reputation
summary: Cross-platform AI agent reputation checker with trust scoring and PayLock escrow recommendations.
Telecom Agent Skill
Turn your AI Agent into a Telecom Operator. Bulk calling, ChatOps, and Field Monitoring.
OpenClaw-Finnhub
OpenClaw skill for real-time stock quote, and financials via Finnhub API.
```markdown
# OpenClaw-Last.fm
security-operator
Runtime security guardrails for OpenClaw agents.