Okareo

Agent simulation and evaluation for voice and text agents — run dozens of synthetic conversations against your agent from inside Claude Code, Cursor, or any MCP-ready editor.

The Okareo MCP server turns your AI coding assistant into an evaluation and testing workbench. Instead of writing SDK scripts or switching to a dashboard, you describe what you want in natural language and your copilot does the rest: build synthetic users from your codebase, run multi-turn simulations against your agent, and pull back transcripts, traces, and scores for analysis.

Okareo is hosted — there's nothing to install and nothing to run locally. Point your editor at the endpoint, sign in once through your browser, and the tools appear.

https://tools.okareo.com/mcp

What you can do from a prompt

• Generate Drivers from your code. Okareo reads your agent's system prompt, tools, and policy logic and proposes synthetic users (Drivers) that stress the parts most likely to break — not generic personas.
• Run voice and text simulations in parallel. Spin up dozens of synthetic callers across languages, run them against your target, and get results back in moments.
• Replay production failures as tests. Pull a failed call from a connected voice integration, turn it into a reusable Driver, and add it to your suite.
• Evaluate and re-score. Run judge, symbolic, and audio-based checks; re-score a past run against a new check without re-running the model.
• Query analytics. Trend pass rates across models and pin dashboards.

Why Okareo

• Voice depth. Synthetic callers across many languages, with the noise, crosstalk, and clipping real customers call from.
• One timeline, three views. Simulation transcript, observability trace, and audio land together — so Conversation Designers and Engineers debug the same call.
• Agent-first, multi-turn-first. The unit of evaluation is a full conversation, not a single prompt/response pair.

Supported editors

Claude Code · Claude Desktop · Cursor · VS Code 1.101+ · Cline · Gemini Code Assist · GitHub Copilot · Windsurf — and any client that speaks MCP's streamable-HTTP transport.

Connect

Claude Code (one line):

claude mcp add --transport http okareo https://tools.okareo.com/mcp

Any MCP client (config):

{
  "mcpServers": {
    "okareo": {
      "type": "http",
      "url": "https://tools.okareo.com/mcp"
    }
  }
}

Authentication

OAuth on first connect — a browser tab opens to app.okareo.com, you sign in and consent, and your editor stores the token. If your client doesn't yet implement MCP OAuth discovery, set OKAREO_API_KEY in your environment and use the Bearer-header fallback (see the docs).

Example prompts

• "Run my refund scenarios against the voice agent in Spanish and Portuguese. Show me what regressed."
• "Pull this week's calls where the language switch failed and turn them into Drivers."
• "Re-score last night's evaluation against my new pii_leak check without re-running the model."
• "Read my agent in src/ and create 12 Drivers that stress the policy boundaries, then run them."

Tools

• Scenarios: save_scenario, list_scenarios, get_scenario, create_scenario_version
• Drivers: create_or_update_driver, list_driver_voices, get_driver, list_drivers
• Targets: create_or_update_target, get_target, list_targets
• Simulations: run_simulation, list_simulations
• Tests & results: run_test, list_test_runs, get_test_run_results, get_conversation_transcript, reevaluate_test_run
• Checks: create_or_update_check, generate_check
• Voice: ingest_conversations, connect_voice_integration, get_voice_webhook_url
• Models & analytics: list_available_llms, register_generation_model, query_analytics, save_dashboard

Links

• Docs: https://docs.okareo.com/mcp/introduction
• Configuration: https://docs.okareo.com/mcp/configuration
• Website: https://okareo.com/mcp
• GitHub: https://github.com/okareo-ai/okareo-mcp
• Python SDK: https://pypi.org/project/okareo/
• TypeScript SDK: https://www.npmjs.com/package/okareo-ts-sdk