Voice MCP Client

Created By

baryhuang8 months ago

A iOS/MacOS Swift MCP Client using voice interacting with python MCP servers both natively

# python

# ios-app

Overview Content Tools Comments

Content

Voice MCP Client

A multimodal voice agent platform using LiveKit, supporting macOS, iOS, and Python server-side components.

Overview

This project implements a voice agent platform with these key components:

Python Server (server_mcp_client): Backend server leveraging LiveKit Agents Framework with support for Anthropic Claude via API/AWS Bedrock
macOS Client (voice_macos_client): Native application for voice interactions and Apple ecosystem integration
iOS Client (voice_ios_client): Mobile application providing the same voice agent capabilities on iOS devices

Value Proposition

Seamless Voice Interaction: Enable natural conversations with an AI assistant across all your Apple devices
Apple Ecosystem Integration: Access and control Apple Notes, Calendar, Messages, and more through voice commands
Multi-Platform Support: Use the same voice assistant experience on both desktop and mobile
Privacy-Focused: Process sensitive data locally while leveraging cloud AI capabilities
Extensible Architecture: Add custom commands and integrations through the MCP (Machine Control Protocol) framework

Key Features

Real-time voice interactions with AI agent powered by Anthropic Claude
Deep integration with Apple ecosystem applications and services
Cross-platform support for macOS and iOS
LiveKit-powered real-time audio streaming with high quality and low latency
Configurable server components with Docker support for easy deployment
Extensible MCP framework for adding custom capabilities

Getting Started

Each component has its own detailed setup instructions:

For server setup, see the README in the server_mcp_client directory
For macOS client setup, see the README in the voice_macos_client directory
For iOS client setup, see the README in the voice_ios_client directory

Architecture

The system uses LiveKit for real-time audio streaming between clients and the server. The server processes audio input, communicates with the Anthropic Claude AI service, and sends responses back to the clients. The MCP framework allows the AI to access and control various Apple ecosystem functionalities when requested by the user.