# Open-Source Real-Time Voice Agent Stack

> A low-cost, open-source stack for building real-time AI voice assistants

This recipe pairs LiveKit's VoicePipelineAgent with Groq-hosted Whisper and Llama 3.3-70B, using self-hosted Kokoro TTS for speech generation. It provides a drop-in, significantly cheaper alternative to commercial voice APIs like ElevenLabs while maintaining low latency.

## How it works

Stack: TTS/voice = Kokoro (open source, OpenAI TTS-compatible endpoint), Chat/LLM = Groq Llama 3.3-70B Versatile, STT = Whisper large-v3 via Groq, orchestration = LiveKit VoicePipelineAgent; coded with Aider + DeepSeek.

Code (from photo): kokoro\_tts = openai.TTS(model="tts-1" (kept for LiveKit compatibility), voice="am\_michael" (voices listed at http://localhost:8880/v1/audio/voices), api\_key="not-needed", base\_url="http://localhost:8880/v1") pointing at a Kokoro FastAPI endpoint. agent = VoicePipelineAgent(vad=ctx.proc.userdata\["vad"\], stt=openai.STT.with\_groq(model="whisper-large-v3"), llm=openai.LLM.with\_groq(model="llama-3.3-70b-versatile"), tts=kokoro\_tts, chat\_ctx=initial\_ctx); agent.start(ctx.room, participant); await agent.say("Hey, how can I help you today?", allow\_interruptions=True). cli.run\_app(WorkerOptions(entrypoint\_fnc=entrypoint, prewarm\_fnc=prewarm)).

## What it is for

- Build low-latency conversational voice bots without expensive proprietary TTS APIs
- Self-host text-to-speech using an OpenAI-compatible Kokoro endpoint
- Orchestrate real-time speech-to-speech pipelines using LiveKit and Groq

## Key points

- LiveKit VoicePipelineAgent orchestration for handling WebRTC audio streaming and turn-taking
- Self-hosted Kokoro TTS exposed via an OpenAI-compatible FastAPI endpoint
- Whisper large-v3 hosted on Groq for ultra-fast speech-to-text
- Llama 3.3-70B Versatile on Groq for conversational reasoning
- Native support for interruption handling via LiveKit voice activity detection (VAD)

## Pricing

Open source. Free and open-source stack; pay only for Groq API usage and local or cloud compute.

## Details

- Type: Tip
- Category: [Development](https://xiry.app/topics/development.md)
- Topic: [Voice AI](https://xiry.app/topics/ai/voice-ai.md)
- Open source: yes
- Platforms: Self-hosted, Linux, macOS, Windows
- For: Developers building real-time voice agents looking to reduce API costs.
- Alternative to: ElevenLabs Conversational AI, Vapi, Retell AI
- Tags: voice ai, livekit, kokoro, groq, tts, stt, llm

## Why it was saved

- [@TheArk_Master, 8 February 2025](https://x.com/TheArk_Master/status/1888342922698863096): Concrete, copyable stack and code for building a voice agent without ElevenLabs.

## Compare

- [Alternatives to Vapi](https://xiry.app/alternatives/vapi.md)
- [Alternatives to Retell AI](https://xiry.app/alternatives/retell-ai.md)

## Related

- [ElevenLabs Conversational AI](https://xiry.app/resources/elevenlabs-conversational-ai): Platform for building and deploying interactive voice AI agents. (Web app, Freemium)
- [LiveKit Agents](https://xiry.app/resources/livekit-agents): Open-source Python framework for building realtime voice and multimodal AI assistants (GitHub repo, Open source)
- [Miso-TTS](https://xiry.app/resources/miso-tts): Open-source emotive text-to-speech foundation model with low conversational latency. (AI model, Open source)
- [OpenHome](https://xiry.app/resources/openhome): Platform and SDK for building custom AI voice agents and smart devices (Web app, Freemium)
- [Synthflow](https://xiry.app/resources/synthflow): No-code platform to build, test, and deploy conversational voice AI agents (Web app, Freemium)
- [Firecrawl](https://xiry.app/resources/firecrawl-1984): Turn websites and documents into clean, LLM-ready markdown or structured data. (Website, Freemium)

Updated 11 October 2026. Page: https://xiry.app/resources/open-source-real-time-voice-agent-stack

Make your own library: Xiry takes your saved reels, tweets, Reddit bookmarks and Watch Later videos, makes them searchable and pulls the useful stuff out, like this library. It's invite-only for now; join the waitlist at https://xiry.app/#waitlist.