# Hibiki-Zero

> Open-source real-time multilingual speech-to-speech translation model with voice preservation

Hibiki-Zero is a 3B-parameter open-source model developed by Kyutai for real-time speech-to-speech translation. It translates spoken French, Spanish, Portuguese, and German into English while preserving the original speaker's voice. Built using reinforcement learning to optimize the trade-off between latency and translation accuracy, it can run locally on modern consumer GPUs.

Website: https://github.com/kyutai-labs/hibiki-zero

## What it is for

- Translate spoken foreign languages to English in live real-time conversations
- Translate recorded multilingual audio files in batch mode
- Preserve speaker vocal characteristics and tone during cross-language translation
- Host a local speech-to-speech translation server on consumer GPU hardware

## Features

- Real-time speech-to-speech translation into English
- Supports input in French, Spanish, Portuguese, and German
- Voice transfer capability that retains the original speaker's timbre and tone
- Trained via reinforcement learning for word-level alignment and low latency
- Batch inference support for pre-recorded audio files
- Built-in local real-time server with optional Gradio tunnel forwarding
- Runs on NVIDIA GPUs with 8 GB to 12 GB VRAM

## Pricing

Open source. Free and open source (MIT license)

## Details

- Type: AI model
- Category: [AI](https://xiry.app/topics/ai.md)
- Open source: yes
- Platforms: Linux, Command line, Self-hosted, Code library
- For: AI researchers, audio engineers, and developers building live multilingual voice applications
- Alternative to: SeamlessM4T, Whisper
- Tags: speech to speech, translation, audio, voice cloning, kyutai, open source

## Why it was saved

- [@kyutai_labs, 12 February 2026](https://x.com/kyutai_labs/status/2022007408898511113): Real-time speech translation into English from French, Spanish, Portuguese and German, with voice transfer (keeps the speaker's own voice), low latency and high audio quality. Built on the original Hibiki architecture but trained with reinforcement learning instead of complex aligned synthetic data heuristics: only sentence-level alignment is required, word-level alignment is learned via RL, which simplifies data creation, lowers latency and scales to more languages. Reward is based on BLEU of intermediate translations with a parameter trading latency vs accuracy. Weights and reference code are open source.

## Related

- [Chatterbox](https://xiry.app/resources/chatterbox): Open-source zero-shot text-to-speech and voice cloning model (AI model, Open source)
- [Coqui TTS](https://xiry.app/resources/coqui-tts): Open-source deep learning toolkit for text-to-speech synthesis and voice cloning. (GitHub repo, Open source)
- [Pocket TTS](https://xiry.app/resources/pocket-tts): Lightweight, CPU-friendly text-to-speech model with fast voice cloning. (AI model, Open source)
- [ElevenLabs](https://xiry.app/resources/elevenlabs): AI voice generator, voice cloning, and conversational agent platform (Web app, Freemium)
- [HeyGen](https://xiry.app/resources/heygen): AI avatar video generation and translation platform driven by text prompts (Web app, Freemium)
- [PersonaPlex-7B (nvidia/personaplex-7b-v1)](https://xiry.app/resources/personaplex-7b-nvidia-personaplex-7b-v1): Open full-duplex speech-to-speech conversational AI model by NVIDIA (AI model, Open source)

Updated 10 October 2026. Page: https://xiry.app/resources/hibiki-zero

Make your own library: Xiry takes your saved reels, tweets, Reddit bookmarks and Watch Later videos, makes them searchable and pulls the useful stuff out, like this library. It's invite-only for now; join the waitlist at https://xiry.app/#waitlist.