In development: phases 0 to 4 built, phase 5 in progress

Speech for iOS, Android, and Meta Quest.

One Rust core for speaking, listening, and conversation. The work runs on the phone, on a computer nearby, or in the cloud, and your app never has to know which.

Profiles, not engines. You pick a voice with a job, like a calm narrator, and VoiceLib picks the engine on each tier.

On the device first. Three TTS and two STT engines run in the core through ONNX Runtime and whisper.cpp, chosen per phone by a probe that measures them.

Settings you embed. SwiftUI and Compose pieces drop into your own settings screen, built from the platform's controls.

let voice = VoiceModel(
    appId: "com.example.calm",
    profiles: [.wren])          // a calm narrator

// in your settings screen
Form { VoiceSettingsSection(model: voice) }

let session = try await voice.client
    .session(consumer: "guide", profile: .wren)
try await session.start()
session.speak(SpeechChunk("Let your eyes close.",
                         pauseAfter: .seconds(2)))
session.ramp(to: .slow, over: .seconds(90))

val voice = VoiceModel(context,
    profiles = listOf(VoiceProfile.WREN))

// in your settings screen
Column { VoiceSettings(voice) }

val session = voice.client
    .session("guide", VoiceProfile.WREN)
session.start()
session.speak(SpeechChunk("Let your eyes close.",
                         pauseAfter = 2.seconds))
session.ramp(Pace.SLOW, 90.seconds)

Five roles, one stock voice for each

A role is the behavior your app needs. It fixes the audio mode, who writes the words, how deep the buffers run, and what happens when a tier fails mid-session. A profile plays one role and carries its engines per tier; the stock profiles are named like actors so you can swap one for another.

narration

wren, cove

Long-form app text with deep lookahead, pacing you drive with ramps or a callback, and gapless seams.

prompt

atlas

Short lines with the fastest first audio, for cues and announcements.

dictation

sage

Partial and final transcripts with Silero VAD endpointing.

conversation

juniper

Your turn handler answers each final transcript; the user can talk over the reply.

persona

ember

A speech-to-speech model writes its own words from a character prompt.

Four tiers, one ladder

Each role has a default order across the tiers. At session start VoiceLib takes the first rung whose models are downloaded, whose device test passed, and, for a nearby computer, that answers within half a second. Users can turn tiers off; your policy decides which exist.

Nearby computervoicelib-relayd on Windows, Linux, or macOS: larger engines over an encrypted QUIC link, paired once by code.phase 5
On this devicePocket TTS, Kokoro, Kitten TTS, Moonshine, and Whisper in the core, on the CPU or Core ML or NNAPI where the probe approves.built
CloudElevenLabs, Cartesia, OpenAI, Deepgram, Gemini Live, and Grok Voice, with app-managed keys through your own Worker.phase 8
System voiceAVSpeechSynthesizer and SpeechAnalyzer on iOS, TextToSpeech and SpeechRecognizer on Android, through the same queue.built

Engines

Licenses are a per-app choice. An engine's weights terms apply only to apps whose profiles list it, and copyleft code sits behind build features that are off by default and produce separately named artifacts.

Engine	Does	Runs	License	Status
Pocket TTS	Speaks, streaming, seven languages	Phone, computer	Code MIT, weights CC-BY-4.0	built
Kokoro-82M	Speaks, English-first	Phone, computer	Apache 2.0	built
Kitten TTS	Speaks, tiny and fast	Phone	Apache 2.0	built
Moonshine Streaming	Listens, streaming, English	Phone, computer	MIT	built
Whisper	Listens, about 99 languages	Phone, computer	MIT	built
System	Speaks and listens	Phone	OS	built
Kyutai TTS and STT	Streaming pair with end-of-turn detection	Computer, CUDA	CC-BY-4.0 (STT)	planned
Nemotron ASR, Magpie TTS	The NVIDIA pair, 40 languages in	Computer	NVIDIA Open Model License, OpenMDW-1.1	planned
PersonaPlex	Speech-to-speech persona	Computer, Linux	NVIDIA Open Model License	planned

Measured on phones

Every number below comes from a phase exit test on an iPhone 16 Pro (iOS 26.7.1) or a Samsung Galaxy SM-S731U1 (Exynos 2400, Android 17), with the configuration the device probe chose.

56 msPocket TTS to first audio, iPhone, real-time factor 0.27

21 msMoonshine's final transcript after the endpoint, iPhone (Galaxy: 25 ms)

65 msWhisper tiny.en's final through its Core ML encoder, iPhone

80 msFrom the user's first word to barge-in, with the output faded 43 ms later

Measure	iPhone 16 Pro	Galaxy SM-S731U1
Kokoro FP32, first audio	588 ms	not in the exit test
Kitten TTS nano, first audio	174 ms on Core ML	not in the exit test
Moonshine tiny, real-time factor	0.08	0.07
Whisper tiny.en, final after the endpoint	65 ms	473 ms
Conversation, barge-in after the user's first word	80 ms (system), 389 ms (VoiceLib canceller)	at the first word (system)
Conversation, false barge-ins from its own voice	0 in all three echo modes	0 in all three echo modes

Barge-in that trusts the room

A conversation listens while it speaks. When the user talks over it, the output fades within 100 ms, the queue drops, and a reply your handler is still writing is discarded. Three echo cancellation modes keep the session's own voice out of its ear: the platform's voice processing, VoiceLib's own canceller in the core, or a microphone muted while it speaks. The core canceller holds barge-in back until it has learned the room, so it never interrupts itself.

A computer nearby

Install voicelib-relayd on a PC or Mac on the same network and run voicelib-relayd pair. The phone finds it, pins its certificate from the code, and streams Opus over QUIC both ways; the turn handler stays on the phone. A Quest headset gets its strongest tier this way.

voicelib-relayd serve
voicelib-relayd pair     # a QR code and an eight-letter code
voicelib-relayd status

Roadmap

0
Core and native narrationRing buffers, the C audio ABI, both audio layers, profiles, pacing, the settings section.
1
Pocket TTS on deviceThe model store and mirror, the device probe, the readiness card, engine comparison.
2
Kokoro and Kitten TTSA Rust port of the misaki phonemizer, the Android chip allowlist.
3
DictationNative recognition, Whisper, Moonshine Streaming, Silero VAD, the Microphone row.
4
ConversationBarge-in, three echo cancellation modes, the Output row.
5
Nearby computerThe daemon, discovery and pairing, the QUIC link, host engines, installers.
6
Meta QuestCompose panels, a CPU budget under a rendering scene, spatial voices.
7
PersonasMonologue providers, PersonaPlex, the Characters row.
8
CloudHosted TTS, STT, and speech-to-speech with app-managed keys.