BEFORE AUDIO
Retry with a reset.
A real primary-process kill is followed by a completed turn through the secondary. Token previews are explicitly reset before retry.
GO GATEWAY / REAL OPEN MODELS / REACT CONSOLE
Every stage
has a deadline.
A voice inference system that knows when to retry, when to stop speaking, and how to show the difference.
Recorded evidenceThis Vercel site presents saved CPU-model runs and the console
recording. The interactive model services are not hosted
here.
pulse / voice systemRECORDED SESSION
p
CLIENT FIRST AUDIO / P998-token cap · concurrency 4 · saved CPU run
VADASRLLMTTS
Audio arrival, before speaker playback.
01 / MEASURE THE WHOLE PATH
Raw event traces ↗Text → LLM → TTS through the Go WebSocket gateway. First audio is measured when the client receives a PCM frame, before device playback. Local CPU results; these are not Vercel capacity measurements.
Latency under concurrency
| Concurrency | Samples | First token p99 | First audio p50 | First audio p99 |
|---|
Small development samples, with interpolated percentiles. A smaller output cap reduces answer length and model/speech work. It does not establish a speedup at equal answer quality.
The failure boundary
AFTER AUDIO
Fail without repetition.
Once speech is visible, retry could repeat it. The gateway ends that turn instead of silently replaying audible content.
ON INTERRUPTION
Fence the old generation.
A gateway mutex guards generation checks and writes. The browser stops scheduled audio; old worker frames cannot cross the cancellation acknowledgement.
02 / FOLLOW THE TURN
Speech, tokens, audio. Then interrupt.
The actual authenticated console processes public speech, shows stage timings, stops playback, completes a replacement turn, and reloads persisted timing history.
03 / THE SYSTEM, WITH ITS LIMITS
Go gateway→Energy VAD→Whisper→SmolLM2→Piper
Bounded native work
Cancellation drops output and cancels transport. An ongoing native operation can still finish; its admission slot remains occupied until it returns.
Durable timing history
A flushed, single-process journal records IDs, outcomes and stage-boundary timings. It retains no prompt, transcript or audio.
Actual CPU inference
Transformers and open speech models run locally. vLLM/GPU throughput, continuous microphone capture and container restore remain unfinished.