Cascade voice consistency: before and after
A streaming voice cascade (speech recognition, a language model, then Qwen3-TTS) speaks every sentence of every reply as its own TTS request, and each request is sampled with a fresh seed. With a built-in CustomVoice speaker, the only thing tying the requests together is a speaker token, so the voice moves from sentence to sentence and from turn to turn.
The fix clones one 5.4 s recording of the same speaker with Qwen3-TTS Base for every request, from the recording's speaker embedding. Each file below plays the same six replies (three sentences each) back to back, with 0.8 s between replies.
Anchor recording
CustomVoice speaker Vivian, seed 1234. Every request in the "after" files clones this clip.
1. Before: CustomVoice, one request per sentence
What the cascade did. Listen for the timbre changing between replies.
2. After: Base cloning the anchor's speaker embedding, one request per sentence
The change. Requests are still independent; each one is conditioned on the same recording.
3. Not used: Base cloning the anchor with its transcript
Closer to the recording, but every sentence opens with a pause of about 0.4 s, which makes each reply start later.
4. Next step: one generation per reply
Each reply spoken in one request (cloned with the transcript here), so the sentences of a reply also share prosody.
5. Rejected: clone the previous sentence instead of the anchor
Smooth from one sentence to the next, but the voice drifts away over a session.
Numbers
Full-Duplex-Bench v1.5, 100 sessions and about 300 replies per column, the Qwen3-4B cascade with only the speech model changed. Voice consistency: speaker similarity (WavLM-large ECAPA cosine x 100, middle 3 s) of each reply against the mean of all other replies. Response latency: from the end of the user's turn to the reply's first word.
| Metric | Before | With transcript | After |
|---|---|---|---|
| Voice consistency, mean / p10 | 67.3 / 57.9 | 79.5 / 74.8 | 76.8 / 70.6 |
| UTMOS, mean / p10 | 4.20 / 3.70 | 4.53 / 4.49 | 4.52 / 4.49 |
| Word error rate | 4.1% | 2.4% | 2.7% |
| Response latency | 1.06 s | 1.42 s | 1.01 s |
| Responds when interrupted | 96% | 96% | 96% |
In a controlled run (18 three-sentence replies per condition), the same consistency measure goes from 67.8 to 81.6 with the speaker embedding and 82.2 with the transcript, and the opening silence of a sentence is 0.04 s against 0.38 s.
Clips, scores and scripts: gavinlaw/sglang-omni-fdb-native-runs.