Release 0.3.0

troed commented

2026-09-07 22:32:43 +02:00

Owner

Release 0.3.0: OmniVoice TTS backend (quantization + FlashInfer options), live-script test fixes.

troed added 24 commits

2026-09-07 22:32:44 +02:00

chore(hygiene): replace internal hostnames and paths with placeholders 57350c5e3a

feat: add parakeet-onnx STT handler (int8 ONNX Runtime) 914716fbf1

New --stt parakeet-onnx choice runs parakeet-tdt-0.6b-v3 via onnx-asr
with a bounded CUDA arena (kSameAsRequested + gpu_mem_limit, default
1.5 GiB), sized to coexist with a local LLM server on a shared GPU.
Final-turn-only transcription; language detection reuses the parakeet
lingua detector. Installed via the new 'parakeet-onnx' extra.

fix: accept injected gen_kwargs in ParakeetOnnxSTTHandler.setup cd8a3027a0

rename_args() adds gen_kwargs to every parsed argument dataclass, so
STT handler setup signatures must tolerate it (as parakeet-tdt does).

fix: default parakeet-onnx model to the -onnx HF repo 86820e51d2

istupakov/parakeet-tdt-0.6b-v3-onnx is the repo that hosts the int8
ONNX weights (and the one cached on the server); the non-onnx repo
401s on config.json.

fix: use onnx-asr built-in nemo-parakeet-tdt-0.6b-v3 model name 663fcfccd3

The built-in registry entry resolves to the istupakov int8 ONNX files
that are already cached on the server; passing the raw HF repo name
made onnx-asr fall back to config.json fetching from a different repo.

fix: bound LLM turns, warm real prompt, make STT turn ids session-safe 78af59026e

Three production failures addressed:
- gemma runaway generation (47k tokens/170s) silently consumed by the
  sentence-batching consumer: cap output tokens (max_output_tokens,
  default 512, 0 disables), raise FirstOutputTimeout when a stream
  yields nothing within first_output_timeout_s (default 15s) and emit
  the spoken apology, and force-flush audio text that exceeds 2000
  chars without a sentence boundary.
- toy warmup never warmed the real prompt (first question paid 7.5s
  prefill): warmup now sends the actual init_chat_prompt with
  max_tokens=1 so llama prefix-cache holds it; warmup failure no
  longer aborts setup.
- reconnect after a session wedged the pipeline: STT turn ids restart
  at turn_1 while BaseSTTHandler persists completed (turn, revision)
  keys, so the next session's first turn was dropped as stale.
  VAD turn ids are now monotonic across sessions, dropped FINAL inputs
  always log at INFO, on_session_end logs cleared key counts, and the
  parakeet-onnx handler no longer shadows the base on_session_end.

docs: document TTS-language guard flags in server.sh example 7c92684ed3

The per-turn language constraint (46219e2) only activates with
--enable_lang_prompt + --tts_supported_languages + --tts_default_language;
the example never showed them, so deployments could silently ship
without it (as the live server.sh had).

fix: put TTS language instruction in system prompt; stop recording user turns as echoes 7e7960ee5b

- The 'reply in <lang>' instruction was appended as a trailing user-role
  message after the actual question; gemma then treated it as the final
  instruction and answered conversationally instead of calling tools.
  It now rides in the system prompt via the instructions text.
- TranscriptionNotifier recorded every user transcript into the echo
  filter history, so repeating a question verbatim was discarded as an
  echo of itself (turn_3 live: repeated weather question dropped). The
  echo history stays assistant-output only (lm_output_processor).

fix: embed reply-language note in last user message; one status chunk per turn 9a171ca1fa

gemma-4-12b ignored the system-prompt language stamp once Swedish tool
output dominated the context (probes A-D); embedding '(Reply in X.)'
in the same user message as the question survives it and does not
derail tool calls. Multiple server-side tools in one turn now emit a
single 'Working.' status chunk instead of one per tool.

fix: re-embed reply-language note on every tool round afaeca7d86

_generate re-serializes the chat for each server-side tool iteration,
but _serialize consumed the note after the first round — the round
that actually answers (after Swedish tool output) lost it. Keep the
note on the handler; process() resets it per turn.

docs: document parakeet-onnx VRAM profile and usage

CI / Sanity check (ubuntu-latest) (pull_request) Failing after 2m50s

Details

eaf6ced819

fix: auto-reconnect client after server restart instead of exiting

CI / Sanity check (ubuntu-latest) (pull_request) Failing after 2m13s

Details

f9bf39670a

feat: crew client session-prompt and device-owner fetches deb462b967

feat: polling crew device-owner cache fd4a24d7e1

feat: session-prompt TTL cache and device prompt provider c81b9fa49c

feat: per-speaker session prompt swap in realtime service fbe6b973e1

feat: forward device id into realtime session register 00d4edecd8

feat: wire crew prompt provider into realtime pipeline 491994be43

docs: per-user session prompt flags and behavior 25309151da

fix: guard prompt swaps against stale and unregistered sessions d7827abcd1

style: format pre-existing ruff deviations c15ba2fec9

feat: add OmniVoice TTS backend with quantization and FlashInfer options (#15 ) 1a376270a2

Reviewed-on: ⁨#15⁩

test: skip live script tests cleanly without a live host 2214418056

chore: release 0.3.0

CI / Sanity check (ubuntu-latest) (pull_request) Failing after 2m16s

Details

8ca8ab2e81

troed referenced this pull request from a commit

2026-09-07 22:38:47 +02:00

Release 0.3.0 (merges PR #16)

troed commented

2026-09-07 22:40:13 +02:00

Author

Owner

Content merged to main via direct push (9c5c5d4) while the Forgejo merge-checker was returning 405 Please-try-again-later; actual git conflicts (parakeet/client fixes on main vs release branch) were resolved in merge commit 9c5c5d4. Closing this PR as superseded.

Content merged to main via direct push (9c5c5d4) while the Forgejo merge-checker was returning 405 Please-try-again-later; actual git conflicts (parakeet/client fixes on main vs release branch) were resolved in merge commit 9c5c5d4. Closing this PR as superseded.

troed closed this pull request

2026-09-07 22:40:13 +02:00

CI / Sanity check (ubuntu-latest) (pull_request) Failing after 2m16s

Details

Pull request closed

Please reopen this pull request to perform a merge.

Rows
Columns

Release 0.3.0 #16

Pull request closed