docs: NemotronLabs VoiceChat-11B investigation #6

Merged

troed merged 6 commits from devel into main

2026-08-26 15:53:11 +02:00

troed commented

2026-08-26 15:51:21 +02:00

Owner

Documents the 2026-08-26 investigation into NVIDIA NemotronLabs VoiceChat-11B as a candidate end-to-end speech-to-speech backend for the assistant.

Covers: the model, why our upstream (huggingface/speech-to-speech) does not support it, the community GGUF quantization (Q4_0, ~6.2 GiB, fits the A2000), the sansamour/llama-voicechat.cpp fork's stdio JSON-lines serve protocol, how it would map onto our s2s_pipeline.py as a single handler replacing STT+LLM+TTS, and the caveats (no audio streaming, English-only, Linux build required, per-session prompt re-decode).

Verdict: feasible prototype (~2-3 days) but not a primary backend candidate; viable as an experiment only.

Documents the 2026-08-26 investigation into NVIDIA NemotronLabs VoiceChat-11B as a candidate end-to-end speech-to-speech backend for the assistant. Covers: the model, why our upstream (`huggingface/speech-to-speech`) does not support it, the community GGUF quantization (Q4_0, ~6.2 GiB, fits the A2000), the `sansamour/llama-voicechat.cpp` fork's stdio JSON-lines serve protocol, how it would map onto our `s2s_pipeline.py` as a single handler replacing STT+LLM+TTS, and the caveats (no audio streaming, English-only, Linux build required, per-session prompt re-decode). Verdict: feasible prototype (~2-3 days) but not a primary backend candidate; viable as an experiment only.

troed added 6 commits

2026-08-26 15:51:21 +02:00

docs: add Nari Labs TTS latency techniques to backlog 4a7fcf7cde

docs: add s1-mini ASR post-processing idea to backlog 5fe620d98b

chore(hygiene): remove internal network topology from tracked files (#4 ) 4e7daea732

Follow-up to the public-repo security audit: tracked files carried real internal network details (LAN IP, hostnames, username, home paths, SSH port). This removes them; real values stay in gitignored local files and the deploy environment, mirroring the policy already used in starfleet/roster.

Changes:
- `AGENTS.md`: added a **Public repo hygiene** section (no internal hostnames/IPs/usernames/home paths/credentials in tracked files; pre-push scan command). Sanitized the deploy/fj sections that documented the live server's address, user, repo path, and SSH commands.
- `deploy.sh`: `SERVER_HOST`, `SERVER_USER`, `SERVER_REPO_DIR` are now **required env vars** (were hardcoded LAN defaults); setup comments use placeholders.
- `client.sh.example` / `server.sh.example` / `mcp.json.example`: placeholder hosts (`<server-host>`, `<llm-host>`, `<home-assistant-host>`).
- `README.md` + `src/computer/client/__init__.py`: example IPs replaced with placeholders.

Deploy note: run deploys with `SERVER_HOST=... SERVER_USER=... SERVER_REPO_DIR=... ./deploy.sh` (export them in your shell or a gitignored env file).

Verification: `bash -n deploy.sh` OK; edited Python module compiles; hygiene scan (`git grep -nIEi ...`) clean over tracked tree.
Co-authored-by: Troed Sångberg <github@troed.se>
Reviewed-on: #4

feat: switch response language per turn to what TTS can speak 81b7126a8e

docs: note wake word model training is complete

CI / Sanity check (ubuntu-latest) (pull_request) Failing after 3m33s

Details

8a67ed25fd

docs: add NemotronLabs VoiceChat-11B investigation

CI / Sanity check (ubuntu-latest) (pull_request) Failing after 3m19s

Details

d3b4b67576

troed merged commit 45646967ec into main

2026-08-26 15:53:11 +02:00

troed referenced this pull request from a commit