chore: keeping documentation up to date #12

Merged

troed merged 56 commits from devel into main 2026-09-04 12:20:58 +02:00

Conversation 0 Commits 56 Files changed 32 +4936 -1459

troed commented 2026-09-04 11:14:09 +02:00

Owner

Copy link

Docs: Speaker ID section gains the device-id / connected-device registry and guided-capture (enroll/verify) bullets; CONFORMANCE.md gains the optional server.capture_control extension.

Includes eeede3f: fixes 9 pre-existing mypy errors and two unformatted files from the capture-mode work so CI passes on the merge.

Docs: Speaker ID section gains the device-id / connected-device registry and guided-capture (enroll/verify) bullets; CONFORMANCE.md gains the optional server.capture_control extension. Includes eeede3f: fixes 9 pre-existing mypy errors and two unformatted files from the capture-mode work so CI passes on the merge.

troed added 54 commits 2026-09-04 11:14:09 +02:00

docs: update demo video to vwTvn6qsVtQGsSW8TJMBDj 77082e83cb

feat: add qwentts CUDA wheel rebuild for Q4_K_M (q6_K) + README ... dd8163c620

The PyPI qwentts-cpp-python 0.3.1 wheel (qwentts.cpp 7df559a) lacks the
ggml-cuda getrows kernel for GGML_TYPE_Q6_K, so Q4_K_M GGUFs abort with
unsupported src0 type: q6_K. Rebuilding against main/a8a7716+ includes the
case. Add scripts/build_qwentts_wheel.sh (clone+build_native+wheel) and
document VRAM table and rebuild steps in README.

fix: build_qwentts_wheel default ref master and tolerant checkout ... 91a59532ba

qwentts.cpp default branch is master, not main. Also shallow clone
already on correct ref, so skip unnecessary checkout that aborted the
build with set -e.

feat: empirical VRAM measurement (static + transient + per concurrent user) ... 06394e2681

- scripts/measure_vram.py polls nvidia-smi at 50ms, records static idle vs
  peak during qt_synthesize for empty/typical/worst scenarios
  (~100 chars vs ~700 chars / 1536 tokens) and concurrent=1 vs 2 pipelines
  (weights shared, scratch per concurrent synth). Outputs per-combo JSON
  + README_table.md + combined JSON to artifacts/vram/.
- README: add Empirical VRAM section with repro command and placeholder
  table for Q8_0/Q4_K_M empty/typical/worst c1/c2 (fill *to fill* after
  headless run). Clarify that idle pipelines don't add VRAM.
- artifacts/vram/.gitkeep to track dir.

fix: measure_vram use CustomVoice + fix concurrent import shadowing 993dbee986

docs: empirical VRAM 2026-08-27 Q8_0 + Q4_K_M empty (headless A2000) ... cc40ceeda2

Q8_0 CustomVoice measured via scripts/measure_vram.py (50ms nvidia-smi):
 empty 11619/0/287, typical c1 11627/11639/12/267, c2 11631/11647/16/259 (+8),
 worst c1 11631/11669/38/237, c2 11641/11679/38/227 (+10) on 12282 total
 with LLM 8256 baseline (server stopped). Q4_K_M empty 10619/0/1287
 measured pre-rebuild; typical/worst estimated from Q8_0 transient
 pending q6_K wheel rebuild for Q4_K_M CustomVoice. Artifacts in
 artifacts/vram/2026-08-27_*.json + combined. Clarify concurrent
 pipelines share weights, only overlap adds ~8-10 MB scratch.

docs: update empirical VRAM with Q4_K_M worst/typical after q6_K wheel rebuild (a8a7716) ... e1e28a7f1f

Q4_K_M now measured on headless A2000 12GB (12282) with qwentts.cpp a8a7716 CUDA wheel (q6_K getrows fix):
- empty 10799/0, typical c1 10813 c2 10821 (+8), worst c1 10843 c2 10851 (+8)
- Q8_0: empty 11641, typical 11655/11663 (+8), worst 11685/11693 (+8)
Saves ~842 MiB peak worst (10851 vs 11693). Fixes huggingface-hub pin <1.0 after rebuild.
Raw JSON + README_table in artifacts/vram/2026-08-27_*.json

docs: crew user-system design (speaker ID + user registry) d3a20feb4e

feat: add speaker field to pipeline messages and events 0599947e2f

feat: add UsersArguments CLI dataclass 97b3aa7eaa

feat: add CrewClient for crew user-system service 11f1e806f2

feat: add UserRegistry with atomic-swap snapshot and three-state tag 45710dbffa

feat: add SpeakerEmbedder wrapping FunASR CAM++ 4f8ca1e423

feat: add speaker tag helpers (GUEST, format_user_turn) 604f4b7946

feat: propagate speaker through STT output and TranscriptionNotifier 34976b2121

fix: keep speaker prefix on revised legacy-chat turns b209949b55

feat: add SpeakerIDHandler (sync and parallel identification) c746cd68db

feat: wire speaker ID singletons and handler into the pipeline c569d38ace

feat: surface speaker in realtime service and add enroll endpoint a0468ff800

fix: keep speaker prefix on revised speculative turns; pin enroll length boundary 64b0d43382

docs: add speaker-ID smoke script, example flags, and ops notes 383ee3c270

fix: relocate commented users_* block out of shell continuation in server.sh.example 46dc9469b5

Merge remote-tracking branch 'origin/main' into devel ... c8ab8c9f7e

# Conflicts:
#	README.md

fix: self-heal user registry after initial crew outage c55f168cab

docs: speaker enroll enablement design 9631afb4ce

docs: speaker enroll enablement plan 68e559deae

docs: fix Task 1 venv sync for paraformer extra 563ae97763

docs: add crew API token lockdown to plan 1467461413

docs: fix token interpolation in ssh probe commands b1765ef2a9

fix: pin rebuilt qwentts-cpp-python wheel with Q6_K CUDA kernel ... d31bbe4416

PyPI qwentts-cpp-python 0.3.1 (qwentts.cpp 7df559a) is missing the CUDA getrows kernel for K-quants, so any Q4_K_M GGUF TTS attempt SIGABRTs and takes down the whole server (silent ESP32 after the 2026-09-03 deploy).

- scripts/build_qwentts_wheel.sh: default arch 86-real (deployed GPU), uv fallback when python3 lacks the build module, copy the wheel into gitignored wheels/ and install into .venv
- pyproject.toml: qwentts-cpp-python==0.3.1 as a direct dep + [tool.uv.sources] path pin to wheels/ (uv source overrides are not applied to transitive deps)
- .gitignore: wheels/
- README.md: build + pinning docs

fix: start speaker join budget at resolve, run CAM++ on CUDA ... 154f9b3adc

resolve_speaker() previously waited on a deadline set at VAD-final spawn,
but it is called after STT finishes (60-250 ms later), so the budget was
already spent and tags were always dropped. The join now gets the full
join_budget_ms from the point the resolution is requested.

SpeakerEmbedder now uses CUDA when available (warm ~60 ms / 62 MiB VRAM
vs 68-878 ms on CPU) so identification finishes inside the join budget.
The voice system prompt now explains the [name] transcript prefix so the
LLM knows who is speaking. README documents the VRAM cost.

fix: propagate speaker tag through native-llm STT handler ... 511d8bbb85

In sync mode (--stt native-llm), SpeakerIDHandler sets VADAudio.speaker
inline and registers no pending entry, but NativeLLMSTTHandler built the
Transcription without speaker=, dropping the tag before
TranscriptionNotifier. The resolver then correctly returned None (nothing
pending in sync mode), so the LLM never saw the [name] prefix.

Now Transcription carries speaker=vad_audio.speaker. Regression test added.

fix: patch qwentts binding for ABI v4 to stop TTS SIGSEGV crash loop ... 7b3992d73b

The rebuilt wheel pairs the native lib (ServeurpersoCom/qwentts.cpp
a8a7716+, ABI v4) with the stale upstream binding (andimarafioti/
qwentts-cpp-python 0.3.1, ABI v2). qt_init_default_params() writes
codec_chunk_sec 4 bytes past the undersized QtInitParams ctypes struct,
corrupting the heap and causing random SIGSEGVs during TTS model load.
qt_tts_params also carries two fields the new lib dropped.

build_qwentts_wheel.sh now applies an idempotent patch after the binding
checkout: bump QT_ABI_VERSION to 4, add max_batch/codec_chunk_sec to
QtInitParams, and remove the stale codec_chunk_sec/codec_left_context_sec
from QtTTSParams and its two write sites. A clean QWENTTS_PY_SOURCE
checkout reproduces the crash without this patch; the rebuilt wheel and
both venvs (server + local) are updated and verified (836 tests pass,
server stays up with max_batch=1 loading cleanly).

debug: log speaker match score at identify time 3c7c64c9e4

debug: dump embedded PCM to /tmp/speaker_debug.wav for speaker-score analysis 2908f15d67

docs: device-based speaker enrollment design spec 3c32cc44a6

docs: device enrollment phase 1 implementation plan a039b3cf68

feat: connected-device registry for speaker enrollment b047d798d2

feat: register connected devices in realtime handshake 8226cc733d

feat: guided mic-capture endpoint for device enrollment 26fea01af7

fix: capture divert before service buffer, single capture source of truth, 409 atomicity note 99ab341911

feat: PC client device-id and capture-mode control handling 7a57584a9a

fix: capture runtime test coverage, blank device_id regen f21e5a9d1f

fix: pause wake-word detector while server capture is active 81cb1b1a6b

debug: temporary capture diagnostics (vad type, chunk counts) bd1d22751b

debug: fix diagnostics indentation d61658e857

revert: remove temporary capture diagnostics fb8bb5f355

debug: temporary client capture sent-chunks counter b32ff03f81

fix: bootstrap mic sender on capture-start pre-wake 7a598fdc3d

debug: temporary capture drain counter 75326416dc

Revert "debug: temporary capture drain counter" ... c1da6f1cfd

This reverts commit 75326416dc.

Revert "debug: temporary client capture sent-chunks counter" ... 09335f2a41

This reverts commit b32ff03f81.

fix: mypy and ruff-format on capture-mode client and router eeede3f94a

chore: keeping documentation up to date 0ea279cb3a

troed added 1 commit 2026-09-04 11:47:48 +02:00

Merge remote-tracking branch 'origin/main' into devel ... 8c191b30fe

# Conflicts:
#	README.md
#	src/computer/api/openai_realtime/server.py
#	src/computer/api/openai_realtime/websocket_router.py
#	src/computer/s2s_pipeline.py
#	src/computer/users/speaker_id_handler.py
#	tests/users/test_speaker_id_handler.py

troed added 1 commit 2026-09-04 12:10:54 +02:00

fix: resolve qwentts-cpp-python from PyPI in CI when the local wheel is absent ... 8cc8fad3ce

The [tool.uv.sources] pin points qwentts-cpp-python at a gitignored
local CUDA wheel (wheels/, built by scripts/build_qwentts_wheel.sh).
CI checkouts don't have that wheel, so uv sync failed with 'No such
file or directory'. Drop the pin in CI when wheels/ is empty and let
uv resolve the package from PyPI, restoring the pre-pin resolution.

troed merged commit df0de6a4bf into main 2026-09-04 12:20:58 +02:00

troed referenced this pull request from a commit 2026-09-04 12:21:00 +02:00

Merge pull request 'chore: keeping documentation up to date' (#12) from devel into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

starfleet/computer!12

No description provided.