Bring devel into main: echo gate, per-turn timing, CI and logo work #2

Merged

troed merged 30 commits from devel into main 2026-08-07 11:19:25 +02:00

Conversation 0 Commits 30 Files changed 27 +1713 -58

troed commented 2026-08-07 11:06:22 +02:00

Owner

Copy link

Syncs the 42 commits on devel that have accumulated since the last merge to main.

Highlights:

• Echo gate / playback-ack: client output_audio.played ack, echo gate held until playback drain, realtime-paced echo reference feed (audio feedback prevention)
• Per-turn latency timing: turn-timing tracker, queue get/put boundaries, TTS first-audio latency, finalization on response done / barge-in cancel
• Native-llm STT fixes: PCM wrapped in WAV before base64, empty-transcript-with-audio handling, singleton loading guard
• CI hardening: uv bootstrap via curl, libportaudio2, mypy/lint fixes, single sanity job
• Deploy workflow: deploy.sh + server deploy docs, ssh background-start fix
• Branding: new purple app logo (SVG source + 1024x1024 PNG) in the README, docs/example-script alignment with renamed computer binaries

Syncs the 42 commits on `devel` that have accumulated since the last merge to `main`. Highlights: - **Echo gate / playback-ack**: client `output_audio.played` ack, echo gate held until playback drain, realtime-paced echo reference feed (audio feedback prevention) - **Per-turn latency timing**: turn-timing tracker, queue get/put boundaries, TTS first-audio latency, finalization on response done / barge-in cancel - **Native-llm STT fixes**: PCM wrapped in WAV before base64, empty-transcript-with-audio handling, singleton loading guard - **CI hardening**: uv bootstrap via curl, libportaudio2, mypy/lint fixes, single sanity job - **Deploy workflow**: `deploy.sh` + server deploy docs, ssh background-start fix - **Branding**: new purple app logo (SVG source + 1024x1024 PNG) in the README, docs/example-script alignment with renamed `computer` binaries

troed added 42 commits 2026-08-07 11:06:22 +02:00

Fix NativeLLMSTTHandler.setup() rejecting unexpected kwargs from faster-whisper args dict 401e6dd9c2

Fix native-llm: add user message to chat when transcript is empty but audio present ... b956287747

When --stt native-llm is used, transcript is empty but audio_bytes are present.
The _on_transcription_completed condition was 'if transcript' which skipped
adding a user message, causing:
  1. Audio injected into wrong message or not at all → stale responses
  2. Consecutive assistant messages at end of history → llama-server 400 error

Fix: use placeholder text '[audio]' when audio is present but transcript empty.
The _serialize() method then finds the user message and injects the input_audio
content part correctly.

Fix native-llm audio: wrap raw PCM in WAV container before base64 ... afe8652646

llama-server requires a valid WAV file for input_audio, not bare int16 PCM
samples. Added _pcm_to_wav() helper that writes a RIFF/WAV header around the
raw PCM bytes (16kHz mono int16).

Decouple from upstream fork, rename to computer, publish via Forgejo Packages ... e32e134d1b

- Rename importable module speech_to_speech -> computer (all src, tests, scripts, docs)
- Move workflows to .forgejo/ (Forgejo Actions), publish PyPI-compatible package to
  git.sync.wtf package registry instead of PyPI; drop GitHub-only artifacts
- Rewrite AGENTS.md for devel/main branch workflow + Forgejo Packages releases
- Remove unused files: Dockerfiles, demo/, archive/, star-history assets
- Make openwakeword a core dependency (wake word is core functionality)
- Fix 30 stale tests (qwen3 _setup_faster quant param, speculative-turns interrupt attrs)

chore: remove .DS_Store from repo, add to .gitignore aeef6d21a5

ci: simplify to single ubuntu-latest sanity job cb08dadb7f

ci: retrigger sanity job f789cf6cd0

ci: retrigger after runner URL fix f17f351caa

ci: install uv via curl, drop setup-uv action (node incompat) 3264f90b61

ci: retrigger with node:24-trixie image 78a00315ee

ci: fix lint and format errors that blocked the sanity job ae07e07495

fix: resolve mypy errors blocking CI (typing, None guards) 9bb669da4c

ci: install libportaudio2 for sounddevice in sanity job f644fcb388

ci: include faster-whisper extra for test_model_singletons efaac30be0

test: skip real wake word model tests when computer.onnx is absent d73fb1656f

Merge main into devel to resolve PR #1 conflict d5fed8f0e1

fix: only load faster_whisper singleton when stt backend is faster-whisper 90392f0dd9

docs: document server deploy workflow in AGENTS.md; add deploy.sh be7486509c

docs: per-turn latency breakdown design spec 1d988c3638

feat: per-turn latency timing tracker e8249a9d69

test: strengthen turn timing tracker thread-safety test 4c6d4a2713

feat: record per-turn get/put boundaries in BaseHandler 8cff7ee583

docs: clarify turn_timing_stage shadow guard in BaseHandler a57ece2d0a

feat: inject turn timing tracker into realtime pipeline handlers 6438f55800

fix: annotate turn_timing_stage in BaseHandler for mypy 540d439c6f

feat: record TTS first-audio latency boundary 2da7094e9f

feat: finalize per-turn timing on response done and barge-in cancel ed5ceee9ec

test: per-turn latency integration chain test 7cee47d0c5

test: ensure pipeline threads shut down on integration test failure 5b8a34b5d8

fix: distinct STT→LLM handoff slot and true queue-wait fields in turn summary 7d6feea133

fix: deploy.sh ssh background start returns and logs to repo dir 63273bc71d

docs: echo-gate playback-ack design spec + native-llm echo-filter caveat a0befe9bb2

feat: register output_audio.played client event a4ec24093d

feat: gate state fields on PipelineUnit 1f09d784a6

feat: hold echo gate until playback ack with duration fallback 327b99037e

feat: handle output_audio.played ack to release echo gate 44d678b5e7

feat: realtime-paced echo reference feed 9cba259813

feat: client tracks real played audio bytes 2622f43b13

feat: client sends output_audio.played ack when playback drains 283576cd67

test: cover client playback-ack trigger logic 242485f958

docs: align READMEs and example scripts with renamed binaries fbb858d77a

feat: add purple app logo with white-filled interior to README a684a65aea

troed force-pushed devel from a684a65aea to d61d263f88 2026-08-07 11:17:57 +02:00 Compare

troed merged commit 47de46b4c4 into main 2026-08-07 11:19:25 +02:00

troed referenced this pull request from a commit 2026-08-07 11:19:27 +02:00

Bring devel into main: echo gate, per-turn timing, CI and logo work (#2)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

starfleet/computer!2

No description provided.