crew

User system for starfleet speaker ID: users, Matrix identities, enrolled voiceprint embeddings, and the connected-device registry (guided enrollment and verification through computer's capture endpoint). Sister service to computer (voice pipeline).

Design: docs/specs/2026-09-02-crew-user-system-design.md.

computer owns recognition. crew owns records. computer polls crew's GET /registry; enrollment clips are proxied to computer's POST /internal/users/enroll, and only the returned embedding is stored.

Setup

uv sync --group dev

Configure

Copy the example and edit the live file. Never commit the live file:

mkdir -p ~/.config/crew
cp crew.toml.example ~/.config/crew/crew.toml

Important settings:

Setting	Meaning
api.bind, api.port, api.token	crew HTTP listener and bearer token
store.db_path	SQLite database
computer.enroll_url, computer.token	existing computer enroll endpoint and token
computer.devices_url	computer's connected-device list (admin UI device view)
computer.capture_url	computer's guided-capture endpoint ({device_id} placeholder required)
clips.min_clip_s	minimum accepted enrollment-clip duration
ingest.token	token roster must send when POSTing context to /internal/context (empty trusts the LAN)
roster.url, roster.token	roster API crew provisions per-user context jobs against
roster.scripts_dir	allowlisted scripts dir on the roster host; job script_path values are built from it

Run

uv run crew serve --config ~/.config/crew/crew.toml

API

Endpoint	Purpose
GET /health	status, version, user count, clip count
GET /users	user metadata and clip counts
POST /users	create {id?, display_name, matrix_user_id?}
GET /users/{id}	user detail and non-vector clip metadata
PATCH /users/{id}	update display name or Matrix ID
DELETE /users/{id}	delete a user and cascade its clips
POST /users/{id}/clips	validate multipart WAV and store its embedding
DELETE /users/{id}/clips/{clip_id}	delete one clip
POST /users/{id}/clips/enroll	guided capture on a connected device; store the returned vector
GET /devices	stored devices merged with computer's live connection list
PUT /devices/{id}	patch name / device_type / owner_user_id (upserts)
POST /devices/{id}/verify	guided capture; identify who is speaking right now
GET /registry	computer-compatible user/vector export
GET /public-config	public min_clip_s for the admin UI (no token)
GET /device-owners	device_id → owner_user_id map for computer prompt resolution
GET /users/{id}/session-prompt	composed session prompt plus the context blocks behind it
GET/PUT/DELETE /users/{id}/prompt-sections/{key}	per-user prompt sections (about, instructions)
GET /context-options	catalog of available context options
GET/PUT/DELETE /users/{id}/context-options/{option_id}	per-user context options; PUT provisions the roster job
GET /users/{id}/context-options	stored context options for a user
POST /internal/context	roster context deliveries (ingest token, not the API token)

Enrollment uploads must be 16 kHz mono 16-bit WAV files at least min_clip_s seconds long. Recordings are never committed to this repository.

Devices

computer reports the clients currently connected to its realtime endpoint; crew merges that live list with its own stored rows (device_type, name, owner_user_id) at GET /devices. device_type is one of esp32, pc, phone; PUT /devices/{id} upserts a row, so a connected-but-unstored device can be labeled on first sight.

Guided enrollment and verification run through computer's capture endpoint (capture_url, {device_id} placeholder):

• POST /users/{id}/clips/enroll with {"device_id": "..."} makes the connected device record, computer extracts the vector, and crew stores it as that user's clip — no file upload needed.
• POST /devices/{id}/verify captures briefly and returns best_user and score, i.e. who is speaking right now.

Both pass through computer's 404 (no such user/device), 408 (no speech), 409 (already capturing), and 503 (speaker model down) verbatim.

Admin UI

Open http://127.0.0.1:4030/admin after crew serve starts. Enter the crew API token once; the browser keeps it in localStorage and sends it as the bearer token with X-Client-Id: admin-ui.

The page lists users with clip counts, creates users, edits display names and Matrix IDs, and deletes users.

Each user detail view has four tabs: Identity edits the display name and Matrix ID, Prompts edits the prompt sections (About, Instructions), Context toggles the per-user context options (see below), and Voice manages the enrollment clips — uploading audio files (normalized to 16 kHz mono WAV in the browser before upload) and recording from the microphone (encoded to 16 kHz mono WAV in the browser).

A devices section shows every stored or connected device with a live connection badge, lets you set type, name, and owner, and offers per-device enroll into user and verify speaker actions driving the guided-capture flow above. Enrollment clips store embeddings only and cannot be played back. Enrollment requires computer running; upload errors report the enroll failure verbatim.

Settings shows the stored token, the health of the API, and the minimum accepted clip length.

These screenshots use fictitious data (see scripts/makedoc_seed.py) and are generated with a headless Chromium; run bash scripts/makedoc.sh to regenerate them.

Per-user prompts and context

GET /users/{id}/session-prompt returns the session prompt computer injects for a device owner: an opening line, the user's non-blank prompt sections in about, instructions order (the About the user: header appears only when the about section has content), and one block per enabled context option with its latest fetched content and time. It is empty until the user has sections or enabled options with delivered context.

Two context options ship in v1 (see GET /context-options):

Option	Param	Schedule	Context
weather	location	daily 06:05	weather report text
briefing	calendar_entity	daily 06:15	today's calendar summaries

How an option flows: enabling it in the admin UI has crew provision a scheduled job on the roster host (crew-{user_id}-{option_id}) that runs the option's script with the user's params as environment variables → the job runs on schedule and its stdout is delivered back to crew's POST /internal/context (ingest token from [ingest]) → crew stores the latest content per user and key → computer injects it via session-prompt. Disabling an option pauses its roster job so it stops firing and delivering, and enabling it resumes the job; removing the option deletes the job. Stale stored content is still filtered out at render until fresh data arrives.

Config: [ingest] token protects the delivery endpoint; [roster] url, [roster] token, and [roster] scripts_dir point at the roster API and its allowlisted scripts directory (see the Configure table above).

Example scripts live in examples/context_scripts/: crew_weather.sh (fetches wttr.in) and crew_briefing.sh (Home Assistant calendar REST via a sourced env file). Copy them to the roster host's scripts dir, make them executable, and adjust the data sources to your own.

CLI

crew list --url http://127.0.0.1:4030 --token "$CREW_TOKEN"
crew create-user --name "Ada Lovelace" --matrix-user-id "@ada:example.org"
crew delete-user ada-lovelace
crew enroll ./ada-1.wav --user ada-lovelace
crew doctor

Deploy

The deploy environment lives in the local-only ~/.config/crew/deploy.env (never commit): it exports SERVER_HOST, SERVER_USER, SERVER_REPO_DIR:

source ~/.config/crew/deploy.env
./deploy.sh

Development

uv run ruff check src/ tests/
uv run ruff format --check src/ tests/
uv run mypy src/
uv run pytest tests/ -x -q