Kiosk display

A paired browser — a phone today, a Raspberry Pi 3 with a 7" touchscreen next — shows your Ramble bird and talks with the Crow assistant you bind to it. Crow does all the work: speech-to-text, the assistant's turn, text-to-speech. The browser only captures audio after a tap and plays the reply.

Pieces

• bundles/kiosk/ — the bundle: page (public/), routes + WebSocket (server/runtime.js, server/session.js), pairing (server/pairing.js), windows (server/wm.js), MCP tools (server/server.js), dashboard panel (panel/kiosk.js).
• servers/gateway/voice/turn.js — the transport-free voice turn (bound bot, routing with an 8 s cold fallback, think gate, barge-in, memory stripped by default).
• servers/shared/device-store.js — paired devices (device_kind: kiosk), token hashes only.

Network and auth

• Tailnet only: https://<host>:8444/display. Never Funnel (/display, /api/kiosk are not public prefixes; the router and the WebSocket upgrade also refuse a Funnel header).

• Pairing: the display shows a 6-digit code; an owner approves it in Kiosk → Pair a display and picks the assistant. The display collects its token once with a poll secret.

• The device token is accepted only in the session's first hello frame — never in a URL, never by any other route.

• MCP tools reach live sessions through loopback-only /api/kiosk/internal/* with $CROW_HOME/kiosk-announce-token.

• Theme: each paired display has a Theme setting in the Kiosk panel (kiosk_settings.theme: auto | light | dark, default auto). Automatic turns the page dark during the display's sleep hours (default 22:30–06:30); Light or Dark holds all day. A save reaches an open page at once. The dashboard overlay has no stored settings and follows the OS color scheme.

• The page is served at /display, not /kiosk, because the installed maker-lab bundle owns /kiosk/*. The API stays under /api/kiosk/* and the dashboard panel at /dashboard/kiosk.

Talk to Crow in the dashboard (session mode)

The Crow's Nest header bird opens the same page inside the dashboard, with no pairing: tap the bird and choose Talk to Crow, or press and hold the bird (a held Enter/Space on the focused bird does the same). It uses the microphone and speaker of the device you are on. The row and the long-press exist only while the Kiosk extension is installed.

• Page: /display/session; WebSocket: /api/kiosk/session/dashboard. Both sit behind the same network rule as the dashboard (never Funnel) and require the dashboard's own session cookie — the bundle mints no token for this. The page answers 401 without a session; the socket is refused before the upgrade.
• The socket handshake must come from this origin (Sec-Fetch-Site: same-origin, or an Origin that names the Host / X-Forwarded-Host), and its first hello frame must echo the crow_csrf cookie — the same double-submit rule as every dashboard POST. A reverse proxy in front of Crow must forward Host or X-Forwarded-Host.
• Nothing is stored: the display exists in memory for the life of the socket, under an id derived from a hash of the session. It is not a paired device, so it is not listed in Kiosk, cannot be announced to, and cannot be claimed by a device id. The login is re-checked at every question and once a minute; logging out closes the display. Its windows, timers and short conversation are dropped two minutes after the overlay closes.
• Assistant: the one chosen under Kiosk → Talk to Crow in the dashboard, else (Automatic) the first enabled assistant that fits the quick voice model — in full, else without its skills, never one that is too large (see "Does the assistant fit?"). The setting refuses a too-large assistant. It runs with the same tool limits, display-tool rules and clock as a paired display (the browser reports its time zone), and memories stay off. With no enabled assistant that fits — or a chosen one that no longer does — the page says so and links to the Kiosk panel. The picker names who Automatic would pick right now (dashboard_voice.auto_bot_id on the listing and on the save — the same resolver with no choice), so the option reads "Automatic — now {name}" even while another assistant is chosen. Bot Builder's cards read the same listing to say where each bot answers voice, and its ⋯ → Use for voice… writes through these same two admin routes (dashboard-voice {bot_id}, displays/:id {bound_bot_id} only); reading the listing never starts or warms a model (tests/voice-assess-no-acquire.test.js).
• Closing (the close button, or Esc) stops the microphone and any audio at once and removes the frame.
• The page is framed by the dashboard only (frame-ancestors 'self'); the paired page at /display still refuses all framing.
• Player bar (the Talk to Crow overlay opens its page with ?bar=1 when the dashboard's bar has the kiosk backend; the page then plays through the bar): GET/POST /api/kiosk/session/media lets the dashboard's player bar read and steer what the overlay started after the overlay is closed. Same gate as the socket — the network rule (never Funnel), the dashboard session cookie (401), and on POST the X-Crow-Csrf double-submit (403). It reaches only the caller's own login's display, and only when that display's page asked for the bar in its hello (caps.bar plus a per-tab nonce; a paired display can never ask). GET hands the current item (and its ticket) only to the tab that owns the audio, named by the X-Crow-Owner header carrying its nonce; any other tab, a missing or malformed header, or a display without a bar gets item: null. POST { do, id, owner } acts only on the current item for the tab that holds it (else 409 superseded), except that the owner's stop ends the session even with a stale item id (after a station reload); answers are never cached and the ticket path in them is never logged. Audio handed to the bar is not cut when the overlay closes: it ends at logout (checked every minute, open streams are cut) or when its ticket expires.

Voice services

• Speech-to-text: the Faster-Whisper bundle (loopback :8004); the kiosk adds a distil-small.en profile at first pairing. Each display can switch to the faster, less accurate tiny.en (Speech model) and set its end-of-speech wait (300–900 ms, default 450). Crow warms the display's speech model at gateway start, on connect, and after a settings change, so the first question is not slow.
• Voice: the Kokoro TTS bundle (loopback :8880) when installed.

Does the assistant fit?

The quick voice model has a small context (8,192 tokens on the stock local model), and an assistant's system prompt carries the full text of every skill it has. Before each model call the voice turn estimates the request (servers/gateway/voice/prompt-fit.js) and never sends one that cannot fit:

• the full prompt fits → nothing changes;
• it does not → the turn runs without the assistant's skill bodies (persona and tools stay); the turn's metrics carry prompt_fit: "no_skills";
• still too large → no model call: the display says that the assistant is too large and to choose another one in the Kiosk settings (failed: "bot_too_large").

Saved conversation is trimmed, oldest exchange first, when it would push a request over. The Kiosk panel shows the same result for every assistant in the picker — fits, works without its skills, or too large — and refuses to bind a too-large one. A general-purpose assistant with many skills belongs on a larger model; give a display a small household assistant.

How a spoken request is resolved

A request goes through the cheapest step that can answer it, and every step ends in the same executor (server/executor.js) with the same result shape.

1. A control phrase with a live target (server/phrases.js): "close that", "next step", "cierra todo". A fixed table of whole utterances in English and Spanish, compared word by word. It acts only when its target exists (a window to close, a recipe to step through) and names a window by its whole title or whole words of it; otherwise the request goes on.
2. An anchored pattern whose slot resolves on this display (server/patterns.js): the verb at the start of the utterance, then a name that must match something the display has. No match means no action here.
3. The assistant, with the display tools (server/tools.js, server/display-tools.js): crow_show (a card: text, a list, steps, a timer), crow_wm (close windows, move through steps), and, once a display has something to play or open, crow_play and crow_open. Each has at most three flat arguments, and every list of choices is built on the server for this display.

The clock answers without a model: the time, today's date, tomorrow and yesterday, the weekday of a named date, and the days until one (server/clock.js).

Offered, and required. A display tool is offered to the model only on a turn whose words are about it (or, for crow_wm, while a window is open). It is required when the person is asking for the thing now: new content, a change to the open card, a play or open request. On a required turn the model's text is held until the call has succeeded; a turn that ends without one gets one corrective round, and after that the display says that it could not, in its own words, instead of claiming it did. A sentence with two requests ("close the timer and then show me a list") is given to the model whole, with every tool that applies.

Results. A tool result is { ok, outcome, say, final }. A final result ends the turn on say, one sentence in the display's language, so a second model round is not needed to confirm what happened. If the model already spoke a sentence and the call changed something, say is not added on top. If the call changed nothing or failed, say is always spoken. A non-final result goes back to the model with the fix to make.

Forcing. A forced tool call is sent only to an engine known to honour it (read from the model server's own model list). An engine that ignores a forced call is sent none, and the corrective round does the work. "required" is never sent to an engine that is not known to honour it.

Other tools. The assistant's other tool families (projects, messages, files, news, add-ons) are offered to the quick model only on a turn whose words ask for them, or on the turn right after one that used them. A family that is not on offer is never run, whatever the model calls. Each turn's log line and the Diagnostics list say which families were not offered. Memory keeps its own rule (when asked to remember or recall).

Action requests. A request to add something to a calendar or list, set a reminder, send a message, or cancel an appointment (server/action-intent.js, en + es, object-anchored) never ends on a claim that no tool backs (servers/gateway/voice/turn.js). On an assistant with no tool for the turn, the display says it can't do that from here, with no model call. When the assistant has tools the quick model was not offered, the turn escalates to the larger model; if that model can't be had, the display says so. On every other action turn (and the turn right after one, when it reads as a change to it: "move it to 9 instead"), the model's words are held until a tool really changed something: a call to a tool that creates, sends or deletes (a list or search proves nothing), or a display call the words asked for. If the first-audio budget runs out after a change was sent, the display says it may have gone through rather than "try again". A reply still held at the end that says an action was done, promised or offered ("I've added…", "I'll remind you…"; servers/gateway/voice/action-claims.js) is replaced by the could-not-do line. The [kiosk-metrics] timings carry action_intent, action_no_tool, action_guard, claim_blocked and action_not_ready.

Display profiles. Each display has a type (small wall display, phone, tablet, desktop). What a display may show is the lesser of its type and what its page reports it can draw. A display with no type set is treated as the small wall display: audio first. The type is not stored until someone picks it in the Kiosk panel or the display's first connection guesses it from what its page reports (a phone says it is mobile; a small screen on an ARM Linux browser is a wall display); a guessed type is marked as guessed in the panel.

Display tool and clock

• A plain question is answered aloud in one model round with no display tool on offer. While a window is open, crow_wm alone is offered on such a turn (closes and steps).
• A card is refused when the question did not ask to see anything, and any card whose title or text is empty or a placeholder is refused, with the fix. A card sent under the title of the card that is open updates it; a new content card replaces the previous one; a timer always gets its own window, so setting one never cancels another.
• Every turn carries the display's local date, time and time zone on the user message (the page reports its zone when it connects; without one the server's zone is used).
• Each tool call's outcome is logged as name:outcome (shown, updated, placeholder, no_intent, not_offered, refused_policy, …), never its arguments or text; the page's debug list (long press on the clock) shows the same line.
• POST /api/kiosk/internal/turn-check (loopback and the announce token only) runs three fixed sentences through the same turn a display uses, on a display that does not exist, and reports what happened. It never escalates to the larger model (so it can never start one), and it passes only when the card it asks for is really on the screen (that sentence is tried up to three times). It is the check to run after an update.

Audio (radio stations)

A display plays audio through one <audio> element on its page. The page is never given a stream address or a credential.

• Stations. The Kiosk → Radio stations list (this instance only; never synced) holds a name, up to five spoken names ("H D two") and a public stream address per station. Test asks the address for its response headers only. A station is a public address on every hop, unless the operator ticks Home network for it: then its own host and port may be on the home LAN (inside a subnet of the interface that carries the default route) or on the tailnet (100.64/10, fd7a:115c:a1e0::/48), and a redirect may go only to that same host and port (never down from https) or to a public address. The addresses the host resolves to are checked by the server and recorded when the station is saved; a play must resolve inside that set ("its address has changed: test and save it again"). Refused even with the tick: loopback, link-local (cloud metadata), site-local, multicast, a subnet's network and broadcast addresses, any translated or tunnelled form, every container or VM bridge and point-to-point link of this host, and this host itself. On every hop of every station, ticked or not, this host's own addresses and every one of its interfaces' subnets (including its LAN's global IPv6 /64) count as private. A home-network stream never receives a credential.
• Asking. "Play <station>" plays an exact name or spoken name with no model call. The full name list is not sent to speech-to-text as a prompt (it made the speech model spell letters, repeat itself and mishear short commands). An operator can turn on CROW_KIOSK_STT_HOTWORDS=1 on the gateway to send the stations' ALL-CAPS call signs to a faster-whisper server as hotwords; it is off by default and only ever used when both the display and the speech-to-text profile are English. A name or spoken name that is itself a command, like "Stop", is refused. A call sign written by its sounds ("KDBF", "Cady PF", "kay tee pee eff" for KTPF) still finds the one station that sounds the same; one that is a single sound off ("BTPF") is asked about ("Did you mean KTPF HD1?"); two stations that sound alike are offered, never guessed between. A transcript that is a long repetition loop ("K-P-P-P-P…") ends the turn as if nothing was heard; ordinary repeated words ("no, no, no") are speech. Anything less certain goes to the assistant, whose crow_play tool lists only the sources this instance has. "Which one?" is followed by a bare name for 30 seconds.
• Transport. "Pause", "keep playing", "louder", "quieter", "mute", "next song", "what's playing", "stop" act at once with no model, but only while something is playing and only when they would change something; otherwise the sentence goes to the assistant like any other. Any pause request while the music is already paused is answered with no model, and a model's playback call never runs against the sentence's own command ("Pause." never resumes). The single words "Pause." and "Play." are the exception: with something loaded they are always answered with no model, and the current item is sent to the page again (so a page that shows something else is put right); "Play." with nothing loaded answers "Nothing is paused. What would you like to hear?". Volume 0–100 spans −50…0 dB (0 is mute); each "louder"/"quieter" moves 10 dB, "quieter" stops at a quiet but audible floor, and a spoken level ("volume 50") is that point on the scale; a new session starts one "louder" below full. A Loudest volume stored before this change is moved once to the step that is as loud as it was or quieter (never louder). While something plays, a playback word with only filler around it also acts ("louder louder", "skip it", "a lot quieter"). When the assistant gets a request to change the playback ("turn the radio up a little"), it must make the call: a claim with no call is never heard, and the display says it could not change the playback. A ringing timer takes "stop" first. "Close everything" also stops the audio.
• Tickets and the relay. Each item gets a display ticket: 128 random bits in a path under /display/t/, bound to one display and one stream, behind the same network gate as the page, GET only, at most two requests at once, ended when the item changes, when the display is unpaired or its dashboard login ends, and at 12 hours (open requests are cut then). The gateway fetches the stream itself (server/relay.js): every hop is checked, the address that was checked is the address that is dialled, redirects are followed by hand, and only audio passes. Ticket paths never reach a log.
• The session. The media session lives on the server (server/media.js), so a reload or a reconnect restores it; a display that is away for more than 30 seconds loses it. A station that drops after it has played is reloaded up to three times before the display says it stopped.
• Ducking. While the display listens or speaks, the music is turned down to 15 % of its level and comes back when the reply has finished (or after 30 seconds, as a backstop). Pause music while listening pauses it instead, for a room where the music drowns the microphone; it is on by default for a phone or a tablet (a stored choice always wins). Loudest volume caps a display.
• Phones and tablets. On an Android phone or tablet the page lets the microphone go when the turn has settled — after the spoken reply, never in the middle of it (with follow-up on, at the end of the conversation) and opens it again at the next tap (no new permission prompt): an open microphone keeps Android in call audio mode, which plays music through the call path. Other displays, and iPhones until checked, keep it open.
• Now playing. A chip on the page shows what is playing; a tap on it, "what's playing" (or crow_open) opens a window with ⏮ ⏯ ⏭, quieter, louder and mute buttons. On a display with a screen the window also opens by itself when playback starts: one window, reused; it comes to the front only when nothing else is open (otherwise it opens behind the window in front, and the chip brings it forward), and opening by itself never pushes another window out. While the display has lost its server the chip is dimmed and the buttons are off; after 60 seconds the audio stops (the page tries to reconnect every few seconds while audio is live). The window closes when the audio ends.
• The music library. With the Funkwhale extension installed (and its access token), the display plays the library read-only once Kiosk → Music library has the library address and the file storage address the server sends its songs from, both entered by the operator and never guessed (Check compares the storage address with the server's real redirect). Names are matched on the display's side against an index of albums, artists and genres built in the background ("play some hip hop" finds the tag HipHop; "Side by Side" stays a title); track titles are searched live. The first request goes to the library address with the token, to the listen path only; its one redirect must lead to the storage address, and the token never goes there. A library that does not answer is said as such, never as "not found". Nothing is written to the library.
• The news. "Play the news" plays the newest briefing with audio when it is at most seven days old; otherwise the display says there is no recent briefing. Only an .mp3 inside the data directory's media/audio/ is ever served, checked again at every request.
• The library's own tools. An assistant's fw_play / fw_play_album result joins the display's session only from those tools, only for the server's listen path; the stream is rebuilt on the server, never fetched from the address in the result.
• No listening history is written.

Privacy

• The page sends its time zone name when it connects, so the display can tell the time.
• Nothing is sent before a tap. No audio is stored; transcripts live only in the display's short in-memory conversation (15 min).
• Memories are off by default on a display (memory_integration). When they are on, the memory tool is offered only when the question asks to remember, recall or forget something (server/memory-intent.js): the assistant uses memories when asked, not on every question.