Dashboard UX telemetry

The dashboard emits a structural-only event stream to POST /api/v1/ui-events, which lands in the ui_events ClickHouse table beside request_logs. It answers the questions spans cannot: which screens are slow to become usable, where people back out, which forms get abandoned, which error and empty states are actually reached.

The rules the stream lives by are in the ADR (dashboard UX telemetry) and, in code, at the top of ui/src/lib/ux.ts and ui/src/lib/ux-react.tsx. This page is the practical half: where the instrumentation lives and what a new screen has to do.

Instrument the shared component, never the screen

This is the one rule that decides whether the data is complete. A hook wired into a screen is instrumentation that stops at whoever remembered it; a hook wired into the component every screen renders is instrumentation that cannot be forgotten.

EditorSheet backs thirteen screens and ConfirmDialog backs fifteen, so the two of them cover most of what the dashboard does. Both take a required name, which is what stops a new call site from being silently uninstrumented: adding one without a name is a type error rather than a gap nobody notices until the data is queried.

The screen key is not a prop. It travels through UxScreenProvider, mounted once by the app shell, so a sheet rendered outside one is silent rather than mislabelled.

Names are keys, not content

name and target are stable slugs — virtual-key-create, alert-channel-delete, sso-connection-edit. Never a title, never a row’s own name, never anything derived from what was typed. sanitizeKey in ux.ts rejects anything that is not a plausible key rather than truncating it, because a truncated secret is still a secret; the cost of getting this wrong is one missing event, and that is deliberate.

Where a sheet serves both create and edit, name both — custom-role-create and custom-role-edit are different findings, and an abandon rate that mixes them answers neither.

Adding a screen

A new screen calls useScreenReady(ready) with its own primary-data loading flag. Only the screen knows which of its several queries is the one the user is waiting on, so this stays with the screen rather than being inferred centrally — a guess would produce a number that looks authoritative and is not. Take care with a query that is enabled: false: it stays pending forever, so gate it the way Playground does rather than letting it suppress the event.

Everything else — forms, confirmations, empty and error placeholders — comes free from the shared components, provided the screen uses them.

Asserting the events

The events are asserted in stories, through the recording helpers in ui/src/pages/story-harness.tsx:

• recordUxEvents in the meta’s beforeEach, so a story neither inherits the previous one’s events nor leaves its own behind
• expectUxEvent(action, target?) waits for one event and hands it back, so the story can go on to assert the screen, the outcome or the duration
• expectNoUxEvent(action, target?) is the other half, and it is the one that matters for a cancel: a dialog that emitted form_submit on the way out passes every positive assertion and still makes the data say the delete went through
• uxEvents() for the cases where the sequence is the assertion — a refused save is ["ok", "error"] on form_submit, two rows, not one

The queue is never flushed under a story, so it can be read directly.

Run them with bun run test:stories <file> from ui/.

What duration_ms measures on a form

useFormTelemetry keeps two clocks, and each action reads one of them:

The dwell since opening is what separates a misclick from a decision, so every row about the operator’s time in the form carries it. save_confirmed is read the other way: the save-latency query in the observability docs takes it as the wait between pressing save and being told it worked. A clock started at the opening would count the form fill too, and a twenty-second read of a delete dialog would come out as a twenty-second save (#1894). The press clock outlives the form closing, because a caller that closes from its mutation’s onSuccess reports the landing a commit after the form went away (#1761).

The struggle signals

Three of the actions record what the operator could not do (#1731). They are the reason the stream is worth pointing at a dogfood week at all — everything else says what happened, and these say where it went wrong.

form_abandon keeps its old meaning and is now only the clean case — a form closed untouched, which is a misclick rather than a design problem. The two are separate actions rather than one action with a flag precisely so the existing series is still comparable across the change.

A retry is state the hook carries: failed() arms it, the next submitted() spends it. A third save after a success is a fresh attempt, not a retry of a failure two saves ago.

Why a refused click takes a wrapper

A disabled button is inert. The HTML spec has the user agent withhold the click event from a disabled form control, so the one interaction worth measuring is the one the DOM refuses to report — and every alternative to disabled is worse. aria-disabled with a swallowed handler makes a screen reader announce a control that is not one; removing the control takes away the explanation of why it is missing.

So the control stays genuinely disabled and useRefusedClick catches pointerdown in the capture phase on a display: contents wrapper beside it. Capture runs on every node on the event’s path before the target, and a pointer event is dispatched to a disabled control where a mouse or click event is not, so the wrapper sees the reach the button never will. The wrapper generates no box, so the control keeps its place in the parent’s layout.

A disabled control is not focusable, so there is no keyboard path to miss, and nothing is deduplicated: reaching for the same refused control four times is the signal, not noise.

GatedButton, GatedSwitch, GatedCombobox, RowIconButton and a gated DeleteIconButton are wired, which is every shared gated control — and the only way a screen gates one (#1759). A screen that read useGate() itself and disabled a plain button got the same look and recorded nothing: Keys and Models shipped their row edit and delete that way, the exact controls the struggle signal was meant to tell apart. The guard in ui/src/components/gated-controls.test.ts fails on a useGate() outside the primitives unless it carries a // use-gate-allow: <reason> waiver, and RBAC gating states the rule from the gating side.

Naming the control

Every event already carries its screen and the capability that refused it, which says which permission boundary is in the way. The control slug is the part that says which of the refused controls on that screen was reached for — on Users, the difference between “the role grant is gated” and “deactivating is gated” (#1750). So every gated primitive takes a required control — DeleteIconButton as soon as it is handed a gate — the way EditorSheet requires name: a required prop is what stopped form instrumentation drifting, and it does the same here.

The slug is

• kebab-case <noun>-<verb> — provider-new, alert-rule-toggle, sso-provider-secret-clear. The noun is the thing the control acts on as the screen names it, the verb is what the control does to it.
• a string literal at the call site — never the button’s label, never a translation, never anything read off the row it sits in. A label changes with the locale and a row’s name is the operator’s content; either one turns the target from a key back into free text. sanitizeKey drops a control key that is not a plausible slug, but the rule is that it never gets that far.
• unique within its file, so no two controls share a target. The empty state’s call to action is a different control from the header button that does the same thing, and says so with an -empty suffix — provider-new-empty — because “people reach for the empty-state button” is itself worth knowing.
• stable across copy changes. A slug is a series key: renaming one splits its history in two. A control whose label flips with row state (retire and restore, drain and return) keeps one slug, named for the action it gates.

A slug does not repeat the resource for its own sake — the capability travels beside it, so attribution-new on Cost Attribution resolves to attribution-new:business_unit:create or …:customer:create by itself.

ui/src/components/gated-controls.test.ts enforces the shape from the source: every shipped call site passes a literal, kebab-case slug, and no file repeats one. The component still falls back to its kind — button, switch, row-action — for anything that reaches it untyped, so such a call site is degraded rather than silent: it still records the capability and the screen.

What a refused row carries, and why it is on by default

This one is a privacy-shaped call, so the reasoning is written down rather than left in a review thread. The row records that a control was reached for and refused:

screen  = providers
action  = refused_click
target  = provider-new:provider:create
outcome = error

plus the session_id, org_id/team_id/project_id and app_version every event in this stream already carries, and the user_id the server fills from the session for every event alike. No label, no message, no identity beyond that. It says “this permission boundary is in somebody’s way”, not “this person tried to do something they should not have” — which is the same structural-only bar the rest of the stream is held to, and what makes shipping it on by default defensible rather than surveillance.

sanitizeKey is what holds that line, and it is deliberately asymmetric. The capability is the signal, so one that is not a plausible key drops the event entirely; a control key that is not one — a call site that passed the button’s text — drops only the control and still records which boundary was hit.

Adding an action

The action enum is a ClickHouse Enum8, where the ordinal is the stored value. It is append-only: renumbering an existing value rewrites the meaning of every row already written rather than migrating it. Adding one means

1. a new clickhouse/NNN_*.sql that re-declares the enum with the existing ordinals untouched and the new value appended,
2. the value appended to ACTIONS in crates/rolter-control/src/ui_events.rs, in the same order — the_action_list_is_the_enum8_in_ordinal_order pins it, so a value sorted into the middle is a failing test rather than a silent remapping,
3. the value added to the UiEvent["action"] union in ui/src/lib/api.ts and a named emitter in ux.ts, so a typo at a call site is a type error.

Proving the pipeline end to end

Stories assert what the dashboard emits. They say nothing about whether a row ever reaches ClickHouse, and the client is built so that it never will say: ux.ts swallows every transport failure on purpose, because a dashboard whose save button breaks when analytics 500s is worse than a dashboard with no analytics. A pipeline that has quietly stopped working is therefore indistinguishable, from every screen and every log, from a pipeline nobody is using.

crates/rolter-control/tests/ux_pipeline.rs closes that gap: it posts the payload ux.ts builds to the real POST /api/v1/ui-events, then reads the rows back out of ClickHouse and asserts screen, action and ts. Reading ts back is the only place #1224 — a batch stamped at flush time, collapsing a whole session onto one instant — is observable at all, since a row stamped at ingest is otherwise a perfectly valid row.

It runs wherever both ROLTER_TEST_DATABASE_URL and ROLTER_TEST_CLICKHOUSE_URL are set, and self-skips otherwise. CI sets both: the nextest / doctests job in quality.yml runs a ClickHouse service beside its Postgres one. Locally:

docker run -d --name ux-ch -p 18123:8123 clickhouse/clickhouse-server:24-alpine
ROLTER_TEST_CLICKHOUSE_URL=http://127.0.0.1:18123   cargo test -p rolter-control --features postgres --test ux_pipeline

The test creates the ui_events table itself, from the shipped clickhouse/008_ui_events.sql rather than from a copy — a private copy would let the table under test drift away from the one a deployment gets, which is the class of failure it exists to catch.

Failure modes, and who notices

Every row below was produced by running it, not by reading the code. The middle column is what decides the blast radius: ux.ts treats 401, 403, 404 and 405 as terminal and stops sending for the life of the tab, and treats everything else as transient — dropping the batch it was holding and trying again on the next flush.

“Nobody” is literal where it appears: the dashboard shows nothing, and the control plane does not log a rejected request. A store that cannot take the write is different, because it is a deployment fault rather than a client bug, and since #1747 the server reports it even though the client stays quiet:

• rolter_control_ingest_failures adds one per lost batch, labelled stream (ui_events, or mcp_logs for the MCP ingest endpoint, which shares the path) and reason (insert for a refused or unreachable store, unconfigured for a missing CLICKHOUSE_URL) — alert on any sustained rate;
• a telemetry ingest failed warning names the store’s own error, at most once a minute per stream, with a suppressed count of the failures since the previous one. Once a minute rather than once a batch because this is a per-flush path from every open tab, and a warn per batch would bury the log it is meant to be found in;
• the 500 body says only that the event store did not accept the write. It no longer quotes ClickHouse’s error or the insert URL, which are internal topology on a client-facing response; any credentials in the URL are masked in the log too.

The status stays 500 on purpose: ux.ts reads it as transient, so capture resumes by itself once the store is back. The logic lives in crates/rolter-control/src/ingest_failure.rs.

The last row is the only deliberate one. In a Postgres-backed deployment it is the ui_events column on the logging_settings singleton (migration 0073, #1748): the Dashboard Usage Events switch on Logs Settings, or ui_events in the PUT /api/v1/logging-settings body. PostgresConfigStore projects it into logging.ui_events, which is what the handler reads, so the switch takes effect on the next batch with no restart. rolter-seed --import writes it too, but only when the file names logging.ui_events — the default is on, so a file silent about it must not undo an operator’s opt-out. The same rule holds for the API: a PUT that omits ui_events leaves the stored value alone. Only a TOML-only control plane reads the key from rolter.toml directly.

switching_ui_events_off_answers_202_and_stores_nothing in crates/rolter-control/tests/ux_pipeline.rs pins the row: it switches the stream off through the API, sends a valid batch, asserts the 202 and an empty read-back, then switches it on and watches the same batch land.