Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

LLM / ML / software engineer

Member at a project. Calls models from code, notebooks and coding agents; wants a key without filing a ticket, and — when something breaks — wants to find out where without waiting for an admin. Has no business with providers, budgets, people or policy, and the dashboard should not put those in the way.

Two things make this persona sharper than “a user with a key”:

  • Their MCP servers and models are often local — an MCP server a coding agent launched over stdio, a model on their workstation — so “where is the problem?” means their machine, rolter, or the upstream, and they need to tell which.
  • They read their own request bodies to debug. A member of the project reads captured bodies by default (#1820); a viewer does not.

Goal: “I call models through rolter with my own key, and when a call fails I can see why.”

Dogfood account: engineer@rolter.local, member of project default/default (the fleet’s project). engineer2@rolter.local is a member of research/sandbox and should see none of it. See dogfood data.

The observer watches: the time from first sign-in to the first successful call (ui_events screen views → the first request_logs row on the engineer’s key), and every screen the engineer opens while debugging E5 — the order they look in is the finding.

E1 — arrive

#stepwhereexpectstatus
E1.1sign inlogin screen (password or SSO)the dashboard, scope default/default; admin-only controls disabled with the role namedbug — #1846
E1.2find out what models existModels → Model Catalogthe project’s routes: gpt-4o, llama-3.1-8b, … with their capabilitiesverified

E2 — get a key

#stepwhereexpectstatus
E2.1mint a personal keySettings → My Virtual Keys → Generate virtual key · POST /api/v1/me/projects/{id}/virtual-keysthe key once, a name, an expiry; a viewer would be refusedbug — #1846 in the dashboard; verified through the API
E2.2narrow it to the models the job needsthe mint form’s model list/v1/models with that key lists only thoseverified
E2.3rotate or delete itthe key card · POST /api/v1/me/virtual-keys/{id}/rotatea new secret; the old one refusedverified

E3 — make the first call

#stepwhereexpectstatus
E3.1from the Playground, with no key handling at allPlayground (mints a 30-minute session key for the project)an answer, streamedbug — #1846, #1853
E3.2from code, OpenAI dialectOpenAI SDK with base_url=<gateway>/v1, api_key=<key>the same answer; x-request-id on the responseverified
E3.3from code, Anthropic dialectAnthropic SDK with base_url=<gateway>, the key as x-api-keythe same model family through /v1/messagesverified
E3.4streaming, embeddingsstream: true; /v1/embeddings on text-embeddingSSE chunks; vectorsverified

E4 — point a coding agent at rolter

#stepwhereexpectstatus
E4.1repoint Claude Code, Codex or Cursorthe third prompt in agent setupthe agent’s own traffic appears in LLM Logs under the engineer’s keyworks

E5 — “my calls fail: is it me, rolter, or the upstream?”

The debugging loop, in the order an engineer should be able to walk it.

#stepwhereexpectstatus
E5.1read the error the client gotthe response bodyOpenAI-style JSON whose type and code name the cause: 401 authentication_error; 403 model_not_allowed / route_not_allowed / provider_not_allowed / required_header_missing; 400 guardrail_blocked / plugin_blocked; 404 model_not_found (no such route); 402; 429 rate_limit_exceeded with Retry-After, or queue_full / queue_timeout; 503 queue_dropped; 502 pii_sanitizer_unavailable; other 5xx (no target, or the upstream)verified (401, 404 here; 403 in P1.2)
E5.2find the requestGET /api/v1/analytics/invocations?request_id=<x-request-id> (#1863); LLM Logs, filtered by status = error and modelthe row: status, target, provider, error text, latency, retries, and the captured bodiespartial — #1861 (LLM Logs has no request-id search; the API filter lands with #1863)
E5.3follow it into a tracethe row’s trace_id, copied from the LLM Logs drawer (#1983), in SigNozwhere the time went: queue, upstream, streamworks
E5.4check whether the upstream is sickCircuit Breaker, provider healththe provider behind the route: breaker state, uptime, recent failuresgap — #1833 (health needs an org-level role)
E5.5a call the client gave up onLLM Logs, status 499which target the client was waiting onbug — #1816
E5.6a call that is slow only under loadconcurrent calls to one routelatency stays flat as concurrency risesbug — #1815

E6 — “my MCP tool call failed”

#stepwhereexpectstatus
E6.1connect an MCP server that needs OAuthMCP Gateway → MCP Catalog, then consenta session of their own under Auth Sessionsworks
E6.2call a tool through the gatewaythe agent, via <gateway>/mcp/{server}the tool answersworks
E6.3see the failed tool callObservability → MCP Logsthe engineer’s own tool-call rowsgap — #1831 (superadmin-only)
E6.4a session expiredAuth Sessionsthe session’s state and a way to renew itworks

E7 — “my local MCP server misbehaves”

#stepwhereexpectstatus
E7.1see the raw JSON-RPC between the agent and a stdio server—every request, response and notification, live — the job tools like mcp-snoop do todaygap — #1832

E8 — “try my local model through rolter”

#stepwhereexpectstatus
E8.1point rolter at http://localhost:11434 for personal use—a route only the engineer’s keys reachgap — #1839 (an idea, not a plan)
E8.2the workaroundrolter easy-up on the workstationa personal gateway with the same API and logs, pointed at the local model through rolter.tomlworks

E9 — “what have I spent?”

#stepwhereexpectstatus
E9.1usage per key over the last weekSettings → My Virtual Keys (/api/v1/me/usage)requests and cost per keyverified
E9.2how much of my allowance is left—a personal budget and its remaindergap — #1830

E10 — make the dashboard mine

#stepwhereexpectstatus
E10.1set a display name and a short bio—other people see the name instead of an e-mailgap — #1823
E10.2language, default project, Playground modelthe language picker; the scope switcherremembered — but in this browser onlypartial — #1824
E10.3save “my errors this week” as a view—a named filter on LLM Logsgap — #1825
E10.4protect the account with a second factorSettings → My Virtual Keys → Two-factor authenticationTOTP and recovery codesverified