Security

Secret handling

• Upstream provider keys are never stored in plaintext in the database. They are envelope-encrypted with AES-256-GCM: a per-record data key/nonce, wrapped by a master key (KEK) supplied via ROLTER_KEK (env/file). Pluggable backends (HashiCorp Vault, cloud KMS) are a roadmap item.
• In the bootstrap file, prefer api_key_env over inline api_key so secrets stay in the environment, not on disk.
• Virtual keys are stored as hashes with a short display prefix; the raw key is shown once at creation.
• Secrets are never logged. The gateway redacts auth headers from traces.

Transport

• Upstream calls use rustls (no OpenSSL). HTTP/2 keep-alive with connection pooling.
• Optional per-provider egress proxy (egress_proxy, HTTP/HTTPS/SOCKS5) for networks where providers aren’t directly reachable.
• Optional global or per-provider custom CA bundles add private PKI roots to outbound upstream clients while retaining public roots, certificate-chain validation, and hostname verification.
• Terminate TLS at the gateway or a fronting proxy/ingress in production.

Cross-origin policy (CORS)

security_settings.allowed_origins and allowed_headers are enforced by a middleware on the control plane (crates/rolter-control/src/cors.rs). The policy is read from an ArcSwap per request, so an edit through PUT /api/v1/security-settings applies immediately rather than at the next restart.

Only the control plane has one, because it is the only origin a browser talks to: it serves the dashboard, and the Playground reaches the data plane through the /gw/* reverse proxy rather than calling the gateway directly. A gateway CORS layer would govern requests browsers do not make, and would need SecuritySettings in the snapshot to do it.

Behaviour:

• No origins configured is the default, and the same-origin deployment everyone runs. The middleware adds no headers at all and the request path is unchanged — CORS is not needed when the control plane serves both the dashboard and the API.
• Matching is exact against the configured list, modulo case and a trailing slash. There is no wildcard and no suffix match, so evil-dash.example.com and dash.example.com.evil.test do not inherit an entry for dash.example.com. validate_origin already rejects * at write time.
• A denied origin gets a normal response with no CORS headers; the browser enforces the block. Vary: Origin is set on every answer so a shared cache cannot serve one origin’s response to another.
• Preflights are answered by the middleware with 204 and never reach the router, since they carry no credentials.
• authorization, content-type, traceparent, tracestate and x-request-id are always allowed, unioned with whatever is configured. The trace headers are what let the dashboard’s spans join the gateway’s on a split-origin deployment; leaving them to configuration means browser trace propagation silently does not work until someone remembers them.
• x-request-id is exposed via Access-Control-Expose-Headers, so a browser can read the correlation id the API already echoes.

Egress policy (SSRF)

A provider’s api_base decides where the gateway sends traffic, so an admin surface that accepts an arbitrary one turns the proxy into an SSRF primitive aimed at whatever the gateway’s network position can reach.

rolter is built to run self-hosted and air-gapped, so upstreams legitimately live on loopback, RFC1918 and container networks — blanket-denying private destinations would break the core use case. What no legitimate LLM upstream needs is the link-local range, which is where cloud instance metadata lives (169.254.169.254, fe80::/10). That is denied by default; everything else is opt-in:

[egress]
block_link_local = true   # default — cloud instance metadata
block_loopback   = false  # sidecar / single-host deployments
block_private    = false  # on-prem clusters
allow_hosts      = []     # exact-host escape hatch

Enforced in two places: the control plane rejects a denied api_base at write time (400, so it never reaches the database), and snapshot validation re-checks it, so a bootstrap toml can’t smuggle one in either.

Config-time validation classifies IP literals only. Resolving hostnames there would make validation depend on live DNS — reintroducing the “one bad row freezes every gateway” failure mode described in config-and-hot-reload.md — and would be bypassable by DNS rebinding regardless.

So the same policy is enforced a third time, at connect time, by a custom resolver on every upstream client: whatever DNS actually returns is classified immediately before the connection is made. That covers the two cases config validation cannot see — a hostname like metadata.internal, and a name that resolved to a public address when it was configured but resolves to 169.254.169.254 at request time. A denied destination surfaces as a policy error naming the host, not an opaque connect failure.

A name that resolves to several addresses keeps the permitted ones: refusing the whole name would take down a legitimate multi-homed upstream. Only a name left with nothing is refused — which is exactly what rebinding to a denied address produces. The resolver reads the policy from a live handle, so a hot reload re-tunes enforcement without discarding pooled connections.

Control-plane input validation

Every control-plane mutation body is decoded through a SafeJson extractor rather than axum’s Json. Before the body is deserialized into its typed struct, every string in it — nested objects, arrays and object keys included — is screened for control characters, and the request is rejected with a 400 naming the offending field.

The concrete failure this prevents: Postgres text columns cannot store a NUL byte, so a field carrying one failed deep inside the store and surfaced as an unhandled 500 instead of input validation, violating the “bounded error, no unwrap/expect on a request path” invariant. Screening the whole C0/C1 range (and U+007F) also closes the log-injection vector a raw escape or newline would otherwise open in operator-facing logs.

Tab, newline and carriage return stay allowed — multi-line values are legitimate, a PEM CA bundle being the obvious one. Malformed JSON now also comes back in the same OpenAI-style error envelope as every other failure instead of axum’s default rejection body.

Open mode (no admin token)

With no ROLTER_ADMIN_TOKEN set, Principal short-circuits to Superadmin for every request: the management API and /internal/snapshot have no authentication step to fail. This is the zero-credential local-dev shape, and crates/rolter-control/src/open_mode.rs is what keeps it from being anything else. Before either listener is opened, it evaluates “is a token set” against every address about to be bound:

--internal-addr counts as a listener here: an exposed credential channel is no better than an exposed API, so either one alone is enough to refuse.

The decision also rides into the dashboard through window.__ROLTER_CONFIG__ (openMode: true), which renders a persistent banner. Without it the dashboard looks identical whether the control plane is gated or wide open, which is the property that made this dangerous rather than merely permissive (#970).

This is why ROLTER_CONTROL_HOST defaults to 127.0.0.1 rather than 0.0.0.0: containers and clusters set it explicitly, and by then they have a reason to have set a token too.

Control↔data-plane trust boundary

GET /internal/snapshot returns provider api_keys decrypted. That is by necessity — the data plane needs the upstream credential to authenticate to the provider — but it makes the snapshot channel a different trust boundary from the operator-facing management API, and it should be configured as one:

ROLTER_INTERNAL_TOKEN=...            # gates /internal/*, distinct from ROLTER_ADMIN_TOKEN
ROLTER_INTERNAL_ADDR=127.0.0.1:4002  # serves /internal/* on its own socket

With ROLTER_INTERNAL_TOKEN set, the operator admin token no longer opens the snapshot — only the gateway’s own credential does. With ROLTER_INTERNAL_ADDR set, /internal/* is not mounted on the public API router at all, so it is absent from the port the dashboard and management API are served from rather than merely gated on it. Bind it to loopback or a private interface. The gateway sends ROLTER_INTERNAL_TOKEN on snapshot polls, falling back to ROLTER_ADMIN_TOKEN.

Both are optional. Unset, the historical behavior holds — /internal/* shares the public listener and accepts the admin token — and the control plane logs a warning at startup saying so. The tenant-facing CRUD surface never returns a provider key in any configuration.

Two things this deliberately does not do. There is no mTLS between the planes: a shared secret over a private interface is comparable strength when the network path is already trusted, and mTLS adds certificate lifecycle to an air-gapped deployment. And the snapshot still carries plaintext rather than sealed ciphertext: envelope-passing would require the gateway to hold the KEK, moving the master secret onto every data-plane node, which is a worse trade for most deployments. Revisit both if the planes ever cross an untrusted network.

Wire transparency

• Outbound requests to upstream providers carry no rolter-identifying marks: no User-Agent, no added X-*/Via headers, no metadata injected into the JSON body, no marks in SSE framing. The only headers sent are functionally required ones — content-type, the provider’s auth header, and anthropic-version for Anthropic.
• Responses back to clients likewise gain no rolter-added headers.
• This is a tested guarantee: golden wire tests in rolter-proxy capture the raw outbound request head and fail on any unexpected header (see openai_wire_carries_no_rolter_signature).

External PII sanitization (#848)

The gateway can hand request content — and optionally response content — to a self-hosted de-identification service before it reaches a provider. Configured under [pii_sanitizer]; see user-docs/security/pii-sanitizer.mdx for the operator-facing contract.

The design property worth stating precisely: the placeholder→plaintext mapping never enters the gateway process. The sanitizer substitutes deterministic placeholders and returns an opaque restoration token; rolter holds the token and nothing else. There is therefore no log line, metric label, span attribute or cached body from which the original values can be recovered, and no in-memory table for a crash dump to expose.

Consequences that fall out of that choice:

• RestorationTicket is not printable. Its Display renders <restoration token redacted>, and the token is reachable only through token_for(&scope), which returns None unless the caller’s org/team/project/route match the scope the ticket was minted under. A token from one project cannot restore content in another.
• Restoration is opt-out, not opt-in-by-default. RestorationPolicy::Never is the default; CallerAuthorized honours x-rolter-pii-restore only because the policy allows it, never because the caller asked.
• The response leg never requests reversibility. Restoring provider-generated content would return the very data that leg exists to remove.
• A streamed response with an active response leg is refused (pii_streaming_unsupported, HTTP 400), mirroring guardrails.streaming_post_call. Correct streaming restoration would need either a network round trip per chunk — destroying TTFT — or the mapping held in-process, destroying the property above. streaming = "passthrough" waives the response leg instead.
• A malformed sanitizer reply is a failure, unlike the guardrail webhook’s decision parsing which defaults to allow. Defaulting would forward content the gateway believes is sanitized and is not.
• Restore runs after the cache store, so what is cached is what the upstream said. A later hit on the same entry is re-evaluated against its own request’s policy and ticket rather than replaying someone else’s restored plaintext.
• fail_open is the default and is the risk to monitor. It forwards unsanitized content when the service is down. rolter_pii_sanitizer_errors_total rising under fail_open means personal data is reaching providers; deployments where the sanitizer is a compliance control should set fail_closed.

A response-leg failure never fails the request even under fail_closed: the upstream call already happened and was already billed, so refusing to deliver would spend the caller’s money and return nothing. The counter records it.

Threat model (high level)

• Tenant isolation: virtual keys are scoped to a project; model allow-lists prevent access to unconfigured models; cache keys are namespaced to avoid cross-tenant cache poisoning.
• Abuse: RPM/TPM rate limits and budgets bound spend and load (roadmap enforcement).
• AuthZ: control-plane mutations are RBAC-checked and recorded in audit_log.
• Supply chain: cargo deny/advisory scanning in CI is a roadmap item.

Operational guidance

• Always set a strong ROLTER_KEK (e.g. openssl rand -hex 32) and rotate provider keys periodically.
• Run the control plane on a private network; expose only the gateway publicly.
• Back up Postgres; treat the master key as the most sensitive secret.