rolter documentation

Start here. rolter is a high-performance OpenAI/Anthropic-compatible AI gateway and load balancer (Rust data plane + control plane, shadcn/ui dashboard).

Architecture

• Overview — system shape, crates, data/control plane split
• Load balancing — strategies and the cache-aware design
• Caching — response cache and KV-cache affinity
• Provider prompt caching — portable Anthropic prompt-cache controls
• Multi-region deployment — active-active/passive state and failover topology
• Config & hot reload — reload-free updates
• Data model — tenancy, keys, pricing, budgets
• RBAC & auth — roles, virtual keys, membership sources
• Invitations — one-time onboarding links for deployments with no IdP
• Single sign-on (OIDC) — optional OIDC login and group→role mapping
• MCP OAuth — MCP servers, consent grants, sealed token sessions
• SCIM provisioning — IdP-driven user lifecycle and provisioning tokens
• Security — secret handling, threat model
• Observability — metrics, tracing, logs
• Performance — hot-path principles and targets

Decisions

• ADRs / decision log

API

• OpenAI & Anthropic surface

Development

• Setup
• Testing
• Contributing
• Commit conventions
• Packaging (uv / cargo / docker)

Deployment

• Zero-config quickstart — start with no keys, add providers at runtime
• Configuration reference
• Custom CA bundles — trust private upstream PKI without disabling TLS verification
• Docker

Planning

• Roadmap
• TODO