rolter documentation

Start here. rolter is a high-performance OpenAI/Anthropic-compatible AI gateway and load balancer (Rust data plane + control plane, shadcn/ui dashboard).

Architecture

• Overview — system shape, crates, data/control plane split
• Load balancing — strategies and the cache-aware design
• Caching — response cache and KV-cache affinity
• Config & hot reload — reload-free updates
• Data model — tenancy, keys, pricing, budgets
• RBAC & auth — roles, virtual keys, SSO/LDAP roadmap
• Security — secret handling, threat model
• Observability — metrics, tracing, logs
• Performance — hot-path principles and targets

Decisions

• ADRs / decision log

API

• OpenAI & Anthropic surface

Development

• Setup
• Testing
• Contributing
• Commit conventions
• Packaging (uv / cargo / docker)

Deployment

• Zero-config quickstart — start with no keys, add providers at runtime
• Configuration reference
• Custom CA bundles — trust private upstream PKI without disabling TLS verification
• Docker

Planning

• Roadmap
• TODO