For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.
LLM consumption
Overview of LLM provider integration and consumption patterns
Consume LLM services by setting up AI backends for your LLM providers.
About
Overview of supported LLM providers and their capabilities
API types
Supported LLM API endpoint types and route configurations
Providers
Virtual key management
Issue API keys with per-key token budgets and cost tracking (also known as virtual keys).
Manage API keys
Manage API keys for LLM provider authentication.
Content-based routing
Route requests to different LLM backends based on request body content, such as the requested model …
Control spend
Control cost with token budgets and spend limits to prevent unexpected bills and LLM misuse.
Transform requests
Dynamically compute and set LLM request fields using CEL expressions.
Guardrails
Protect LLM interactions with prompt guards that evaluate and filter requests and responses for …
Observe traffic
Get prompt logging, cost tracking, and an audit trail for LLM traffic.
Routing-based configuration for LLMs
Use traditional HTTP routing configuration for advanced use cases like path-based routing and …
