📄️ Router - Load Balancing
LiteLLM manages:
📄️ [BETA] Request Prioritization
Beta feature. Use for testing only.
📄️ Proxy - Load Balancing
Load balance multiple instances of the same model
📄️ Fallbacks
If a call fails after num_retries, fallback to another model group.
📄️ Timeouts
The timeout set in router is for the entire length of the call, and is passed down to the completion() call level as well.
📄️ Tag Based Routing
Route requests based on tags.
📄️ Budget Routing
LiteLLM Supports setting the following budgets:
📄️ Provider specific Wildcard routing
Proxy all models from a provider