[Beta] Routing based on request metadata
Create routing rules based on request metadata.
Setup
Add the following to your litellm proxy config yaml file.
router_settings:
enable_tag_filtering: True # 👈 Key Change
1. Create a tag
On the LiteLLM UI, navigate to Experimental > Tag Management > Create Tag.
Create a tag called private-data and only select the allowed models for requests with this tag. Once created, you will see the tag in the Tag Management page.
2. Test Tag Routing
Now we will test the tag based routing rules.
2.1 Invalid model
This request will fail since we send tags=["private-data"] but the model gpt-5.6-terra is not in the allowed models for the private-data tag.
Here is an example sending the same request using the OpenAI Python SDK.
- OpenAI Python SDK
- cURL
from openai import OpenAI
client = OpenAI(
api_key="sk-<your-litellm-api-key>",
base_url="http://0.0.0.0:4000/v1/"
)
response = client.chat.completions.create(
model="gpt-5.6-terra",
messages=[
{"role": "user", "content": "Hello, how are you?"}
],
extra_body={
"tags": ["private-data"]
}
)
curl -L -X POST 'http://0.0.0.0:4000/v1/chat/completions' \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-d '{
"model": "gpt-5.6-terra",
"messages": [
{
"role": "user",
"content": "Hello, how are you?"
}
],
"tags": ["private-data"]
}'
2.2 Valid model
This request will succeed since we send tags=["private-data"] and the model us.anthropic.claude-sonnet-5 is in the allowed models for the private-data tag.
Here is an example sending the same request using the OpenAI Python SDK.
- OpenAI Python SDK
- cURL
from openai import OpenAI
client = OpenAI(
api_key="sk-<your-litellm-api-key>",
base_url="http://0.0.0.0:4000/v1/"
)
response = client.chat.completions.create(
model="us.anthropic.claude-sonnet-5",
messages=[
{"role": "user", "content": "Hello, how are you?"}
],
extra_body={
"tags": ["private-data"]
}
)
curl -L -X POST 'http://0.0.0.0:4000/v1/chat/completions' \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-d '{
"model": "us.anthropic.claude-sonnet-5",
"messages": [
{
"role": "user",
"content": "Hello, how are you?"
}
],
"tags": ["private-data"]
}'
