Skip to main content

👥📊 Team/Key Based Logging

Allow each key/team to use their own Langfuse Project / custom callbacks

This allows you to do the following

Team 1 -> Logs to Langfuse Project 1 
Team 2 -> Logs to Langfuse Project 2
Team 3 -> Disabled Logging (for GDPR compliance)

Team Based Logging

👉 Tutorial - Allow each team to use their own Langfuse Project / custom callbacks

Logging / Caching

Turn on/off logging and caching for a specific team id.

Example:

This config would send langfuse logs to 2 different langfuse projects, based on the team id

litellm_settings:
default_team_settings:
- team_id: my-secret-project
success_callback: ["langfuse"]
langfuse_public_key: os.environ/LANGFUSE_PUB_KEY_1 # Project 1
langfuse_secret: os.environ/LANGFUSE_PRIVATE_KEY_1 # Project 1
- team_id: ishaans-secret-project
success_callback: ["langfuse"]
langfuse_public_key: os.environ/LANGFUSE_PUB_KEY_2 # Project 2
langfuse_secret: os.environ/LANGFUSE_SECRET_2 # Project 2

Now, when you generate keys for this team-id

curl -X POST 'http://0.0.0.0:4000/key/generate' \
-H 'Authorization: Bearer sk-1234' \
-H 'Content-Type: application/json' \
-d '{"team_id": "ishaans-secret-project"}'

All requests made with these keys will log data to their team-specific logging. -->

[BETA] Team Logging via API

info

✨ This is an Enterprise only feature Get Started with Enterprise here

Set Callbacks Per Team

1. Set callback for team

We make a request to POST /team/{team_id}/callback to add a callback for

curl -X POST 'http:/localhost:4000/team/dbe2f686-a686-4896-864a-4c3924458709/callback' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-d '{
"callback_name": "langfuse",
"callback_type": "success",
"callback_vars": {
"langfuse_public_key": "pk",
"langfuse_secret_key": "sk_",
"langfuse_host": "https://cloud.langfuse.com"
}

}'
Supported Values
FieldSupported ValuesNotes
callback_name"langfuse", "gcs_bucket"Currently only supports "langfuse", "gcs_bucket"
callback_type"success", "failure", "success_and_failure"
callback_varsdict of callback settings
    langfuse_public_keystringRequired for Langfuse
    langfuse_secret_keystringRequired for Langfuse
    langfuse_hoststringOptional for Langfuse (defaults to https://cloud.langfuse.com)
    gcs_bucket_namestringRequired for GCS Bucket. Name of your GCS bucket
    gcs_path_service_accountstringRequired for GCS Bucket. Path to your service account json

2. Create key for team

All keys created for team dbe2f686-a686-4896-864a-4c3924458709 will log to langfuse project specified on Step 1. Set callback for team

curl --location 'http://0.0.0.0:4000/key/generate' \
--header 'Authorization: Bearer sk-1234' \
--header 'Content-Type: application/json' \
--data '{
"team_id": "dbe2f686-a686-4896-864a-4c3924458709"
}'

3. Make /chat/completion request for team

curl -i http://localhost:4000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk-KbUuE0WNptC0jXapyMmLBA" \
-d '{
"model": "gpt-4",
"messages": [
{"role": "user", "content": "Hello, Claude gm!"}
]
}'

Expect this to be logged on the langfuse project specified on Step 1. Set callback for team

Disable Logging for a Team

To disable logging for a specific team, you can use the following endpoint:

POST /team/{team_id}/disable_logging

This endpoint removes all success and failure callbacks for the specified team, effectively disabling logging.

Step 1. Disable logging for team

curl -X POST 'http://localhost:4000/team/YOUR_TEAM_ID/disable_logging' \
-H 'Authorization: Bearer YOUR_API_KEY'

Replace YOUR_TEAM_ID with the actual team ID

Response A successful request will return a response similar to this:

{
"status": "success",
"message": "Logging disabled for team YOUR_TEAM_ID",
"data": {
"team_id": "YOUR_TEAM_ID",
"success_callbacks": [],
"failure_callbacks": []
}
}

Step 2. Test it - /chat/completions

Use a key generated for team = team_id - you should see no logs on your configured success callback (eg. Langfuse)

curl -i http://localhost:4000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk-KbUuE0WNptC0jXapyMmLBA" \
-d '{
"model": "gpt-4",
"messages": [
{"role": "user", "content": "Hello, Claude gm!"}
]
}'

Debugging / Troubleshooting

  • Check active callbacks for team using GET /team/{team_id}/callback

Use this to check what success/failure callbacks are active for team=team_id

curl -X GET 'http://localhost:4000/team/dbe2f686-a686-4896-864a-4c3924458709/callback' \
-H 'Authorization: Bearer sk-1234'

Team Logging Endpoints

[BETA] Key Based Logging

Use the /key/generate or /key/update endpoints to add logging callbacks to a specific key.

info

✨ This is an Enterprise only feature Get Started with Enterprise here

How key based logging works:

  • If Key has no callbacks configured, it will use the default callbacks specified in the config.yaml file
  • If Key has callbacks configured, it will use the callbacks specified in the key
curl -X POST 'http://0.0.0.0:4000/key/generate' \
-H 'Authorization: Bearer sk-1234' \
-H 'Content-Type: application/json' \
-d '{
"metadata": {
"logging": [{
"callback_name": "langfuse", # "otel", "gcs_bucket"
"callback_type": "success", # "success", "failure", "success_and_failure"
"callback_vars": {
"langfuse_public_key": "os.environ/LANGFUSE_PUBLIC_KEY", # [RECOMMENDED] reference key in proxy environment
"langfuse_secret_key": "os.environ/LANGFUSE_SECRET_KEY", # [RECOMMENDED] reference key in proxy environment
"langfuse_host": "https://cloud.langfuse.com"
}
}]
}
}'


Help us improve this feature, by filing a ticket here

Check if key callbacks are configured correctly /key/health

Call /key/health with the key to check if the callback settings are configured correctly

Pass the key in the request header

curl -X POST "http://localhost:4000/key/health" \
-H "Authorization: Bearer <your-key>" \
-H "Content-Type: application/json"

Response when logging callbacks are setup correctly:

A key is healthy when the logging callbacks are setup correctly.

{
"key": "healthy",
"logging_callbacks": {
"callbacks": [
"gcs_bucket"
],
"status": "healthy",
"details": "No logger exceptions triggered, system is healthy. Manually check if logs were sent to ['gcs_bucket']"
}
}