Skip to main content

๐Ÿ™‹โ€โ™‚๏ธ Customers

Track spend, set budgets for your customers.

Tracking Customer Creditโ€‹

1. Make LLM API call w/ Customer IDโ€‹

Make a /chat/completions call, pass 'user' - First call Works

curl -X POST 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer sk-1234' \ # ๐Ÿ‘ˆ YOUR PROXY KEY
--data ' {
"model": "azure-gpt-3.5",
"user": "ishaan3", # ๐Ÿ‘ˆ CUSTOMER ID
"messages": [
{
"role": "user",
"content": "what time is it"
}
]
}'

The customer_id will be upserted into the DB with the new spend.

If the customer_id already exists, spend will be incremented.

2. Get Customer Spendโ€‹

Call /customer/info to get a customer's all up spend

curl -X GET 'http://0.0.0.0:4000/customer/info?end_user_id=ishaan3' \ # ๐Ÿ‘ˆ CUSTOMER ID
-H 'Authorization: Bearer sk-1234' \ # ๐Ÿ‘ˆ YOUR PROXY KEY

Expected Response:

{
"user_id": "ishaan3",
"blocked": false,
"alias": null,
"spend": 0.001413,
"allowed_model_region": null,
"default_model": null,
"litellm_budget_table": null
}

Setting Customer Budgetsโ€‹

Set customer budgets (e.g. monthly budgets, tpm/rpm limits) on LiteLLM Proxy

Quick Startโ€‹

Create / Update a customer with budget

Create New Customer w/ budget

curl -X POST 'http://0.0.0.0:4000/customer/new'         
-H 'Authorization: Bearer sk-1234'
-H 'Content-Type: application/json'
-D '{
"user_id" : "my-customer-id",
"max_budget": "0", # ๐Ÿ‘ˆ CAN BE FLOAT
}'

Test it!

curl -X POST 'http://localhost:4000/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-D '{
"model": "mistral",
"messages": [
{
"role": "user",
"content": "What'\''s the weather like in Boston today?"
}
],
"user": "ishaan-jaff-48"
}

Assign Pricing Tiersโ€‹

Create and assign customers to pricing tiers.

1. Create a budgetโ€‹

  • Go to the 'Budgets' tab on the UI.
  • Click on '+ Create Budget'.
  • Create your pricing tier (e.g. 'my-free-tier' with budget $4). This means each user on this pricing tier will have a max budget of $4.

2. Assign Budget to Customerโ€‹

In your application code, assign budget when creating a new customer.

Just use the budget_id used when creating the budget. In our example, this is my-free-tier.

curl -X POST 'http://localhost:4000/customer/new' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-D '{
"user_id": "my-customer-id",
"budget_id": "my-free-tier" # ๐Ÿ‘ˆ KEY CHANGE
}

3. Test it!โ€‹

curl -X POST 'http://localhost:4000/customer/new' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-D '{
"user_id": "my-customer-id",
"budget_id": "my-free-tier" # ๐Ÿ‘ˆ KEY CHANGE
}