Skip to main content

Azure OpenAI

Azure OpenAI Service provides REST API access to OpenAI models such as GPT-5 and GPT-4o; LiteLLM calls your Azure deployments with the azure/ prefix through the same OpenAI-format API from the Python SDK or AI Gateway.

Overview​

PropertyDetails
DescriptionAzure OpenAI Service provides REST API access to OpenAI's powerful language models including o1, o1-mini, GPT-5, GPT-4o, GPT-4o mini, GPT-4 Turbo with Vision, GPT-4, GPT-3.5-Turbo, and Embeddings model series. Also supports Claude models via Azure Foundry.
Provider Route on LiteLLMazure/, azure/o_series/, azure/gpt5_series/. Claude models via Azure Foundry use the azure_ai/claude-* route instead.
Supported Operations/chat/completions, /responses, /completions, /embeddings, /audio/speech, /audio/transcriptions, /fine_tuning, /batches, /files, /images
Link to Provider DocAzure OpenAI ↗, Azure Foundry Claude ↗

API Keys, Params​

api_key, api_base, api_version etc can be passed directly to litellm.completion - see here or set as litellm.api_key params see here

import os
os.environ["AZURE_API_KEY"] = "" # "my-azure-api-key"
os.environ["AZURE_API_BASE"] = "" # "https://example-endpoint.openai.azure.com"
os.environ["AZURE_API_VERSION"] = "" # "2023-05-15"

# optional
os.environ["AZURE_AD_TOKEN"] = ""
os.environ["AZURE_API_TYPE"] = ""
Azure Foundry Claude Models

Claude models via Azure Foundry are routed through the azure_ai/ provider, not azure/. Use azure_ai/claude-* model names (e.g., azure_ai/claude-sonnet-5) with Azure authentication. See the Azure Anthropic documentation for details.

Usage - LiteLLM Python SDK​

Open In Colab

Completion - using .env variables​

from litellm import completion

## set ENV variables
os.environ["AZURE_API_KEY"] = ""
os.environ["AZURE_API_BASE"] = ""
os.environ["AZURE_API_VERSION"] = ""

# azure call
response = completion(
model = "azure/<your_deployment_name>",
messages = [{ "content": "Hello, how are you?","role": "user"}]
)

Completion - using api_key, api_base, api_version​

import litellm

# azure call
response = litellm.completion(
model = "azure/<your deployment name>", # model = azure/<your deployment name>
api_base = "", # azure api base
api_version = "", # azure api version
api_key = "", # azure api key
messages = [{"role": "user", "content": "good morning"}],
)

Completion - using azure_ad_token, api_base, api_version​

import litellm

# azure call
response = litellm.completion(
model = "azure/<your deployment name>", # model = azure/<your deployment name>
api_base = "", # azure api base
api_version = "", # azure api version
azure_ad_token="", # azure_ad_token
messages = [{"role": "user", "content": "good morning"}],
)

Usage - LiteLLM Proxy Server​

Here's how to call Azure OpenAI models with the LiteLLM Proxy Server

1. Save key in your environment​

export AZURE_API_KEY=""

2. Start the proxy​

model_list:
- model_name: gpt-6-luna
litellm_params:
model: azure/chatgpt-v-2
api_base: https://openai-gpt-4-test-v-1.openai.azure.com/
api_version: "2023-05-15"
api_key: os.environ/AZURE_API_KEY # The `os.environ/` prefix tells litellm to read this from the env.

3. Test it​

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--data ' {
"model": "gpt-6-luna",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}
'

Setting API Version​

You can set the api_version for Azure OpenAI in your proxy config.yaml in the following ways

Option 1: Per Model Configuration​

config.yaml
model_list:
- model_name: gpt-6.1-sol
litellm_params:
model: azure/my-gpt4-deployment
api_base: https://your-resource.openai.azure.com/
api_version: "2024-08-01-preview" # Set version per model
api_key: os.environ/AZURE_API_KEY

Azure OpenAI Chat Completion Models​

tip

We support ALL Azure models, just set model=azure/<your deployment name> as a prefix when sending litellm requests

Model NameFunction Call
o1-miniresponse = completion(model="azure/<your deployment name>", messages=messages)
o1-previewresponse = completion(model="azure/<your deployment name>", messages=messages)
gpt-5response = completion(model="azure/<your deployment name>", messages=messages)
gpt-4o-minicompletion('azure/<your deployment name>', messages)
gpt-4ocompletion('azure/<your deployment name>', messages)
gpt-4completion('azure/<your deployment name>', messages)
gpt-4-0314completion('azure/<your deployment name>', messages)
gpt-4-0613completion('azure/<your deployment name>', messages)
gpt-4-32kcompletion('azure/<your deployment name>', messages)
gpt-4-32k-0314completion('azure/<your deployment name>', messages)
gpt-4-32k-0613completion('azure/<your deployment name>', messages)
gpt-4-1106-previewcompletion('azure/<your deployment name>', messages)
gpt-4-0125-previewcompletion('azure/<your deployment name>', messages)
gpt-3.5-turbocompletion('azure/<your deployment name>', messages)
gpt-3.5-turbo-0301completion('azure/<your deployment name>', messages)
gpt-3.5-turbo-0613completion('azure/<your deployment name>', messages)
gpt-3.5-turbo-16kcompletion('azure/<your deployment name>', messages)
gpt-3.5-turbo-16k-0613completion('azure/<your deployment name>', messages)

Latest model aliases​

Since v1.109.0

azure/<family>-latest names follow a model family as LiteLLM moves them to newer models, so your application keeps one model string. Use them like any other Azure OpenAI model name. LiteLLM sends Azure OpenAI the current target model and bills at that model's price. See Latest model aliases for how aliases work and the full list across providers.

AliasModel family
azure/gpt-luna-latestGPT Luna
azure/gpt-sol-latestGPT Sol
azure/gpt-terra-latestGPT Terra
azure/gpt-astra-latestGPT Astra
azure/gpt-mini-latestGPT Mini

Azure OpenAI Vision Models​

Model NameFunction Call
gpt-4-visioncompletion(model="azure/<your deployment name>", messages=messages)
gpt-4ocompletion('azure/<your deployment name>', messages)

Usage​

import os 
from litellm import completion

os.environ["AZURE_API_KEY"] = "your-api-key"

# azure call
response = completion(
model = "azure/<your deployment name>",
messages=[
{
"role": "user",
"content": [
{
"type": "text",
"text": "What’s in this image?"
},
{
"type": "image_url",
"image_url": {
"url": "https://awsmp-logos.s3.amazonaws.com/seller-xw5kijmvmzasy/c233c9ade2ccb5491072ae232c814942.png"
}
}
]
}
],
)

Usage - with Azure Vision enhancements​

Note: Azure requires the base_url to be set with /extensions

Example

base_url="https://gpt-4-vision-resource.openai.azure.com/openai/deployments/gpt-4-vision/extensions"
# base_url="{azure_endpoint}/openai/deployments/{azure_deployment}/extensions"

Usage

import os 
from litellm import completion

os.environ["AZURE_API_KEY"] = "your-api-key"

# azure call
response = completion(
model="azure/gpt-4-vision",
timeout=5,
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "Whats in this image?"},
{
"type": "image_url",
"image_url": {
"url": "https://avatars.githubusercontent.com/u/29436595?v=4"
},
},
],
}
],
base_url="https://gpt-4-vision-resource.openai.azure.com/openai/deployments/gpt-4-vision/extensions",
api_key=os.getenv("AZURE_VISION_API_KEY"),
enhancements={"ocr": {"enabled": True}, "grounding": {"enabled": True}},
dataSources=[
{
"type": "AzureComputerVision",
"parameters": {
"endpoint": "https://gpt-4-vision-enhancement.cognitiveservices.azure.com/",
"key": os.environ["AZURE_VISION_ENHANCE_KEY"],
},
}
],
)

O-Series Models​

Azure OpenAI O-Series models are supported on LiteLLM.

LiteLLM routes any deployment name with o1 or o3 in the model name, to the O-Series transformation logic.

To set this explicitly, set model to azure/o_series/<your-deployment-name>.

Automatic Routing

import litellm

litellm.completion(model="azure/my-o3-deployment", messages=[{"role": "user", "content": "Hello, world!"}]) # Note: 'o3' in the deployment name

Explicit Routing

import litellm

litellm.completion(model="azure/o_series/my-random-deployment-name", messages=[{"role": "user", "content": "Hello, world!"}]) # Note: 'o_series/' in the deployment name

GPT-5 Models​

PropertyDetails
DescriptionAzure OpenAI GPT-5 models
Provider Route on LiteLLMazure/gpt5_series/<custom-name> or azure/gpt-5-deployment-name

LiteLLM supports using Azure GPT-5 models in one of the two ways:

  1. Explicit Routing: model = azure/gpt5_series/<deployment-name>. In this scenario the model onboarded to litellm follows the format model=azure/gpt5_series/<deployment-name>.
  2. Inferred Routing (If the azure deployment name contains gpt-5 in the name): model = azure/gpt-5.6-luna. In this scenario the model onboarded to litellm follows the format model=azure/gpt-5.6-luna.

Explicit Routing​

Use azure/gpt5_series/<deployment-name> for explicit GPT-5 model routing.

import litellm

response = litellm.completion(
model="azure/gpt5_series/my-gpt-5-deployment",
messages=[{"role": "user", "content": "Hello, world!"}]
)

Inferred Routing (gpt-5 in the deployment name)​

If your Azure deployment name contains gpt-5, LiteLLM automatically recognizes it as a GPT-5 model.

import litellm

# Deployment name contains 'gpt-5' - automatically inferred
response = litellm.completion(
model="azure/my-gpt-5-deployment",
messages=[{"role": "user", "content": "Hello, world!"}]
)

Azure Audio Model​

from litellm import completion
import os

os.environ["AZURE_API_KEY"] = ""
os.environ["AZURE_API_BASE"] = ""
os.environ["AZURE_API_VERSION"] = ""

response = completion(
model="azure/azure-openai-4o-audio",
messages=[
{
"role": "user",
"content": "I want to try out speech to speech"
}
],
modalities=["text","audio"],
audio={"voice": "alloy", "format": "wav"}
)

print(response)

Azure Realtime Translation​

For live speech-to-speech translation, add azure/gpt-realtime-translate with your api_base and api_key, then connect to /v1/realtime/translations on the proxy. See Realtime translation for the config, a WebSocket client, WebRTC, and pricing.

Azure Instruct Models​

Use model="azure_text/<your-deployment>"

Model NameFunction Call
gpt-3.5-turbo-instructresponse = completion(model="azure_text/<your deployment name>", messages=messages)
gpt-3.5-turbo-instruct-0914response = completion(model="azure_text/<your deployment name>", messages=messages)
import litellm

## set ENV variables
os.environ["AZURE_API_KEY"] = ""
os.environ["AZURE_API_BASE"] = ""
os.environ["AZURE_API_VERSION"] = ""

response = litellm.completion(
model="azure_text/<your-deployment-name",
messages=[{"role": "user", "content": "What is the weather like in Boston?"}]
)

print(response)

Authentication​

Entra ID - use azure_ad_token​

This is a walkthrough on how to use Azure Active Directory Tokens - Microsoft Entra ID to make litellm.completion() calls.

Note: You can follow the same process below to use Azure Active Directory Tokens for all other Azure endpoints (e.g., chat, embeddings, image, audio, etc.) with LiteLLM.

Step 1 - Download Azure CLI Installation instructions: https://learn.microsoft.com/en-us/cli/azure/install-azure-cli

brew update && brew install azure-cli

Step 2 - Sign in using az

az login --output table

Step 3 - Generate azure ad token

az account get-access-token --resource https://cognitiveservices.azure.com

In this step you should see an accessToken generated

{
"accessToken": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IjlHbW55RlBraGMzaE91UjIybXZTdmduTG83WSIsImtpZCI6IjlHbW55RlBraGMzaE91UjIybXZTdmduTG83WSJ9",
"expiresOn": "2023-11-14 15:50:46.000000",
"expires_on": 1700005846,
"subscription": "db38de1f-4bb3..",
"tenant": "bdfd79b3-8401-47..",
"tokenType": "Bearer"
}

Step 4 - Make litellm.completion call with Azure AD token

Set azure_ad_token = accessToken from step 3 or set os.environ['AZURE_AD_TOKEN']

response = litellm.completion(
model = "azure/<your deployment name>", # model = azure/<your deployment name>
api_base = "", # azure api base
api_version = "", # azure api version
azure_ad_token="", # your accessToken from step 3
messages = [{"role": "user", "content": "good morning"}],
)

Entra ID - use tenant_id, client_id, client_secret​

Here is an example of setting up tenant_id, client_id, client_secret in your litellm proxy config.yaml

model_list:
- model_name: gpt-6-luna
litellm_params:
model: azure/chatgpt-v-2
api_base: https://openai-gpt-4-test-v-1.openai.azure.com/
api_version: "2023-05-15"
tenant_id: os.environ/AZURE_TENANT_ID
client_id: os.environ/AZURE_CLIENT_ID
client_secret: os.environ/AZURE_CLIENT_SECRET
azure_scope: os.environ/AZURE_SCOPE # defaults to "https://cognitiveservices.azure.com/.default"

Test it

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--data ' {
"model": "gpt-6-luna",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}
'

Example video of using tenant_id, client_id, client_secret with LiteLLM Proxy Server

Entra ID - use client_id, username, password​

Here is an example of setting up client_id, azure_username, azure_password in your litellm proxy config.yaml

model_list:
- model_name: gpt-6-luna
litellm_params:
model: azure/chatgpt-v-2
api_base: https://openai-gpt-4-test-v-1.openai.azure.com/
api_version: "2023-05-15"
client_id: os.environ/AZURE_CLIENT_ID
azure_username: os.environ/AZURE_USERNAME
azure_password: os.environ/AZURE_PASSWORD
azure_scope: os.environ/AZURE_SCOPE # defaults to "https://cognitiveservices.azure.com/.default"

Test it

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--data ' {
"model": "gpt-6-luna",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}
'

Azure AD Token Refresh - DefaultAzureCredential​

Use this if you want to use Azure DefaultAzureCredential for Authentication on your requests. DefaultAzureCredential automatically discovers and uses available Azure credentials from multiple sources.

Option 1: Explicit DefaultAzureCredential (Recommended)

from litellm import completion
from azure.identity import DefaultAzureCredential, get_bearer_token_provider

# DefaultAzureCredential automatically discovers credentials from:
# - Environment variables (AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_TENANT_ID)
# - Managed Identity (AKS, Azure VMs, etc.)
# - Azure CLI credentials
# - And other Azure identity sources
token_provider = get_bearer_token_provider(DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default")

response = completion(
model = "azure/<your deployment name>", # model = azure/<your deployment name>
api_base = "", # azure api base
api_version = "", # azure api version
azure_ad_token_provider=token_provider,
messages = [{"role": "user", "content": "good morning"}],
)

Option 2: LiteLLM Auto-Fallback to DefaultAzureCredential

import litellm

# Enable automatic fallback to DefaultAzureCredential
litellm.enable_azure_ad_token_refresh = True

response = litellm.completion(
model = "azure/<your deployment name>",
api_base = "",
api_version = "",
messages = [{"role": "user", "content": "good morning"}],
)

Azure Batches API​

PropertyDetails
DescriptionAzure OpenAI Batches API
custom_llm_provider on LiteLLMazure/
Supported Operations/v1/batches, /v1/files
Azure OpenAI Batches APIAzure OpenAI Batches API ↗
Cost Tracking, Logging Support✅ LiteLLM will log, track cost for Batch API Requests

Quick Start​

Just add the azure env vars to your environment.

export AZURE_API_KEY=""
export AZURE_API_BASE=""

1. Upload a File

from openai import OpenAI

# Initialize the client
client = OpenAI(
base_url="http://localhost:4000",
api_key="your-api-key"
)

batch_input_file = client.files.create(
file=open("mydata.jsonl", "rb"),
purpose="batch",
extra_headers={"custom-llm-provider": "azure"}
)
file_id = batch_input_file.id

Example File Format

{"custom_id": "task-0", "method": "POST", "url": "/chat/completions", "body": {"model": "REPLACE-WITH-MODEL-DEPLOYMENT-NAME", "messages": [{"role": "system", "content": "You are an AI assistant that helps people find information."}, {"role": "user", "content": "When was Microsoft founded?"}]}}
{"custom_id": "task-1", "method": "POST", "url": "/chat/completions", "body": {"model": "REPLACE-WITH-MODEL-DEPLOYMENT-NAME", "messages": [{"role": "system", "content": "You are an AI assistant that helps people find information."}, {"role": "user", "content": "When was the first XBOX released?"}]}}
{"custom_id": "task-2", "method": "POST", "url": "/chat/completions", "body": {"model": "REPLACE-WITH-MODEL-DEPLOYMENT-NAME", "messages": [{"role": "system", "content": "You are an AI assistant that helps people find information."}, {"role": "user", "content": "What is Altair Basic?"}]}}

2. Create a Batch Request

batch = client.batches.create( # re use client from above
input_file_id=file_id,
endpoint="/v1/chat/completions",
completion_window="24h",
metadata={"description": "My batch job"},
extra_headers={"custom-llm-provider": "azure"}
)

3. Retrieve a Batch

retrieved_batch = client.batches.retrieve(
batch.id,
extra_headers={"custom-llm-provider": "azure"}
)

4. Cancel a Batch

cancelled_batch = client.batches.cancel(
batch.id,
extra_headers={"custom-llm-provider": "azure"}
)

5. List Batches

client.batches.list(extra_headers={"custom-llm-provider": "azure"})

Health Check Azure Batch models​

[BETA] Loadbalance Multiple Azure Deployments​

In your config.yaml, set enable_loadbalancing_on_batch_endpoints: true

model_list:
- model_name: "batch-model"
litellm_params:
model: "azure/gpt-5.6-luna"
api_key: os.environ/AZURE_API_KEY
api_base: os.environ/AZURE_API_BASE
model_info:
mode: batch

litellm_settings:
enable_loadbalancing_on_batch_endpoints: true # KEY CHANGE

Note: This works on {PROXY_BASE_URL}/v1/files and {PROXY_BASE_URL}/v1/batches. Note: Response is in the OpenAI-format.

  1. Upload a file

Just set model: batch-model in your .jsonl.

curl http://localhost:4000/v1/files \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-F purpose="batch" \
-F file="@mydata.jsonl"

Example File

Note: model should be your azure deployment name.

{"custom_id": "task-0", "method": "POST", "url": "/chat/completions", "body": {"model": "batch-model", "messages": [{"role": "system", "content": "You are an AI assistant that helps people find information."}, {"role": "user", "content": "When was Microsoft founded?"}]}}
{"custom_id": "task-1", "method": "POST", "url": "/chat/completions", "body": {"model": "batch-model", "messages": [{"role": "system", "content": "You are an AI assistant that helps people find information."}, {"role": "user", "content": "When was the first XBOX released?"}]}}
{"custom_id": "task-2", "method": "POST", "url": "/chat/completions", "body": {"model": "batch-model", "messages": [{"role": "system", "content": "You are an AI assistant that helps people find information."}, {"role": "user", "content": "What is Altair Basic?"}]}}

Expected Response (OpenAI-compatible)

{"id":"file-f0be81f654454113a922da60acb0eea6",...}
  1. Create a batch
curl http://0.0.0.0:4000/v1/batches \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"input_file_id": "file-f0be81f654454113a922da60acb0eea6",
"endpoint": "/v1/chat/completions",
"completion_window": "24h",
"model": "batch-model"
}'

Expected Response:

{"id":"batch_94e43f0a-d805-477d-adf9-bbb9c50910ed",...}
  1. Retrieve a batch
curl http://0.0.0.0:4000/v1/batches/batch_94e43f0a-d805-477d-adf9-bbb9c50910ed \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-H "Content-Type: application/json" \

Expected Response:

{"id":"batch_94e43f0a-d805-477d-adf9-bbb9c50910ed",...}
  1. List batch
curl http://0.0.0.0:4000/v1/batches?limit=2 \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-H "Content-Type: application/json"

Expected Response:

{"data":[{"id":"batch_R3V...}

Advanced​

Azure API Load-Balancing​

Use this if you're trying to load-balance across multiple Azure/OpenAI deployments.

Router prevents failed requests, by picking the deployment which is below rate-limit and has the least amount of tokens used.

In production, Router connects to a Redis Cache to track usage across multiple deployments.

Quick Start​

uv add litellm
from litellm import Router

model_list = [{ # list of model deployments
"model_name": "gpt-6-luna", # openai model name
"litellm_params": { # params for litellm completion/embedding call
"model": "azure/chatgpt-v-2",
"api_key": os.getenv("AZURE_API_KEY"),
"api_version": os.getenv("AZURE_API_VERSION"),
"api_base": os.getenv("AZURE_API_BASE")
},
"tpm": 240000,
"rpm": 1800
}, {
"model_name": "gpt-6-luna", # openai model name
"litellm_params": { # params for litellm completion/embedding call
"model": "azure/chatgpt-functioncalling",
"api_key": os.getenv("AZURE_API_KEY"),
"api_version": os.getenv("AZURE_API_VERSION"),
"api_base": os.getenv("AZURE_API_BASE")
},
"tpm": 240000,
"rpm": 1800
}, {
"model_name": "gpt-6-luna", # openai model name
"litellm_params": { # params for litellm completion/embedding call
"model": "gpt-6-luna",
"api_key": os.getenv("OPENAI_API_KEY"),
},
"tpm": 1000000,
"rpm": 9000
}]

router = Router(model_list=model_list)

# openai.chat.completions.create replacement
response = router.completion(model="gpt-6-luna",
messages=[{"role": "user", "content": "Hey, how's it going?"}])

print(response)

Redis Queue​

router = Router(model_list=model_list, 
redis_host=os.getenv("REDIS_HOST"),
redis_password=os.getenv("REDIS_PASSWORD"),
redis_port=os.getenv("REDIS_PORT"))

print(response)

Tool Calling / Function Calling​

See a detailed walthrough of parallel function calling with litellm here

# set Azure env variables
import os
import litellm
import json

os.environ['AZURE_API_KEY'] = "" # litellm reads AZURE_API_KEY from .env and sends the request
os.environ['AZURE_API_BASE'] = "https://openai-gpt-4-test-v-1.openai.azure.com/"
os.environ['AZURE_API_VERSION'] = "2023-07-01-preview"

tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["location"],
},
},
}
]

response = litellm.completion(
model="azure/chatgpt-functioncalling", # model = azure/<your-azure-deployment-name>
messages=[{"role": "user", "content": "What's the weather like in San Francisco, Tokyo, and Paris?"}],
tools=tools,
tool_choice="auto", # auto is default, but we'll be explicit
)
print("\nLLM Response1:\n", response)
response_message = response.choices[0].message
tool_calls = response.choices[0].message.tool_calls
print("\nTool Choice:\n", tool_calls)

Spend Tracking for Azure OpenAI Models (PROXY)​

Set base model for cost tracking azure image-gen call

Image Generation​

model_list: 
- model_name: dall-e-3
litellm_params:
model: azure/dall-e-3-test
api_version: 2023-06-01-preview
api_base: https://openai-gpt-4-test-v-1.openai.azure.com/
api_key: os.environ/AZURE_API_KEY
base_model: dall-e-3 # set dall-e-3 as base model
model_info:
mode: image_generation

Chat Completions / Embeddings​

Problem: Azure returns gpt-4 in the response when azure/gpt-4-1106-preview is used. This leads to inaccurate cost tracking

Solution ✅ : Set base_model on your config so litellm uses the correct model for calculating azure cost

Get the base model name from here

Example config with base_model

model_list:
- model_name: azure-gpt-3.5
litellm_params:
model: azure/chatgpt-v-2
api_base: os.environ/AZURE_API_BASE
api_key: os.environ/AZURE_API_KEY
api_version: "2023-07-01-preview"
model_info:
base_model: azure/gpt-6.1-sol