Azure AI Foundry Agents

Call Azure AI Foundry Agents in the OpenAI Request/Response format.

Property	Details
Description	Azure AI Foundry Agents provides hosted agent runtimes that can execute agentic workflows with foundation models, tools, and code interpreters.
Provider Route on LiteLLM	azure_ai/agents/{AGENT_NAME} or azure_ai/agents/{ASSISTANT_ID}
Provider Doc	Azure AI Foundry Agents ↗

Authentication

Classic agents (an asst_ ID) require Azure AD authentication, not API keys. Agents reached by name through the project Responses API take a project API key too, as Foundry agents through the Responses API shows. You can authenticate with Azure AD using:

Option 1: Service Principal (Recommended for Production)

Set these environment variables:

export AZURE_TENANT_ID="your-tenant-id"
export AZURE_CLIENT_ID="your-client-id"
export AZURE_CLIENT_SECRET="your-client-secret"

LiteLLM will automatically obtain an Azure AD token using these credentials.

Option 2: Azure AD Token (Manual)

Pass a token directly via api_key:

# Get token via Azure CLI
az account get-access-token --resource "https://ai.azure.com" --query accessToken -o tsv

Required Azure Role

Your Service Principal or user must have the Azure AI Developer or Azure AI User role on your Azure AI Foundry project.

To assign via Azure CLI:

az role assignment create \
  --assignee-object-id "<service-principal-object-id>" \
  --assignee-principal-type "ServicePrincipal" \
  --role "Azure AI Developer" \
  --scope "/subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.CognitiveServices/accounts/<resource>"

Or add via Azure AI Foundry Portal → Your Project → Project users → + New user.

Quick Start

Model Format to LiteLLM

To call an Azure AI Foundry Agent through LiteLLM, use the following model format.

Here the model=azure_ai/agents/ tells LiteLLM to call the Azure AI Foundry Agent Service API.

Model Format to LiteLLM

azure_ai/agents/{AGENT_ID}

Example:

• azure_ai/agents/math-agent, an agent from the current Foundry portal, by name. See Foundry agents through the Responses API.
• azure_ai/agents/math-agent:2, the same agent pinned to version 2.
• azure_ai/agents/asst_abc123, a classic Assistants agent.

You can find the agent name, or the ID of a classic agent, in your Azure AI Foundry portal under Agents.

LiteLLM Python SDK

Basic Agent Completion

import litellm

# Make a completion request to your Azure AI Foundry Agent
# Uses AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET env vars for auth
response = litellm.completion(
    model="azure_ai/agents/asst_abc123",
    messages=[
        {
            "role": "user", 
            "content": "Explain machine learning in simple terms"
        }
    ],
    api_base="https://your-resource.services.ai.azure.com/api/projects/your-project",
)

print(response.choices[0].message.content)
print(f"Usage: {response.usage}")

Streaming Agent Responses

import litellm

# Stream responses from your Azure AI Foundry Agent
response = await litellm.acompletion(
    model="azure_ai/agents/asst_abc123",
    messages=[
        {
            "role": "user",
            "content": "What are the key principles of software architecture?"
        }
    ],
    api_base="https://your-resource.services.ai.azure.com/api/projects/your-project",
    stream=True,
)

async for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

LiteLLM Proxy

1. Configure your model in config.yaml

config.yaml

LiteLLM Proxy Configuration

model_list:
  - model_name: azure-agent-1
    litellm_params:
      model: azure_ai/agents/asst_abc123
      api_base: https://your-resource.services.ai.azure.com/api/projects/your-project
      # Service Principal auth (recommended)
      tenant_id: os.environ/AZURE_TENANT_ID
      client_id: os.environ/AZURE_CLIENT_ID
      client_secret: os.environ/AZURE_CLIENT_SECRET

  - model_name: azure-agent-math-tutor
    litellm_params:
      model: azure_ai/agents/asst_def456
      api_base: https://your-resource.services.ai.azure.com/api/projects/your-project
      # Or pass Azure AD token directly
      api_key: os.environ/AZURE_AD_TOKEN

2. Start the LiteLLM Proxy

Start LiteLLM Proxy

litellm --config config.yaml

3. Make requests to your Azure AI Foundry Agents

Curl

Basic Agent Request

curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -d '{
    "model": "azure-agent-1",
    "messages": [
      {
        "role": "user", 
        "content": "Summarize the main benefits of cloud computing"
      }
    ]
  }'

Streaming Agent Request

curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -d '{
    "model": "azure-agent-math-tutor",
    "messages": [
      {
        "role": "user",
        "content": "What is 25 * 4?"
      }
    ],
    "stream": true
  }'

OpenAI Python SDK

Using OpenAI SDK with LiteLLM Proxy

from openai import OpenAI

# Initialize client with your LiteLLM proxy URL
client = OpenAI(
    base_url="http://localhost:4000",
    api_key="your-litellm-api-key"
)

# Make a completion request to your Azure AI Foundry Agent
response = client.chat.completions.create(
    model="azure-agent-1",
    messages=[
      {
        "role": "user",
        "content": "What are best practices for API design?"
      }
    ]
)

print(response.choices[0].message.content)

Streaming with OpenAI SDK

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:4000", 
    api_key="your-litellm-api-key"
)

# Stream Agent responses
stream = client.chat.completions.create(
    model="azure-agent-math-tutor",
    messages=[
      {
        "role": "user",
        "content": "Explain the Pythagorean theorem"
      }
    ],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end="")

Foundry agents through the Responses API

Since v1.109.0

Agents created in the current Microsoft Foundry portal have a name instead of an asst_ ID. LiteLLM sends azure_ai/agents/<agent name> to the project's /openai/v1/responses route with an agent_reference, so the agent answers on /v1/responses, /v1/chat/completions, and /v1/messages, streamed token by token with "stream": true. Add :<version> to pin a version. Without it, Foundry runs the agent's latest version. An ID that starts with asst_ still goes to the classic Assistants route.

api_base is the project endpoint, https://<account>.services.ai.azure.com/api/projects/<project>. LiteLLM sends Authorization: Bearer to it, so a project API key in api_key works as well as Entra ID credentials. A service principal's token is minted for the https://ai.azure.com/.default scope unless azure_scope on the entry or the AZURE_SCOPE env var sets another one. An AZURE_AI_API_KEY env var wins over the service principal fields, as on every other azure_ai route.

config.yaml

model_list:
  - model_name: math-agent
    litellm_params:
      model: azure_ai/agents/math-agent
      api_base: https://<account>.services.ai.azure.com/api/projects/<project>
      tenant_id: os.environ/AZURE_TENANT_ID
      client_id: os.environ/AZURE_CLIENT_ID
      client_secret: os.environ/AZURE_CLIENT_SECRET

  - model_name: math-agent-v2
    litellm_params:
      model: azure_ai/agents/math-agent:2
      api_base: https://<account>.services.ai.azure.com/api/projects/<project>
      api_key: os.environ/FOUNDRY_PROJECT_API_KEY

Call the agent through the Responses API:

curl http://localhost:4000/v1/responses \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "math-agent", "input": "What is 2 + 2?"}'

Or through chat completions:

curl http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "math-agent", "messages": [{"role": "user", "content": "What is 2 + 2?"}]}'

The agent owns its model settings, so Foundry refuses temperature, top_p, tools, text, and reasoning on an agent call. LiteLLM drops them under drop_params: true (in litellm_settings, on the entry, or in the request) and answers 400 without it. The params Foundry accepts with an agent, like max_output_tokens, tool_choice, store, metadata, and truncation, pass through.

A system prompt (instructions, a chat system message, or a Messages system) reaches the agent as a leading system message next to the agent's own instructions. Foundry stores it with the turn, so a later turn that sets previous_response_id already follows it, and instructions on that turn answers 400 without drop_params and is dropped with it.

The entry always calls its own agent. A request whose agent_reference (or the deprecated agent) names another agent answers 400 without drop_params, and with it the entry's agent wins, so a key that can call one entry cannot reach another agent in the project. The request may pick a version of the entry's agent unless the entry pins one. Responses come back with instructions set to null, so callers cannot read the system prompt the agent was given in Foundry.

/v1/responses/compact on an agent answers 400 before any Foundry call, because the project's compact endpoint takes a model deployment, not an agent. Call compact on a model entry for that deployment instead.

Spend is priced from the model Foundry reports in the response. To charge a fixed price per agent, set input_cost_per_token and output_cost_per_token on the entry.

Environment Variables

Variable	Description
AZURE_TENANT_ID	Azure AD tenant ID for Service Principal auth
AZURE_CLIENT_ID	Application (client) ID of your Service Principal
AZURE_CLIENT_SECRET	Client secret for your Service Principal

export AZURE_TENANT_ID="your-tenant-id"
export AZURE_CLIENT_ID="your-client-id"
export AZURE_CLIENT_SECRET="your-client-secret"

Conversation Continuity (Thread Management)

Classic Azure AI Foundry Agents use threads to maintain conversation context. LiteLLM automatically manages threads for you, but you can also pass an existing thread ID to continue a conversation. An agent reached by name continues a conversation with previous_response_id on /v1/responses instead.

Continuing a Conversation

import litellm

# First message creates a new thread
response1 = await litellm.acompletion(
    model="azure_ai/agents/asst_abc123",
    messages=[{"role": "user", "content": "My name is Alice"}],
    api_base="https://your-resource.services.ai.azure.com/api/projects/your-project",
)

# Get the thread_id from the response
thread_id = response1._hidden_params.get("thread_id")

# Continue the conversation using the same thread
response2 = await litellm.acompletion(
    model="azure_ai/agents/asst_abc123",
    messages=[{"role": "user", "content": "What's my name?"}],
    api_base="https://your-resource.services.ai.azure.com/api/projects/your-project",
    thread_id=thread_id,  # Pass the thread_id to continue conversation
)

print(response2.choices[0].message.content)  # Should mention "Alice"

Provider-specific Parameters

Azure AI Foundry Agents support additional parameters that can be passed to customize the agent invocation.

SDK

Using Agent-specific parameters

from litellm import completion

response = litellm.completion(
    model="azure_ai/agents/asst_abc123",
    messages=[
        {
            "role": "user",
            "content": "Analyze this data and provide insights",
        }
    ],
    api_base="https://your-resource.services.ai.azure.com/api/projects/your-project",
    thread_id="thread_abc123",  # Optional: Continue existing conversation
    instructions="Be concise and focus on key insights",  # Optional: Override agent instructions
)

Proxy

LiteLLM Proxy Configuration with Parameters

model_list:
  - model_name: azure-agent-analyst
    litellm_params:
      model: azure_ai/agents/asst_abc123
      api_base: https://your-resource.services.ai.azure.com/api/projects/your-project
      tenant_id: os.environ/AZURE_TENANT_ID
      client_id: os.environ/AZURE_CLIENT_ID
      client_secret: os.environ/AZURE_CLIENT_SECRET
      instructions: "Be concise and focus on key insights"

Available Parameters

Parameter	Type	Description
thread_id	string	Optional thread ID to continue an existing conversation
instructions	string	Optional instructions to override the agent's default instructions for this run

LiteLLM A2A Gateway

You can also connect to Azure AI Foundry Agents through LiteLLM's A2A (Agent-to-Agent) Gateway UI. This provides a visual way to register and test agents without writing code.

1. Navigate to Agents

From the sidebar, click "Agents" to open the agent management page, then click "+ Add New Agent".

2. Select Azure AI Foundry Agent Type

Click "A2A Standard" to see available agent types, then select "Azure AI Foundry".

3. Configure the Agent

Fill in the following fields:

Agent Name

Enter a friendly agent name - callers will see this name as the agent available.

Agent ID

Get the Agent ID from your Azure AI Foundry portal:

1. Go to https://ai.azure.com/ and click "Agents"

2. Copy the name of the agent you want to add (e.g., math-agent, or math-agent:2 to pin a version). A classic agent shows an ID like asst_hbnoK9BOCcHhC3lC4MDroVGG instead, which works too

3. Paste the Agent ID in LiteLLM - this tells LiteLLM which agent to invoke on Azure Foundry

Azure AI API Base

Get your API base URL from Azure AI Foundry:

1. Go to https://ai.azure.com/ and click "Overview"
2. Under libraries, select Microsoft Foundry
3. Get your endpoint - it should look like https://<domain>.services.ai.azure.com/api/projects/<project-name>

4. Paste the URL in LiteLLM

Authentication

Add your Azure AD credentials for authentication:

• Azure Tenant ID
• Azure Client ID
• Azure Client Secret

Click "Create Agent" to save.

4. Test in Playground

Go to "Playground" in the sidebar to test your agent.

Change the endpoint type to /v1/a2a/message/send.

5. Select Your Agent and Send a Message

Pick your Azure AI Foundry agent from the dropdown and send a test message.

The agent responds with its capabilities. You can now interact with your Azure AI Foundry agent through the A2A protocol.

Further Reading

• Azure AI Foundry Agents Documentation
• Create Thread and Run API Reference
• A2A Agent Gateway
• A2A Cost Tracking

Foundry agents over A2A

Agents created in the current Microsoft Foundry portal do not speak the Assistants API. LiteLLM reaches them by name as a model, as Foundry agents through the Responses API shows. They also expose an A2A endpoint, and LiteLLM can register that as a plain A2A agent that authenticates with Microsoft Entra ID

The endpoint is https://<account>.services.ai.azure.com/api/projects/<project>/agents/<agent>/endpoint/protocols/a2a. Foundry serves its agent card at agentCard/v1.0 under it rather than at the well-known paths, and accepts only an Entra bearer for the https://ai.azure.com/.default scope, so the entry sets agent_card_path and the service principal fields:

config.yaml

agents:
  - agent_name: foundry-agent
    agent_card_params:
      name: "Foundry Agent"
      url: "https://<account>.services.ai.azure.com/api/projects/<project>/agents/<agent>/endpoint/protocols/a2a"
      protocolVersion: "1.0"
      capabilities:
        streaming: false
    litellm_params:
      agent_card_path: agentCard/v1.0
      tenant_id: os.environ/AZURE_TENANT_ID
      client_id: os.environ/AZURE_CLIENT_ID
      client_secret: os.environ/AZURE_CLIENT_SECRET

tenant_id, client_id and client_secret are the names the azure_ai provider uses. A bearer you minted yourself goes in azure_ad_token (an oidc/ federated token there needs tenant_id + client_id next to it), a user login in client_id + azure_username + azure_password, and azure_scope overrides the scope. Only the credentials on the agent are used; the process-wide AZURE_* variables are never read for an A2A agent. Without agent_card_path, discovery still reaches agentCard/v1.0 after the two well-known paths fail, at the cost of two extra round trips on every call

Foundry answers message/send with a submitted task unless the request asks for a blocking send, and it has no message/stream. The capabilities.streaming: false line makes LiteLLM run a streaming chat completion as one blocking send and replay the answer as a single chunk

Call the agent over A2A with a blocking send:

curl http://localhost:4000/a2a/foundry-agent \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc": "2.0",
    "id": "1",
    "method": "message/send",
    "params": {
      "message": {
        "kind": "message",
        "role": "user",
        "messageId": "m1",
        "parts": [{"kind": "text", "text": "What is 2 + 2?"}]
      },
      "configuration": {"blocking": true}
    }
  }'

Or through chat completions, where LiteLLM asks for the blocking send itself:

curl http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "a2a/foundry-agent", "messages": [{"role": "user", "content": "What is 2 + 2?"}]}'