Skip to main content

Supported Endpoints

LiteLLM serves chat, responses, embeddings, images, video, audio, realtime, rerank, vector store, file, batch, MCP, and A2A endpoints. Send a request in the OpenAI, Anthropic, or Google format, and LiteLLM calls any provider that supports it. Use the Python SDK or the AI Gateway. Most endpoints work in both.

Which endpoints does LiteLLM support?​

Search by name, path, or Python function, or select a category. Each card opens the guide for that endpoint.

Chat and responses

Send prompts to any model in the OpenAI, Anthropic, Google or Bedrock format.

POST/v1/chat/completions

Chat Completions#

Call any supported model in the OpenAI chat format, with streaming, tools and structured output.

POST/v1/completions

Text Completions#

Legacy OpenAI text completion format, bridged to chat models.

POST · GET/v1/responses

Responses#

OpenAI Responses API for any model, with stateful sessions, tools and background mode.

POST/v1/messages

Anthropic Messages#

Call any model in the Anthropic Messages format, so Claude Code and the Anthropic SDK work unchanged.

POST/v1beta/models/{model}:generateContent

Google generateContent#

Call any model in the Google generateContent format, with streaming through streamGenerateContent.

POST · GET/v1beta/interactions

Interactions#

Google Interactions API for any model, with stateful conversations and streaming.

POST/bedrock/model/{model}/converse

Bedrock Converse#

Native Bedrock Converse calls through the proxy with LiteLLM keys, budgets and logging.

POST/bedrock/model/{model}/invoke

Bedrock Invoke#

Native Bedrock InvokeModel calls through the proxy with LiteLLM keys, budgets and logging.

Embeddings, rerank and retrieval

Embed text, rank results, search the web and query vector stores.

POST/v1/embeddings

Embeddings#

Turn text into vectors with any supported embedding model in the OpenAI format.

POST/v1/rerank

Rerank#

Rank documents by relevance to a query, in the Cohere rerank format.

POST · GET/v1/vector_stores

Vector Stores#

Create, register and search vector stores on OpenAI, Bedrock, Gemini and more.

POST · GET/v1/vector_stores/{vector_store_id}/files

Vector Store Files#

Add, list, read and delete the files inside a vector store.

POST/v1/rag/ingest

RAG Ingest#

Upload, chunk, embed and store documents in a vector store with one call.

POST/v1/rag/query

RAG Query#

Search a vector store, optionally rerank, and answer with an LLM in one call.

Audio and realtime

Transcribe speech, generate speech and stream live audio.

POST/v1/audio/transcriptions

Audio Transcriptions#

Convert speech to text with Whisper, Deepgram, Gemini and other speech models.

POST/v1/audio/speech

Audio Speech#

Convert text to speech with OpenAI, Azure, AWS Polly, ElevenLabs and other providers.

WS/v1/realtime

Realtime#

Stream live audio and text over a WebSocket, load balanced across OpenAI, Azure and more.

WS/v1/realtime/translations

Realtime Translations#

Live speech-to-speech translation with gpt-realtime-translate over WebSocket or WebRTC.

Images and video

Generate and edit images and video.

POST/v1/images/generations

Image Generations#

Generate images from a text prompt with DALL-E, Imagen, Stable Diffusion and others.

POST/v1/images/edits

Image Edits#

Edit one or more images with a text prompt, in the OpenAI images edits format.

SDKlitellm.image_variation()

Image Variations#

Create variations of an image with OpenAI or Topaz. Python SDK only, with no Gateway route.

POST · GET/v1/videos

Videos#

Generate, remix and download videos with Sora, Veo and other video models.

Documents

Read text and typed fields from documents and images.

POST/v1/ocr

OCR#

Extract text from PDFs and images with Mistral, Azure AI, Gemini Enterprise Agent Platform or Cohere.

POST/v1/extraction

Extraction#

Fill typed fields from text and images, with confidence scores. Runs on Cloudflare Clairic.

Agents and tools

Run agents, call MCP tools and manage code containers and memory.

POST/a2a/{agent_id}

Agent Gateway (A2A)#

Invoke agents in the A2A protocol and control which keys and teams can reach each agent.

POST/mcp

MCP Gateway#

One MCP endpoint for all your tool servers, with access control by key and team.

POST · GET/v1/assistants

Assistants#

OpenAI Assistants, threads and runs. OpenAI shut this API down on August 26, 2026. Use Responses.

POST · GET/v1beta/agents

Managed Agents#

Create and run Google AI Studio managed agents through the proxy.

POST · GET/v1/skills

Skills#

Create, list and use Anthropic skills as reusable agent capabilities.

POST · GET/v1/containers

Containers#

Create and manage OpenAI and Azure code interpreter containers.

POST · GET/v1/containers/{container_id}/files

Container Files#

Upload, list and download files in a code interpreter container.

POST/v1/responses

Sandbox#

Run model-written code in e2b or OpenSandbox with the code_interpreter tool on /v1/responses.

POST · GET/v1/memory

Memory#

Store and read key-value memory entries scoped to users and teams.

Safety and routing

Moderate content, run guardrails and get model decisions.

POST/v1/moderations

Moderations#

Check text for harmful content in the OpenAI moderations format. No model setup is needed.

POST/guardrails/apply_guardrail

Apply Guardrail#

Call a guardrail set up on your proxy directly from any service.

POST/v1/decisions

Decisions#

Ask a typed question about an input and get probabilities back instead of text.

Files, batches and tuning

Upload files, run batch jobs, fine tune models and run evals.

POST · GET/v1/files

Files#

Upload and manage files on the provider, in the OpenAI files format.

POST · GET/v1/batches

Batches#

Run large async jobs at lower cost on OpenAI, Azure, Bedrock, Gemini Enterprise Agent Platform and more.

POST · GET/v1/fine_tuning/jobs

Fine Tuning#

Create and track fine tuning jobs on OpenAI, Azure OpenAI and Gemini Enterprise Agent Platform.

POST · GET/v1/evals

Evals#

Create and run OpenAI evals to measure model quality against test criteria.

Tokens

Count and inspect tokens before you send a request.

POST/v1/responses/input_tokens

Token Counting#

Count input tokens with the provider token counting API before you send a request.

POST/v1/messages/count_tokens

Anthropic Count Tokens#

Count tokens for a messages request in the Anthropic format.

POST/tokenize

Tokenize#

Get the token count, IDs and text for one string from vLLM or IBM watsonx.ai. Not in a release yet.

How does LiteLLM handle a request?​

LiteLLM handles a request in one of three ways. The diagram shows each path from your app to the provider.

Three request paths through LiteLLM Gateway. Translated: your app sends a chat request in the OpenAI, Anthropic, or Google format, and LiteLLM maps it to any chat provider. Native: your app calls an endpoint such as embeddings, images, speech, or rerank with one request shape, and LiteLLM maps it to each provider of that endpoint. Pass-through: your app calls the API of one provider through LiteLLM, which adds the provider credentials and forwards the request unchanged.
TranslatedOne chat request, in the format your app already uses.
LiteLLM GatewayMaps the format
POST /v1/chat/completionsOpenAI format
POST /v1/responsesOpenAI format
POST /v1/messagesAnthropic format
generateContentGoogle format
Anthropic, Bedrock, Azure OpenAI, Gemini, and any other chat provider

The answer comes back in the format your app sent.

NativeOne request shape for each endpoint.
LiteLLM GatewayMaps to each provider API
POST /v1/embeddings
OpenAI, Bedrock, Cohere, Mistral, and more
POST /v1/images/generations
OpenAI, Bedrock, Gemini, Stability AI, and more
POST /v1/audio/speech
OpenAI, Azure OpenAI, AWS Polly, ElevenLabs, and more
POST /v1/rerank
Cohere, Bedrock, Jina AI, Voyage AI, and more

The same request works for every provider of that endpoint.

Pass-throughThe API of one provider, called through LiteLLM.
LiteLLM GatewayAdds provider credentials, forwards unchanged
/anthropic/v1/messages
Anthropic
/bedrock/...
Bedrock
/gemini/...
Google AI Studio
/cohere/...
Cohere

Use it for a provider feature that LiteLLM does not map.

Compare the three paths​

TranslatedNativePass-through
What LiteLLM doesTakes a chat request in one of three formats and maps it to the API of any chat providerTakes one request shape for each endpoint and maps it to the API of each provider that supports itSends your request to the provider API without changes
Request formatOpenAI, Anthropic, or GoogleUsually OpenAI. Rerank uses the Cohere format, search the Perplexity format, and OCR the Mistral formatThe API of the provider
Examples/v1/chat/completions, /v1/responses, /v1/messages, generateContent/v1/embeddings, /v1/images/generations, /v1/audio/speech, /v1/rerank/anthropic/v1/messages, /bedrock/..., /gemini/...
LiteLLM keys and logsYesYesYes
Cost trackingYesYesFor the providers that support it. Each provider page says if it does
Use it whenYou want one client for many modelsYou use one capability across providersYou already use the provider SDK, or you need a provider feature that LiteLLM does not map

Which providers have pass-through endpoints?​

A pass-through route sends a request to the API of one provider. LiteLLM checks your LiteLLM key, adds the provider credentials, and logs the request. Cost tracking depends on the provider. Read why to use pass-through endpoints, or select a provider.

For every pass-through route:Managed IDsCustom pass-through routesCost trackingGuardrails

Frequently asked questions​

Which endpoints does LiteLLM support?​

LiteLLM supports more than 40 endpoints. They include chat completions, responses, Anthropic messages, embeddings, images, video, audio, realtime, rerank, vector stores, files, batches, MCP, and A2A. This page lists them, with a link to the guide for each.

Can I send an Anthropic-format request to a model that is not from Anthropic?​

Yes. Send the request to /v1/messages in the Anthropic format and set the model to any chat model in your config. LiteLLM changes the request to the API of that provider and sends the answer back in the Anthropic format. See /v1/messages.

Can I call a model that only supports the Responses API from /chat/completions?​

Yes. LiteLLM bridges /chat/completions to the Responses API when the provider does not support chat completions for that model. The reverse also works. /v1/responses can call providers that only have chat completions. See /responses.

What is the difference between a translated endpoint and a pass-through endpoint?​

A translated endpoint takes one request format and calls many providers. A pass-through endpoint sends your request to the API of one provider without changes. Both use your LiteLLM key and are logged. For pass-through endpoints, cost tracking depends on the provider.

Do the endpoints work in the Python SDK and in the AI Gateway?​

Most endpoints have a Python SDK function and a Gateway route. Each card on this page shows the Python function when one exists. Serving MCP servers is a Gateway feature only.

How do I ask for an endpoint that is not on this page?​

Open an issue in the BerriAI/litellm repository on GitHub with the endpoint and the providers you need.

Next steps​