Supported Endpoints
LiteLLM serves chat, responses, embeddings, images, video, audio, realtime, rerank, vector store, file, batch, MCP, and A2A endpoints. Send a request in the OpenAI, Anthropic, or Google format, and LiteLLM calls any provider that supports it. Use the Python SDK or the AI Gateway. Most endpoints work in both.
Which endpoints does LiteLLM support?
Search by name, path, or Python function, or select a category. Each card opens the guide for that endpoint.
Chat and responses
Send prompts to any model in the OpenAI, Anthropic, Google or Bedrock format.
/v1/chat/completions Chat Completions#
Call any supported model in the OpenAI chat format, with streaming, tools and structured output.
/v1/responses Responses#
OpenAI Responses API for any model, with stateful sessions, tools and background mode.
/v1/messages Anthropic Messages#
Call any model in the Anthropic Messages format, so Claude Code and the Anthropic SDK work unchanged.
/v1beta/models/{model}:generateContent Google generateContent#
Call any model in the Google generateContent format, with streaming through streamGenerateContent.
/v1beta/interactions Interactions#
Google Interactions API for any model, with stateful conversations and streaming.
/bedrock/model/{model}/converse Bedrock Converse#
Native Bedrock Converse calls through the proxy with LiteLLM keys, budgets and logging.
/bedrock/model/{model}/invoke Bedrock Invoke#
Native Bedrock InvokeModel calls through the proxy with LiteLLM keys, budgets and logging.
Embeddings, rerank and retrieval
Embed text, rank results, search the web and query vector stores.
/v1/embeddings Embeddings#
Turn text into vectors with any supported embedding model in the OpenAI format.
/v1/search Search#
One web search API over Perplexity, Tavily, Exa, Brave and other search providers.
/v1/vector_stores Vector Stores#
Create, register and search vector stores on OpenAI, Bedrock, Gemini and more.
/v1/vector_stores/{vector_store_id}/files Vector Store Files#
Add, list, read and delete the files inside a vector store.
/v1/rag/ingest RAG Ingest#
Upload, chunk, embed and store documents in a vector store with one call.
Audio and realtime
Transcribe speech, generate speech and stream live audio.
/v1/audio/transcriptions Audio Transcriptions#
Convert speech to text with Whisper, Deepgram, Gemini and other speech models.
/v1/audio/speech Audio Speech#
Convert text to speech with OpenAI, Azure, AWS Polly, ElevenLabs and other providers.
/v1/realtime Realtime#
Stream live audio and text over a WebSocket, load balanced across OpenAI, Azure and more.
/v1/realtime/translations Realtime Translations#
Live speech-to-speech translation with gpt-realtime-translate over WebSocket or WebRTC.
Images and video
Generate and edit images and video.
/v1/images/generations Image Generations#
Generate images from a text prompt with DALL-E, Imagen, Stable Diffusion and others.
/v1/images/edits Image Edits#
Edit one or more images with a text prompt, in the OpenAI images edits format.
litellm.image_variation()Image Variations#
Create variations of an image with OpenAI or Topaz. Python SDK only, with no Gateway route.
Documents
Read text and typed fields from documents and images.
/v1/ocr OCR#
Extract text from PDFs and images with Mistral, Azure AI, Gemini Enterprise Agent Platform or Cohere.
/v1/extraction Extraction#
Fill typed fields from text and images, with confidence scores. Runs on Cloudflare Clairic.
Agents and tools
Run agents, call MCP tools and manage code containers and memory.
/a2a/{agent_id} Agent Gateway (A2A)#
Invoke agents in the A2A protocol and control which keys and teams can reach each agent.
/mcp MCP Gateway#
One MCP endpoint for all your tool servers, with access control by key and team.
/v1/assistants Assistants#
OpenAI Assistants, threads and runs. OpenAI shut this API down on August 26, 2026. Use Responses.
/v1beta/agents Managed Agents#
Create and run Google AI Studio managed agents through the proxy.
/v1/containers/{container_id}/files Container Files#
Upload, list and download files in a code interpreter container.
Safety and routing
Moderate content, run guardrails and get model decisions.
/v1/moderations Moderations#
Check text for harmful content in the OpenAI moderations format. No model setup is needed.
/guardrails/apply_guardrail Apply Guardrail#
Call a guardrail set up on your proxy directly from any service.
Files, batches and tuning
Upload files, run batch jobs, fine tune models and run evals.
/v1/batches Batches#
Run large async jobs at lower cost on OpenAI, Azure, Bedrock, Gemini Enterprise Agent Platform and more.
/v1/fine_tuning/jobs Fine Tuning#
Create and track fine tuning jobs on OpenAI, Azure OpenAI and Gemini Enterprise Agent Platform.
Tokens
Count and inspect tokens before you send a request.
/v1/responses/input_tokens Token Counting#
Count input tokens with the provider token counting API before you send a request.
/v1/messages/count_tokens Anthropic Count Tokens#
Count tokens for a messages request in the Anthropic format.
How does LiteLLM handle a request?
LiteLLM handles a request in one of three ways. The diagram shows each path from your app to the provider.
POST /v1/chat/completionsOpenAI formatPOST /v1/responsesOpenAI formatPOST /v1/messagesAnthropic formatgenerateContentGoogle formatThe answer comes back in the format your app sent.
POST /v1/embeddingsPOST /v1/images/generationsPOST /v1/audio/speechPOST /v1/rerankThe same request works for every provider of that endpoint.
/anthropic/v1/messages/bedrock/.../gemini/.../cohere/...Use it for a provider feature that LiteLLM does not map.
Compare the three paths
| Translated | Native | Pass-through | |
|---|---|---|---|
| What LiteLLM does | Takes a chat request in one of three formats and maps it to the API of any chat provider | Takes one request shape for each endpoint and maps it to the API of each provider that supports it | Sends your request to the provider API without changes |
| Request format | OpenAI, Anthropic, or Google | Usually OpenAI. Rerank uses the Cohere format, search the Perplexity format, and OCR the Mistral format | The API of the provider |
| Examples | /v1/chat/completions, /v1/responses, /v1/messages, generateContent | /v1/embeddings, /v1/images/generations, /v1/audio/speech, /v1/rerank | /anthropic/v1/messages, /bedrock/..., /gemini/... |
| LiteLLM keys and logs | Yes | Yes | Yes |
| Cost tracking | Yes | Yes | For the providers that support it. Each provider page says if it does |
| Use it when | You want one client for many models | You use one capability across providers | You already use the provider SDK, or you need a provider feature that LiteLLM does not map |
Which providers have pass-through endpoints?
A pass-through route sends a request to the API of one provider. LiteLLM checks your LiteLLM key, adds the provider credentials, and logs the request. Cost tracking depends on the provider. Read why to use pass-through endpoints, or select a provider.
- AWS Bedrock
/bedrock - Amazon Transcribe
/transcribe - Anthropic
/anthropic - AssemblyAI
/assemblyai - Azure OpenAI
/azure - Azure Speech
/azure_speech - Cohere
/cohere - Comprehend Medical
/comprehendmedical - Cursor
/cursor - Deepgram live transcription
/deepgram/v1/listen - EU JEV
/eu_jev/v1/systemone - Gemini Enterprise Agent Platform (formerly Vertex AI)
/vertex_ai - Gemini Enterprise Agent Platform Live API
/vertex_ai/live - Gemini Enterprise Agent Platform Search
/vertex_ai/discovery - GigaChat
/gigachat - Google AI Studio
/gemini - Langfuse
/langfuse - Mistral
/mistral - OpenAI
/openai_passthrough - TinyFish
/tinyfish - Typesafe
/typesafe - vLLM
/vllm
For every pass-through route:Managed IDsCustom pass-through routesCost trackingGuardrails
Frequently asked questions
Which endpoints does LiteLLM support?
LiteLLM supports more than 40 endpoints. They include chat completions, responses, Anthropic messages, embeddings, images, video, audio, realtime, rerank, vector stores, files, batches, MCP, and A2A. This page lists them, with a link to the guide for each.
Can I send an Anthropic-format request to a model that is not from Anthropic?
Yes. Send the request to /v1/messages in the Anthropic format and set the model to any chat model in your config. LiteLLM changes the request to the API of that provider and sends the answer back in the Anthropic format. See /v1/messages.
Can I call a model that only supports the Responses API from /chat/completions?
Yes. LiteLLM bridges /chat/completions to the Responses API when the provider does not support chat completions for that model. The reverse also works. /v1/responses can call providers that only have chat completions. See /responses.
What is the difference between a translated endpoint and a pass-through endpoint?
A translated endpoint takes one request format and calls many providers. A pass-through endpoint sends your request to the API of one provider without changes. Both use your LiteLLM key and are logged. For pass-through endpoints, cost tracking depends on the provider.
Do the endpoints work in the Python SDK and in the AI Gateway?
Most endpoints have a Python SDK function and a Gateway route. Each card on this page shows the Python function when one exists. Serving MCP servers is a Gateway feature only.
How do I ask for an endpoint that is not on this page?
Open an issue in the BerriAI/litellm repository on GitHub with the endpoint and the providers you need.
Next steps
- Tutorial: Start the AI Gateway or make your first SDK call.
- How-to: Call the Gateway from the OpenAI SDK, LangChain, or curl.
- Explanation: Why use pass-through endpoints.
- Reference: All model providers and the endpoints each one supports.