📄️ Integrate as a Model Provider
Quick Start for OpenAI-Compatible Providers
📄️ Add OpenAI-Compatible Provider (JSON)
For simple OpenAI-compatible providers (like Hyperbolic, Nscale, etc.), you can add support by editing a single JSON file.
📄️ Add Model Pricing & Context Window
To add pricing or context window information for a model, simply make a PR to this file:
🗃️ OpenAI
4 items
📄️ OpenAI (Text Completion)
LiteLLM supports OpenAI text completion models
📄️ OpenAI-Compatible Endpoints
Selecting openai as the provider routes your request to an OpenAI-compatible endpoint using the upstream
🗃️ Azure OpenAI
5 items
🗃️ Azure AI
9 items
🗃️ Vertex AI
12 items
🗃️ Google AI Studio
7 items
📄️ Anthropic
LiteLLM supports all anthropic models.
📄️ Tool Search
Tool search enables Claude to dynamically discover and load tools on-demand from large tool catalogs (10,000+ tools). Instead of loading all tool definitions into the context window upfront, Claude searches your tool catalog and loads only the tools it needs.
📄️ Preserved Thinking Prefix Stability
Claude models that preserve thinking across turns sign each thinking block and bind it to the request that produced it. When you replay the conversation with the thinking-binding-controls-2026-08-01 beta, Anthropic checks that the part of the request ahead of the block still matches what the block was created under dropblock, reported by Anthropic as a thinkingdropped entry in inputtransformations) or rejects the request with a 400 that names the part that differs (prefixmismatchbehavior: error). cachecontrol markers are left out of the comparison
📄️ AWS Sagemaker
LiteLLM supports All Sagemaker Huggingface Jumpstart Models
🗃️ Bedrock
13 items
📄️ LiteLLM Proxy (LLM Gateway)
| Property | Details |
📄️ Abliteration
Overview
📄️ AI21
LiteLLM supports the following AI21 models:
📄️ AI/ML API
https://aimlapi.com/
📄️ Aleph Alpha
LiteLLM supports all models from Aleph Alpha.
📄️ Amazon Nova
| Property | Details |
📄️ Anyscale
https://app.endpoints.anyscale.com/
📄️ Apertis AI (Stima API)
Overview
📄️ Baseten
LiteLLM supports both Baseten Model APIs and dedicated deployments with automatic routing.
📄️ Black Forest Labs Image Generation
Black Forest Labs provides state-of-the-art text-to-image generation using their FLUX models.
📄️ Black Forest Labs Image Editing
Black Forest Labs provides powerful image editing capabilities using their FLUX models to modify existing images based on text descriptions.
📄️ Bytez
LiteLLM supports all chat models on Bytez!
📄️ Cerebras
https://inference-docs.cerebras.ai/api-reference/chat-completions
📄️ Chutes
Overview
📄️ Clarifai
Anthropic, OpenAI, Qwen, xAI, Gemini and most of Open soured LLMs are Supported on Clarifai.
📄️ CLF AI Gateway
https://clfaigateway.dev/docs
📄️ Cloudflare Workers AI
https://developers.cloudflare.com/workers-ai/models/text-generation/
📄️ Codestral API [Mistral AI]
Codestral is available in select code-completion plugins but can also be queried directly. See the documentation for more details.
📄️ Cognition
Overview
📄️ Cohere
API KEYS
📄️ CometAPI
LiteLLM supports all AI models from CometAPI. CometAPI provides access to 500+ AI models through a unified API interface, including cutting-edge models like GPT-5, Claude Opus 4.1, and various other state-of-the-art language models.
📄️ CompactifAI
https://docs.compactif.ai/
📄️ Crusoe
Overview
📄️ Custom API Server (Custom Format)
Call your custom torch-serve / internal LLM APIs via LiteLLM
📄️ Dashscope API (Qwen models)
https://dashscope.console.aliyun.com/
📄️ Databricks
LiteLLM supports all models on Databricks
📄️ DataRobot
LiteLLM supports all models from DataRobot. Select datarobot as the provider to route your request through the datarobot OpenAI-compatible endpoint using the upstream official OpenAI Python API library.
📄️ Deepgram
LiteLLM supports Deepgram's /listen endpoint.
📄️ DeepInfra
https://deepinfra.com/
📄️ Deepseek
https://deepseek.com/
📄️ Docker Model Runner
Overview
📄️ Eden AI
Overview
📄️ ElevenLabs
ElevenLabs provides high-quality AI voice technology, including speech-to-text capabilities through their transcription API.
📄️ EmpirioLabs AI
Overview
📄️ Fal AI
Fal AI provides fast, scalable access to state-of-the-art image generation models including FLUX, Stable Diffusion, Imagen, and more.
📄️ Featherless AI
https://featherless.ai/
📄️ Fireworks AI
We support ALL Fireworks AI models, just set fireworks_ai/ as a prefix when sending completion requests
📄️ FriendliAI
We support ALL FriendliAI models, just set friendliai/ as a prefix when sending completion requests
📄️ Galadriel
https://docs.galadriel.com/api-reference/chat-completion-API
📄️ Github
https://github.com/marketplace/models
📄️ GitHub Copilot
https://docs.github.com/en/copilot
📄️ GMI Cloud
Overview
📄️ ChatGPT Subscription
Use ChatGPT Pro/Max subscription models through LiteLLM with OAuth device flow authentication.
📄️ GradientAI
https://digitalocean.com/products/gradientai
📄️ Groq
https://groq.com/
📄️ Helicone
Overview
📄️ Heroku
Provision a Model
🗃️ HuggingFace
2 items
📄️ Hyperbolic
Overview
📄️ Inception
Overview
📄️ Infinity
| Property | Details |
📄️ Jina AI
https://jina.ai/embeddings/
📄️ Lambda AI
Overview
📄️ LangGraph
Call LangGraph agents through LiteLLM using the OpenAI chat completions format.
📄️ Lemonade
Lemonade Server is an OpenAI-compatible local language model inference provider optimized for AMD GPUs and NPUs. The lemonade litellm provider supports standard chat completions with full OpenAI API compatibility.
📄️ Llamafile
LiteLLM supports all models on Llamafile.
📄️ LlamaGate
Overview
📄️ LM Studio
https://lmstudio.ai/docs/basics/server
📄️ Manus
Use Manus AI agents through LiteLLM's OpenAI-compatible Responses API.
📄️ Meta Model API
| Property | Details |
📄️ Meta Llama
| Property | Details |
📄️ Milvus - Vector Store
Use Milvus as a vector store for RAG.
🗃️ Mistral AI
2 items
📄️ MiniMax
MiniMax - v1/messages
📄️ MongoDB - Vector Store (BETA)
The MongoDB vector store integration is a BETA feature. It supports searching existing MongoDB Vector Search indexes through the optional LiteLLM MongoDB sidecar. Prepare your collections, indexes, and embedded documents before connecting them to LiteLLM.
📄️ Moonshot AI
Overview
📄️ Morph
LiteLLM supports all models on Morph
📄️ Nadir
Overview
📄️ Nebius AI Studio
https://docs.nebius.com/studio/inference/quickstart
📄️ NLP Cloud
LiteLLM supports all LLMs on NLP Cloud.
📄️ NanoGPT
Overview
📄️ Novita AI
| Property | Details |
📄️ Nscale (EU Sovereign)
https://docs.nscale.com/docs/inference/chat
🗃️ Nvidia NIM
2 items
📄️ Nvidia Riva (Speech-to-Text)
LiteLLM supports NVIDIA Riva for speech-to-text via /audio/transcriptions. Works with both the NVCF-hosted Riva endpoint (e.g. Parakeet on build.nvidia.com) and self-hosted Riva deployments.
📄️ Oracle Cloud Infrastructure (OCI)
LiteLLM supports the following models for OCI on-demand GenAI API.
📄️ Ollama
LiteLLM supports all models from Ollama
📄️ OpenRouter
LiteLLM supports all the text / chat / vision / embedding models from OpenRouter
📄️ Sarvam.ai
LiteLLM supports all the text models from Sarvam ai
📄️ 🆕 OVHCloud AI Endpoints
Leading French Cloud provider in Europe with data sovereignty and privacy.
🗃️ Perplexity AI
2 items
📄️ Petals
Petals//github.com/bigscience-workshop/petals
📄️ Poe
Overview
📄️ PublicAI
Overview
📄️ Predibase
LiteLLM supports all models on Predibase
📄️ Prism
Overview
📄️ Pydantic AI Agents
Call Pydantic AI Agents via LiteLLM's A2A Gateway.
📄️ QwenCloud and Qianwen AI Platform (Qwen models)
Alibaba unified its model platform brands (DashScope, Bailian, Model Studio) under QwenCloud outside mainland China and Qianwen AI Platform in mainland China. LiteLLM routes both brands through its existing DashScope implementation, and the legacy dashscope/ prefix keeps working as an alias
📄️ RAGFlow
Litellm supports Ragflow's chat completions APIs
📄️ RAGFlow Vector Stores
Litellm support creation and management of datasets for document processing and knowledge base management in Ragflow.
📄️ Recraft
https://www.recraft.ai/
📄️ Replicate
LiteLLM supports all models on Replicate
🗃️ RunwayML
2 items
📄️ AWS S3 Vectors
Use Amazon S3 Vectors as the vector store behind LiteLLM's unified vector store and RAG endpoints. LiteLLM calls the S3 Vectors REST API directly with SigV4-signed requests; no boto3 client is needed for the vector operations themselves.
📄️ SambaNova
https://cloud.sambanova.ai/
📄️ SAP Generative AI Hub
LiteLLM supports SAP Generative AI Hub's Orchestration Service.
📄️ Scaleway
LiteLLM supports all models available on Scaleway Generative APIs ↗.
📄️ SCX.ai
Overview
📄️ Stability AI
https://stability.ai/
📄️ Synthetic
Overview
📄️ Snowflake Cortex
LiteLLM supports all models on the Snowflake Cortex REST API, including models from Anthropic (Claude), OpenAI (GPT), Meta (Llama), Mistral, DeepSeek, and Snowflake.
📄️ Tencent TokenHub
https://www.tencentcloud.com/products/tokenhub
📄️ Tensormesh
Overview
📄️ Together AI
LiteLLM supports all models on Together AI.
📄️ Topaz
| Property | Details |
📄️ Triton Inference Server
LiteLLM supports Embedding Models on Triton Inference Servers
📄️ v0
Overview
📄️ Valkey - Vector Store
Search documents you have already indexed in Valkey through LiteLLM's unified vector store API, so that any virtual key can run retrieval against your datastore without ever holding your Valkey credentials.
📄️ Vercel AI Gateway
Overview
🗃️ vLLM
2 items
📄️ Volcano Engine (Volcengine)
https://www.volcengine.com/docs/82379/1263482
📄️ Voyage AI
https://docs.voyageai.com/embeddings/
📄️ Weights & Biases Inference
https://weave-docs.wandb.ai/quickstart-inference
🗃️ WatsonX
2 items
🗃️ xAI
3 items
📄️ Xiaomi MiMo
https://platform.xiaomimimo.com/#/docs
📄️ Xinference [Xorbits Inference]
https://inference.readthedocs.io/en/latest/index.html
📄️ Z.AI (Zhipu AI)
https://z.ai/