Skip to main content

[BETA] High Availability Control Plane

Deploy a single LiteLLM UI that manages multiple independent LiteLLM proxy instances, each with its own database, Redis, and master key.

info

This is an Enterprise feature.

Enterprise Pricing

Get free 7-day trial key

Why This Architecture?​

In the standard multi-region setup, all instances share a single database and master key. This works, but introduces a shared dependency. If the database goes down, every instance is affected.

The High Availability Control Plane takes a different approach:

Shared Database (Standard)High Availability Control Plane
DatabaseSingle shared DB for all instancesEach instance has its own DB
RedisShared RedisEach instance has its own Redis
Master KeySame key across all instancesEach instance has its own key
Failure isolationDB outage affects all instancesFailure is isolated to one instance
User managementCentralized, one user tableIndependent, each worker manages its own users
UIOne UI per admin instanceSingle control plane UI manages all workers

Benefits​

  • True high availability: no shared infrastructure means no single point of failure
  • Blast radius containment: a misconfiguration or outage on one worker doesn't affect others
  • Regional isolation: workers can run in different regions with data residency requirements
  • Simpler operations: each worker is a self-contained LiteLLM deployment

Architecture​

👤
Admin
Control PlaneADMIN UI ONLY
cp.example.com
Not a router — does not proxy LLM requests.
Lets admins switch between workers to manage them.
UI management only
Worker AUS East
worker-a.example.com
Handles LLM requests
🗄Own Database
âš¡Own Redis
Worker BEU West
worker-b.example.com
Handles LLM requests
🗄Own Database
âš¡Own Redis

The control plane is a LiteLLM instance that serves the admin UI and knows about all the workers. It is not a router — it does not proxy or route any LLM requests. It exists purely so admins can switch between workers and manage them from a single UI.

Each worker is a fully independent LiteLLM proxy that handles LLM requests for its region or team. Workers have their own database, Redis, users, keys, teams, and budgets. No infrastructure is shared between workers.

Setup​

1. Control Plane Configuration​

The control plane needs a worker_registry that lists all worker instances.

cp_config.yaml
model_list: []

general_settings:
  master_key: sk-1234
  database_url: os.environ/DATABASE_URL

worker_registry:
  - worker_id: "worker-a"
    name: "Worker A"
    url: "http://localhost:4001"
  - worker_id: "worker-b"
    name: "Worker B"
    url: "http://localhost:4002"

Start the control plane:

litellm --config cp_config.yaml --port 4000

2. Worker Configuration​

Each worker needs control_plane_url in its general_settings to enable cross-origin authentication from the control plane UI.

PROXY_BASE_URL must also be set for each worker so that SSO callback redirects resolve correctly.

worker_a_config.yaml
model_list: []

general_settings:
  master_key: sk-worker-a-1234
  database_url: os.environ/WORKER_A_DATABASE_URL
  control_plane_url: "http://localhost:4000"
PROXY_BASE_URL=http://localhost:4001 litellm --config worker_a_config.yaml --port 4001
important

Each worker must have its own master_key and database_url. The whole point of this architecture is that workers are independent.

3. SSO Configuration (Optional)​

SSO is configured on the control plane instance the same way as a standard LiteLLM proxy. See the SSO setup guide for full instructions.

If using SSO, make sure to register each worker URL and the control plane URL as allowed callback URLs in your SSO provider's dashboard.

How It Works​

Login Flow​

  1. User visits the control plane UI (http://localhost:4000/ui)
  2. The login page shows a worker selector dropdown listing all registered workers
  3. User selects a worker (e.g. "Worker A") and logs in with username/password or SSO
  4. The UI authenticates against the selected worker using the /v3/login endpoint
  5. On success, the UI stores the worker's JWT and points all subsequent API calls at the worker
  6. The user can now manage keys, teams, models, and budgets on that worker, all from the control plane UI

Switching Workers​

Once logged in, users can switch workers from the navbar dropdown without leaving the UI. Switching redirects back to the login page to authenticate against the new worker.

Discovery​

The control plane exposes a /.well-known/litellm-ui-config endpoint that the UI reads on load. This endpoint returns:

  • is_control_plane: true
  • The list of workers with their IDs, names, and URLs

This is how the login page knows to show the worker selector.

Local Testing​

To try this out locally, start each instance in a separate terminal:

# Terminal 1: Control Plane
litellm --config cp_config.yaml --port 4000

# Terminal 2: Worker A
PROXY_BASE_URL=http://localhost:4001 litellm --config worker_a_config.yaml --port 4001

# Terminal 3: Worker B
PROXY_BASE_URL=http://localhost:4002 litellm --config worker_b_config.yaml --port 4002

Then open http://localhost:4000/ui. You should see the worker selector on the login page.

Configuration Reference​

Control Plane Settings​

FieldLocationDescription
worker_registryTop-level configList of worker instances
worker_registry[].worker_idRequiredUnique identifier for the worker
worker_registry[].nameRequiredDisplay name shown in the UI
worker_registry[].urlRequiredFull URL of the worker instance

Worker Settings​

FieldLocationDescription
general_settings.control_plane_urlRequiredURL of the control plane instance. Enables /v3/login and /v3/login/exchange endpoints on this worker.
PROXY_BASE_URLEnvironment variableThe worker's own external URL. Required for SSO callback redirects.
🚅
LiteLLM Enterprise
SSO/SAML, audit logs, spend tracking, multi-team management, and guardrails — built for production.
Learn more →