---
title: "Migrate from Kubernetes Manifests to the Microservices Chart"
url: "/docs/proxy/manifests_to_microservices"
canonical_url: "https://docs.litellm.ai/docs/proxy/manifests_to_microservices"
type: "docs"
last_updated: "2026-10-14"
summary: "Move a LiteLLM proxy deployed with raw Kubernetes manifests onto the componentized litellm Helm chart so gateway, backend, and UI scale independently."
related:
  - "/docs/proxy/global_control_plane"
  - "/docs/secret_managers/overview"
---
# Migrate from Kubernetes Manifests to the Microservices Chart

> Index of all LiteLLM docs: https://docs.litellm.ai/llms.txt


This guide moves a LiteLLM proxy that runs from hand-written Kubernetes manifests (the [Kubernetes without Helm](./deploy.md#kubernetes-without-helm) layout: one `Deployment`, one `Service`, a `ConfigMap`, and a `Secret`) onto the componentized `litellm` Helm chart, where LLM traffic (`gateway`), management APIs (`backend`), and the Admin UI (`ui`) run as separate Deployments with their own autoscaling. The existing Postgres database and Redis are kept; no data moves. The old Deployment stays up until the new release serves traffic, so the switch is reversible at every step.

You need `helm` 3.8 or newer, `kubectl` access to the target namespace, the current manifests, and network reachability from the cluster to Postgres and Redis. For what each value does, see the chart's [`values.yaml`](https://github.com/BerriAI/litellm/blob/main/helm/litellm/values.yaml); for the architecture behind the two modes, see [Production Deployment](./deploy.md#architecture).

## 1. Capture the current deployment

Record the image tag, replica count, config, and every environment variable the proxy reads today, so nothing is lost in translation.

```bash
kubectl get deploy litellm-deployment -o yaml > old-deployment.yaml
kubectl get cm litellm-config-file -o yaml > old-config.yaml
kubectl get svc litellm-service -o yaml > old-service.yaml
kubectl get ingress -o yaml > old-ingress.yaml
kubectl get secret litellm-secrets -o jsonpath='{.data}' | jq 'keys'
```

Adjust the names to match your manifests. From `old-deployment.yaml` note `spec.replicas`, `spec.template.spec.containers[0].image`, `env`, `envFrom`, `resources`, and any probe overrides. From `old-config.yaml` keep the `config.yaml` body; it is reused verbatim in step 4.

## 2. Split `DATABASE_URL` into a username and password Secret

The chart never reads `DATABASE_URL`. It sets `DATABASE_HOST`, `DATABASE_PORT`, `DATABASE_NAME`, `DATABASE_USER`, and `DATABASE_PASSWORD` from discrete values and a Secret, and the proxy builds the URL itself. Take the pieces out of your existing URL (`postgresql://USER:PASSWORD@HOST:PORT/DBNAME`) and create the Secret:

```bash
kubectl create secret generic litellm-db \
  --from-literal=username='USER' \
  --from-literal=password='PASSWORD'
```

Passwords with `@`, `/`, or `%` need no escaping here; the proxy percent-encodes them.

Keep the existing `litellm-secrets` Secret. The chart references the master key from it directly, and provider keys in it are mounted whole in step 4. If `LITELLM_SALT_KEY` is set in the old Deployment, keep it in the same Secret; changing it makes stored provider credentials unreadable.

## 3. Choose the chart version

Pin the chart to the version that matches, or is newer than, your current proxy image; component image tags default to the chart version, so a proxy on `v1.90.2` moves to chart `1.90.2`. Chart `1.89.0` is the oldest supported.

```bash
helm show chart oci://ghcr.io/berriai/litellm/chart/litellm --version 1.90.2
```

## 4. Write `values.yaml`

Translate the manifests with this mapping.

| Manifest | Chart value |
| --- | --- |
| `ConfigMap` `config.yaml` body | `gateway.config.proxy_config` (shared by gateway and backend) |
| `LITELLM_MASTER_KEY` key in the Secret | `masterKey.secretName`, `masterKey.secretKey` |
| `DATABASE_URL` | `database.writer.host`, `.port`, `.dbname`, `.passwordSecret` (step 2) |
| `REDIS_HOST`, `REDIS_PORT`, `REDIS_PASSWORD` | `redis.host`, `redis.port`, `redis.passwordSecret` |
| Provider keys in `envFrom.secretRef` | `gateway.envSecrets` and `backend.envSecrets` |
| Other `env` entries | `gateway.extraEnv`, `backend.extraEnv` |
| `spec.replicas` | `gateway.hpa.minReplicas` (or `gateway.replicaCount` with `gateway.hpa.enabled: false`) |
| `resources` | `gateway.resources` |
| `args: ["--config", ...]` | drop; the chart sets `CONFIG_FILE_PATH` |
| `Ingress` host | `ingress.host` |

A manifest deployment with two replicas, provider keys in `litellm-secrets`, and Redis at `redis.internal` becomes:

```yaml title="values.yaml"
masterKey:
  secretName: litellm-secrets
  secretKey: LITELLM_MASTER_KEY

database:
  writer:
    host: postgres.internal
    port: 5432
    dbname: litellm
    passwordSecret:
      name: litellm-db
      usernameKey: username
      passwordKey: password

redis:
  host: redis.internal
  port: 6379
  passwordSecret:
    name: litellm-secrets
    passwordKey: REDIS_PASSWORD

gateway:
  envSecrets:
    - litellm-secrets
  resources:
    requests: { cpu: "1", memory: 4Gi }
    limits: { cpu: "2", memory: 4Gi }
  hpa:
    minReplicas: 2
    maxReplicas: 10
  config:
    proxy_config:
      model_list:
        - model_name: gpt-4o
          litellm_params:
            model: openai/gpt-4o
            api_key: os.environ/OPENAI_API_KEY
      router_settings:
        redis_host: os.environ/REDIS_HOST
        redis_port: os.environ/REDIS_PORT
        redis_password: os.environ/REDIS_PASSWORD
      litellm_settings:
        cache: true
        cache_params:
          type: redis
          host: os.environ/REDIS_HOST
          port: os.environ/REDIS_PORT
          password: os.environ/REDIS_PASSWORD

backend:
  envSecrets:
    - litellm-secrets

ingress:
  enabled: false
```

Drop the `redis.passwordSecret` block if your Redis has no password. If the old deployment had no Redis, add one before scaling past one pod; see [What Needs Redis](./redis_requirements.md). Leave `ingress.enabled: false` for now; traffic moves in step 7.

Render and check the result before installing:

```bash
helm template litellm oci://ghcr.io/berriai/litellm/chart/litellm \
  --version 1.90.2 -f values.yaml | kubectl apply --dry-run=server -f -
```

## 5. Install alongside the old deployment

The chart's resource names (`litellm-gateway`, `litellm-backend`, `litellm-ui`, `litellm-gateway-config`) do not collide with the manifest names, so both can run in the same namespace.

```bash
helm upgrade --install litellm oci://ghcr.io/berriai/litellm/chart/litellm \
  --version 1.90.2 -f values.yaml --wait --timeout 10m
```

Helm runs the migrations Job as a pre-install hook before creating the Deployments. It applies the same Prisma migrations the old image applied at startup, so on an up-to-date database it is a no-op. The old pods keep running against the same schema throughout.

```bash
kubectl get pods -l app.kubernetes.io/instance=litellm
kubectl logs job/litellm-migrations
```

If the Job fails, fix the database values and rerun the same `helm upgrade` command; nothing else has been created yet.

## 6. Verify the new release

Test each component through its own Service before exposing it.

```bash
kubectl port-forward svc/litellm-gateway 4000:4000 &
kubectl port-forward svc/litellm-backend 4001:4001 &
kubectl port-forward svc/litellm-ui 3000:3000 &

curl -s localhost:4000/health/readiness
curl -s localhost:4000/v1/models -H "Authorization: Bearer $LITELLM_MASTER_KEY"
curl -s localhost:4000/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"ping"}]}'
curl -s localhost:4001/key/info -H "Authorization: Bearer $LITELLM_MASTER_KEY"
curl -sI localhost:3000/ui/
```

The readiness response should report `"cache": "redis"` and `/v1/models` should return the same models the old deployment served. An existing virtual key must work against the new gateway, which confirms the database and master key are the ones the old deployment used.

## 7. Move traffic

Point the ingress at the chart. Enable the chart's ingress on the same host as the old one and delete the old Ingress in the same step, or, if you manage ingress outside the chart, repoint your rules by path: `/v1/*`, `/chat/*`, `/health`, `/metrics` and other LLM routes to `litellm-gateway:4000`, `/ui`, `/_next`, `/litellm-asset-prefix`, and `/` to `litellm-ui:3000`, and everything else (`/key/*`, `/team/*`, `/user/*`, ...) to `litellm-backend:4001`. The full path split is in [`templates/ingress.yaml`](https://github.com/BerriAI/litellm/blob/main/helm/litellm/templates/ingress.yaml).

```yaml title="values.yaml"
ingress:
  enabled: true
  className: alb            # or nginx
  controller: alb           # or nginx
  host: llm.example.com
  annotations: {}           # copy TLS and load balancer annotations from old-ingress.yaml
```

```bash
kubectl delete -f old-ingress.yaml
helm upgrade litellm oci://ghcr.io/berriai/litellm/chart/litellm \
  --version 1.90.2 -f values.yaml --wait
```

### Separate hostname per component

Use this mode when the UI, backend, and gateway each have a separate DNS hostname and the ingress cannot route requests by path. The chart creates one host rule for each component.

```yaml title="values.yaml"
ingress:
  enabled: true
  host: ""
  extraPaths: []
  componentHosts:
    ui: ui.example.com
    backend: backend.example.com
    gateway: gateway.example.com
  tls:
    - secretName: litellm-component-tls
      hosts:
        - ui.example.com
        - backend.example.com
        - gateway.example.com

ui:
  backendUrl: https://backend.example.com
  publicUrl: https://ui.example.com

backend:
  gatewayUrl: https://gateway.example.com
```

The chart renders these values on the following components:

| Chart value | Component and rendered configuration |
| --- | --- |
| `ui.backendUrl` | The UI Deployment receives `LITELLM_BACKEND_URL` only when `ingress.componentHosts` is configured. The UI serves it as `backend_url` from `/ui/litellm-runtime-config.json`. |
| `ui.publicUrl` | The backend and gateway Deployments receive `LITELLM_CORS_ORIGINS`. The backend also receives `LITELLM_CORS_ALLOW_CREDENTIALS=true`. |
| `backend.gatewayUrl` | The backend Deployment receives `LITELLM_GATEWAY_URL`. The backend includes it as `gateway_base_url` in UI discovery configuration. |

Set `ui.publicUrl` to the browser-facing UI origin, including its scheme and port if present, without a path. The backend needs an explicit allowed origin and credentials because login uses a credentialed request and sets a session cookie on the backend host. Keep the UI and backend hostnames under the same registrable domain so the browser treats them as same-site and can send that cookie on later credentialed requests. The gateway receives the UI origin for CORS, but the chart does not enable gateway credentials because Playground inference requests do not use `credentials: "include"`.

If you manage ingress and DNS yourself, route each hostname to its component Service and set the corresponding environment variables:

| Component | Environment variables |
| --- | --- |
| UI | `LITELLM_BACKEND_URL=https://backend.example.com` |
| Backend | `LITELLM_GATEWAY_URL=https://gateway.example.com`, `LITELLM_CORS_ORIGINS=https://ui.example.com`, and `LITELLM_CORS_ALLOW_CREDENTIALS=true` |
| Gateway | `LITELLM_CORS_ORIGINS=https://ui.example.com` |

In component-host mode, the chart requires all three `ingress.componentHosts` values, an empty `ingress.host`, empty `ingress.extraPaths`, and the three URLs shown above. Rendering fails with one of these messages when the configuration is incomplete or conflicting:

- `ingress.componentHosts.ui, backend, and gateway must all be set together`
- `ingress.host must be empty when ingress.componentHosts is configured`
- `ingress.extraPaths must be empty when ingress.componentHosts is configured`
- `ui.backendUrl is required when ingress.componentHosts is configured`
- `backend.gatewayUrl is required when ingress.componentHosts is configured`
- `ui.publicUrl is required when ingress.componentHosts is configured`

SSO login is not supported yet when the UI uses a separate hostname because the callback is built from the backend request URL and returns to the backend host. The Playground's "Custom Proxy Base URL" setting also continues to override the discovered gateway URL.

Clients that used the in-cluster Service name (`litellm-service:4000`) must switch to `litellm-gateway:4000` for LLM calls and `litellm-backend:4001` for management calls.

Watch the gateway HPA and error rate for a few minutes:

```bash
kubectl get hpa litellm-gateway -w
kubectl logs deploy/litellm-gateway -f | grep -i error
```

## 8. Scale the gateway

The gateway HPA is on by default with CPU and memory targets, and scales the gateway alone; the backend has its own HPA and the UI is fixed at one replica unless `ui.hpa.enabled` is set. Raise `gateway.hpa.maxReplicas` or the floor as load needs, then `helm upgrade`. For LLM traffic, where pods saturate on open connections before CPU, add a requests or tokens per second target as described in [Scale on requests and tokens per pod](./deploy.md#scale-on-requests-and-tokens-per-pod).

If the old deployment was crash-looping, check why before relying on more replicas: `kubectl describe pod` on an old pod shows `OOMKilled` (raise `gateway.resources.limits.memory`), liveness failures during startup (set `gateway.startupProbe`), or a database or Redis connection error in `kubectl logs --previous`. Replicas multiply a bad configuration rather than fixing it.

## 9. Remove the old deployment

Once the new release has taken production traffic without incident, delete the manifest objects. Keep the Secret; the chart references it.

```bash
kubectl delete -f old-deployment.yaml -f old-service.yaml
kubectl delete -f old-config.yaml
```

## Roll back

While the old Deployment still exists, revert the ingress change from step 7 (`kubectl apply -f old-ingress.yaml` and `helm upgrade ... --set ingress.enabled=false`) and traffic returns to the old pods. The database is unchanged by the migration, provided the chart version was not newer than the old image; if you moved to a newer chart, schema changes it applied are forward-only, and the rollback target must be an image of at least that version. To remove the chart entirely, `helm uninstall litellm`, which leaves the external database, Redis, and your Secrets in place.

## Related pages

- [Global Control Plane](https://docs.litellm.ai/docs/proxy/global_control_plane.md)
- [Secret Managers Overview](https://docs.litellm.ai/docs/secret_managers/overview.md)