Skip to main content

Secret Manager

LiteLLM supports reading secrets from Azure Key Vault and Infisical

AWS Secret Manager

Store your proxy keys in AWS Secret Manager.

Proxy Usage

  1. Save AWS Credentials in your environment
os.environ["AWS_ACCESS_KEY_ID"] = ""  # Access key
os.environ["AWS_SECRET_ACCESS_KEY"] = "" # Secret access key
os.environ["AWS_REGION_NAME"] = "" # us-east-1, us-east-2, us-west-1, us-west-2
  1. Enable AWS Secret Manager in config.
master_key: os.environ/litellm_master_key
key_management_system: "aws_secret_manager" # 👈 KEY CHANGE
hosted_keys: ["litellm_master_key"] # 👈 Specify which env keys you stored on AWS
  1. Run proxy
litellm --config /path/to/config.yaml

Azure Key Vault

Quick Start

### Instantiate Azure Key Vault Client ###
from azure.keyvault.secrets import SecretClient
from azure.identity import ClientSecretCredential

# Set your Azure Key Vault URI
KVUri = os.getenv("AZURE_KEY_VAULT_URI")

# Set your Azure AD application/client ID, client secret, and tenant ID - create an application with permission to call your key vault
client_id = os.getenv("AZURE_CLIENT_ID")
client_secret = os.getenv("AZURE_CLIENT_SECRET")
tenant_id = os.getenv("AZURE_TENANT_ID")

# Initialize the ClientSecretCredential
credential = ClientSecretCredential(client_id=client_id, client_secret=client_secret, tenant_id=tenant_id)

# Create the SecretClient using the credential
client = SecretClient(vault_url=KVUri, credential=credential)

### Connect to LiteLLM ###
import litellm
litellm.secret_manager = client


Usage with OpenAI Proxy Server

  1. Install Proxy dependencies
pip install 'litellm[proxy]' 'litellm[extra_proxy]'
  1. Save Azure details in your environment
  1. Add to proxy config.yaml
- model_name: "my-azure-models" # model alias
model: "azure/<your-deployment-name>"
api_key: "os.environ/AZURE-API-KEY" # reads from key vault - get_secret("AZURE_API_KEY")
api_base: "os.environ/AZURE-API-BASE" # reads from key vault - get_secret("AZURE_API_BASE")

key_management_system: "azure_key_vault"

You can now test this by starting your proxy:

litellm --config /path/to/config.yaml

Quick Test Proxy

Google Key Management Service

Use encrypted keys from Google KMS on the proxy

Usage with OpenAI Proxy Server

Step 1. Add keys to env

export GOOGLE_APPLICATION_CREDENTIALS="/path/to/credentials.json"
export GOOGLE_KMS_RESOURCE_NAME="projects/*/locations/*/keyRings/*/cryptoKeys/*"
export PROXY_DATABASE_URL_ENCRYPTED=b'\n$\x00D\xac\xb4/\x8e\xc...'

Step 2: Update Config

key_management_system: "google_kms"
database_url: "os.environ/PROXY_DATABASE_URL_ENCRYPTED"
master_key: sk-1234

Step 3: Start + test proxy

$ litellm --config /path/to/config.yaml

And in another terminal

$ litellm --test 

Quick Test Proxy

Infisical Secret Manager

Integrates with Infisical's Secret Manager for secure storage and retrieval of API keys and sensitive data.


liteLLM manages reading in your LLM API secrets/env variables from Infisical for you

import litellm
from infisical import InfisicalClient

litellm.secret_manager = InfisicalClient(token="your-token")

messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What's the weather like today?"},

response = litellm.completion(model="gpt-3.5-turbo", messages=messages)


.env Files

If no secret manager client is specified, Litellm automatically uses the .env file to manage sensitive data.