AWS Secret Manager

info

✨ This is an Enterprise Feature

Store your proxy keys in AWS Secret Manager.

Feature	Support	Description
Reading Secrets	✅	Read secrets e.g `OPENAI_API_KEY`
Writing Secrets	✅	Store secrets e.g `Virtual Keys`

Proxy Usage

Save AWS Credentials in your environment

os.environ["AWS_ACCESS_KEY_ID"] = ""  # Access key
os.environ["AWS_SECRET_ACCESS_KEY"] = "" # Secret access key
os.environ["AWS_REGION_NAME"] = "" # us-east-1, us-east-2, us-west-1, us-west-2

Enable AWS Secret Manager in config.

Read Keys from AWS Secret Manager
Write Virtual Keys to AWS Secret Manager
Read + Write Keys with AWS Secret Manager

general_settings:
  master_key: os.environ/litellm_master_key 
  key_management_system: "aws_secret_manager" # 👈 KEY CHANGE
  key_management_settings: 
    hosted_keys: ["litellm_master_key"] # 👈 Specify which env keys you stored on AWS 

This will only store virtual keys in AWS Secret Manager. No keys will be read from AWS Secret Manager.

general_settings:
  key_management_system: "aws_secret_manager" # 👈 KEY CHANGE
  key_management_settings: 
    store_virtual_keys: true # OPTIONAL. Defaults to False, when True will store virtual keys in secret manager
    prefix_for_stored_virtual_keys: "litellm/" # OPTIONAL. If set, this prefix will be used for stored virtual keys in the secret manager
    access_mode: "write_only" # Literal["read_only", "write_only", "read_and_write"]
    description: "litellm virtual key" # OPTIONAL, if set will set this as the description for all virtual keys
    tags: # OPTIONAL, if set will set this as the tags for all virtual keys
      Environment: "Prod"
      Owner: "AI Platform team"

general_settings:
  master_key: os.environ/litellm_master_key 
  key_management_system: "aws_secret_manager" # 👈 KEY CHANGE
  key_management_settings: 
    store_virtual_keys: true # OPTIONAL. Defaults to False, when True will store virtual keys in secret manager
    prefix_for_stored_virtual_keys: "litellm/" # OPTIONAL. If set, this prefix will be used for stored virtual keys in the secret manager
    access_mode: "read_and_write" # Literal["read_only", "write_only", "read_and_write"]
    hosted_keys: ["litellm_master_key"] # OPTIONAL. Specify which env keys you stored on AWS

Run proxy

litellm --config /path/to/config.yaml

Using K/V pairs in 1 AWS Secret

You can read multiple keys from a single AWS Secret using the primary_secret_name parameter:

general_settings:
  key_management_system: "aws_secret_manager"
  key_management_settings:
    hosted_keys: [
      "OPENAI_API_KEY_MODEL_1",
      "OPENAI_API_KEY_MODEL_2",
    ]
    primary_secret_name: "litellm_secrets" # 👈 Read multiple keys from one JSON secret

The primary_secret_name allows you to read multiple keys from a single AWS Secret as a JSON object. For example, the "litellm_secrets" would contain:

{
  "OPENAI_API_KEY_MODEL_1": "sk-key1...",
  "OPENAI_API_KEY_MODEL_2": "sk-key2..."
}

This reduces the number of AWS Secrets you need to manage.

Proxy Usage​

Using K/V pairs in 1 AWS Secret​

Proxy Usage

Using K/V pairs in 1 AWS Secret