CLI Arguments

This page documents all command-line interface (CLI) arguments available for the LiteLLM proxy server.

Server Configuration

--host

Default: '0.0.0.0'
The host for the server to listen on.
Usage:
```
litellm --host 127.0.0.1
```
Usage - set Environment Variable: HOST

export HOST=127.0.0.1
litellm

--port

Default: 4000
The port to bind the server to.
Usage:
```
litellm --port 8080
```
Usage - set Environment Variable: PORT
```
export PORT=8080
litellm
```

--num_workers

Default: Number of logical CPUs in the system, or 4 if that cannot be determined
The number of uvicorn / gunicorn workers to spin up.
Usage:
```
litellm --num_workers 4
```
Usage - set Environment Variable: NUM_WORKERS
```
export NUM_WORKERS=4
litellm
```

--config

Short form: -c
Default: None
Path to the proxy configuration file (e.g., config.yaml).
Usage:
```
litellm --config path/to/config.yaml
```

--log_config

Default: None
Type: str
Path to the logging configuration file for uvicorn.

Usage:

litellm --log_config path/to/log_config.conf

--keepalive_timeout

Default: None
Type: int
Set the uvicorn keepalive timeout in seconds (uvicorn timeout_keep_alive parameter).
Usage:
```
litellm --keepalive_timeout 30
```
Usage - set Environment Variable: KEEPALIVE_TIMEOUT
```
export KEEPALIVE_TIMEOUT=30
litellm
```

--max_requests_before_restart

Default: None
Type: int
Restart worker after this many requests. This is useful for mitigating memory growth over time.
For uvicorn: maps to limit_max_requests
For gunicorn: maps to max_requests

Usage:

litellm --max_requests_before_restart 10000

Usage - set Environment Variable: MAX_REQUESTS_BEFORE_RESTART
```
export MAX_REQUESTS_BEFORE_RESTART=10000
litellm
```

Server Backend Options

--run_gunicorn

Default: False
Type: bool (Flag)
Starts proxy via gunicorn instead of uvicorn. Better for managing multiple workers in production.
Usage:
```
litellm --run_gunicorn
```

--run_hypercorn

Default: False
Type: bool (Flag)
Starts proxy via hypercorn instead of uvicorn. Supports HTTP/2.
Usage:
```
litellm --run_hypercorn
```

--skip_server_startup

Default: False
Type: bool (Flag)
Skip starting the server after setup (useful for database migrations only).
Usage:
```
litellm --skip_server_startup
```

SSL/TLS Configuration

--ssl_keyfile_path

Default: None
Type: str
Path to the SSL keyfile. Use this when you want to provide SSL certificate when starting proxy.

Usage:

litellm --ssl_keyfile_path /path/to/key.pem --ssl_certfile_path /path/to/cert.pem

Usage - set Environment Variable: SSL_KEYFILE_PATH
```
export SSL_KEYFILE_PATH=/path/to/key.pem
litellm
```

--ssl_certfile_path

Default: None
Type: str
Path to the SSL certfile. Use this when you want to provide SSL certificate when starting proxy.

Usage:

litellm --ssl_certfile_path /path/to/cert.pem --ssl_keyfile_path /path/to/key.pem

Usage - set Environment Variable: SSL_CERTFILE_PATH
```
export SSL_CERTFILE_PATH=/path/to/cert.pem
litellm
```

--ciphers

Default: None
Type: str
Ciphers to use for the SSL setup. Only used with --run_hypercorn.

Usage:

litellm --run_hypercorn --ssl_keyfile_path /path/to/key.pem --ssl_certfile_path /path/to/cert.pem --ciphers "ECDHE+AESGCM"

Model Configuration

--model or -m

Default: None
The model name to pass to LiteLLM.
Usage:
```
litellm --model gpt-3.5-turbo
```

--alias

Default: None
An alias for the model, for user-friendly reference. Use this to give a litellm model name (e.g., "huggingface/codellama/CodeLlama-7b-Instruct-hf") a more user-friendly name ("codellama").
Usage:
```
litellm --alias my-gpt-model
```

--api_base

Default: None
The API base for the model LiteLLM should call.

Usage:

litellm --model huggingface/tinyllama --api_base https://k58ory32yinf1ly0.us-east-1.aws.endpoints.huggingface.cloud

--api_version

Default: 2024-07-01-preview
For Azure services, specify the API version.

Usage:

litellm --model azure/gpt-deployment --api_version 2023-08-01 --api_base https://<your api base>"

--headers

Default: None
Headers for the API call (as JSON string).

Usage:

litellm --model my-model --headers '{"Authorization": "Bearer token"}'

--add_key

Default: None
Add a key to the model configuration.
Usage:
```
litellm --add_key my-api-key
```

--save

Type: bool (Flag)
Save the model-specific config.
Usage:
```
litellm --model gpt-3.5-turbo --save
```

Model Parameters

--temperature

Default: None
Type: float
Set the temperature for the model.
Usage:
```
litellm --temperature 0.7
```

--max_tokens

Default: None
Type: int
Set the maximum number of tokens for the model output.
Usage:
```
litellm --max_tokens 50
```

--request_timeout

Default: None
Type: int
Set the timeout in seconds for completion calls.
Usage:
```
litellm --request_timeout 300
```

--max_budget

Default: None
Type: float
Set max budget for API calls. Works for hosted models like OpenAI, TogetherAI, Anthropic, etc.
Usage:
```
litellm --max_budget 100.0
```

--drop_params

Type: bool (Flag)
Drop any unmapped params.
Usage:
```
litellm --drop_params
```

--add_function_to_prompt

Type: bool (Flag)
If a function passed but unsupported, pass it as a part of the prompt.
Usage:
```
litellm --add_function_to_prompt
```

Database Configuration

--iam_token_db_auth

Default: False
Type: bool (Flag)
Connects to an RDS database using IAM token authentication instead of a password. This is useful for AWS RDS instances that are configured to use IAM database authentication.
When enabled, LiteLLM will generate an IAM authentication token to connect to the database.
Required Environment Variables:
- DATABASE_HOST - The RDS database host
- DATABASE_PORT - The database port
- DATABASE_USER - The database user
- DATABASE_NAME - The database name
- DATABASE_SCHEMA (optional) - The database schema
Usage:
```
litellm --iam_token_db_auth
```

Usage - set Environment Variable: IAM_TOKEN_DB_AUTH

export IAM_TOKEN_DB_AUTH=True
export DATABASE_HOST=mydb.us-east-1.rds.amazonaws.com
export DATABASE_PORT=5432
export DATABASE_USER=mydbuser
export DATABASE_NAME=mydb
litellm

--use_prisma_db_push

Default: False
Type: bool (Flag)
Use prisma db push instead of prisma migrate for database schema updates. This is useful when you want to quickly sync your database schema without creating migration files.
Usage:
```
litellm --use_prisma_db_push
```

Debugging

--debug

Default: False
Type: bool (Flag)
Enable debugging mode for the input.
Usage:
```
litellm --debug
```
Usage - set Environment Variable: DEBUG
```
export DEBUG=True
litellm
```

--detailed_debug

Default: False
Type: bool (Flag)
Enable detailed debugging mode to view verbose debug logs.
Usage:
```
litellm --detailed_debug
```
Usage - set Environment Variable: DETAILED_DEBUG
```
export DETAILED_DEBUG=True
litellm
```

--local

Default: False
Type: bool (Flag)
For local debugging purposes.
Usage:
```
litellm --local
```

Testing & Health Checks

--test

Type: bool (Flag)
Proxy chat completions URL to make a test request to.
Usage:
```
litellm --test
```

--test_async

Default: False
Type: bool (Flag)
Calls async endpoints /queue/requests and /queue/response.
Usage:
```
litellm --test_async
```

--num_requests

Default: 10
Type: int
Number of requests to hit async endpoint with (used with --test_async).
Usage:
```
litellm --test_async --num_requests 100
```

--health

Type: bool (Flag)
Runs a health check on all models in config.yaml.
Usage:
```
litellm --health
```

Other Options

--version

Short form: -v
Type: bool (Flag)
Print LiteLLM version and exit.
Usage:
```
litellm --version
```

--telemetry

Default: True
Type: bool
Help track usage of this feature. Turn off for privacy.
Usage:
```
litellm --telemetry False
```

--use_queue

Default: False
Type: bool (Flag)
To use celery workers for async endpoints.
Usage:
```
litellm --use_queue
```

Server Configuration​

--host​

--port​

--num_workers​

--config​

--log_config​

--keepalive_timeout​

--max_requests_before_restart​

Server Backend Options​

--run_gunicorn​

--run_hypercorn​

--skip_server_startup​

SSL/TLS Configuration​

--ssl_keyfile_path​

--ssl_certfile_path​

--ciphers​

Model Configuration​

--model or -m​

--alias​

--api_base​

--api_version​

--headers​

--add_key​

--save​

Model Parameters​

--temperature​

--max_tokens​

--request_timeout​

--max_budget​

--drop_params​

--add_function_to_prompt​

Database Configuration​

--iam_token_db_auth​

--use_prisma_db_push​

Debugging​

--debug​

--detailed_debug​

--local​

Testing & Health Checks​

--test​

--test_async​

--num_requests​

--health​

Other Options​

--version​

--telemetry​

--use_queue​

Server Configuration

--host

--port

--num_workers

--config

--log_config

--keepalive_timeout

--max_requests_before_restart

Server Backend Options

--run_gunicorn

--run_hypercorn

--skip_server_startup

SSL/TLS Configuration

--ssl_keyfile_path

--ssl_certfile_path

--ciphers

Model Configuration

--model or -m

--alias

--api_base

--api_version

--headers

--add_key

--save

Model Parameters

--temperature

--max_tokens

--request_timeout

--max_budget

--drop_params

--add_function_to_prompt

Database Configuration

--iam_token_db_auth

--use_prisma_db_push

Debugging

--debug

--detailed_debug

--local

Testing & Health Checks

--test

--test_async

--num_requests

--health

Other Options

--version

--telemetry

--use_queue