LiteLLM is moving to Rust Read the latest updates.
LiteLLM v1.110.2 and v1.111.0rc3 fix Responses stream restarts
Released
Deploy this version​
- Docker
- Pip
docker run \
-e LITELLM_MASTER_KEY=sk-<paste-a-long-random-key> \
-e DATABASE_URL=postgresql://<user>:<password>@<host>:5432/<dbname> \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
docker.litellm.ai/berriai/litellm:1.111.0-rc.3
pip install litellm==1.111.0rc3
This release is published as ghcr.io/berriai/litellm:v1.111.0-rc.3. See the GitHub release and the full releases page.
v1.111.0rc3 is a release candidate on top of v1.111.0rc2. There are no new database migrations. The v1.111.0-rc.3 tag points at 5608526.
What's Changed​
- Anthropic: Fixed
temperature=1returning 400 on Claude models that removed sampling params on Anthropic, Vertex AI and Azure AI, broken since v1.110.0-rc.2 - PR #52341 - Responses API: Fixed streams retrying after output reached the client, which sent duplicate events and billed extra calls - PR #52461
- Responses API: Fixed streams that fail after reasoning output restarting on a fallback group and sending a second response - PR #52616
- Caching: Fixed GPT-6.2 system prompts missing the prompt cache on
/v1/chat/completionsand/v1/messagesunlessstore: false- PR #52906 - Cost tracking and pricing: Fixed forwarded Anthropic credentials on
/v1/messagesignoring the deployment's custom prices, broken since v1.109.0 - PR #52313 - Keys, teams and auth: Fixed 503s on a burst of first requests for new customer org aliases, which left their spend untracked - PR #52080
- Keys, teams and auth: Fixed JWT key mappings without
jwt_issuertaking 5 minutes to apply whenlitellm_jwtauth.issuersis set, broken since v1.103.0 - PR #52350 - Keys, teams and auth: Fixed
/key/infoletting a key with no user read other userless keys by hash or alias - PR #52838 and PR #52925 - Admin UI: Fixed SSO login redirecting to an
http://URL when the proxy runs behind TLS termination, broken since v1.108.0 - PR #52213 - Deployment: Fixed each proxy worker opening six extra Postgres connections at boot - PR #52378
- Performance: Fixed failed calls holding large request bodies in memory until a full GC, growing workers to several GiB - PR #52600
The changes are backported in PR #52462, PR #52470, PR #52691, PR #52752, PR #52764, PR #52914 and PR #52939.
Full Changelog​
https://github.com/BerriAI/litellm/compare/v1.111.0-rc.2...v1.111.0-rc.3