LiteLLM is moving to Rust Read the latest updates.
LiteLLM v1.110.2 and v1.111.0rc3 fix Responses stream restarts
Released
Deploy this version​
- Docker
- Pip
docker run \
-e LITELLM_MASTER_KEY=sk-<paste-a-long-random-key> \
-e DATABASE_URL=postgresql://<user>:<password>@<host>:5432/<dbname> \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
docker.litellm.ai/berriai/litellm:1.110.2
pip install litellm==1.110.2
This release is published as ghcr.io/berriai/litellm:v1.110.2. See the GitHub release and the full releases page.
v1.110.2 is a patch release on top of v1.110.1. There are no new database migrations. The v1.110.2 tag points at c3cafa3.
What's Changed​
- Anthropic: Fixed
temperature=1returning 400 on Claude models that removed sampling params on Anthropic, Vertex AI and Azure AI, broken since v1.110.0-rc.2 - PR #52341 - Responses API: Fixed streams retrying after output reached the client, which sent duplicate events and billed extra calls - PR #52461
- Responses API: Fixed streams that fail after reasoning output restarting on a fallback group and sending a second response - PR #52616
- Caching: Fixed GPT-6.2 system prompts missing the prompt cache on
/v1/chat/completionsand/v1/messagesunlessstore: false- PR #52906 - Cost tracking and pricing: Fixed forwarded Anthropic credentials on
/v1/messagesignoring the deployment's custom prices, broken since v1.109.0 - PR #52313 - Keys, teams and auth: Fixed JWT key mappings without
jwt_issuertaking 5 minutes to apply whenlitellm_jwtauth.issuersis set, broken since v1.103.0 - PR #52350 - Keys, teams and auth: Fixed
/key/infoletting a key with no user read other userless keys by hash or alias - PR #52838 and PR #52925 - Performance: Fixed memory growth from every finished request staying in memory until a full GC pass, broken since v1.110.0-rc.1 - PR #51888
The changes are backported in PR #52463, PR #52469, PR #52690, PR #52751, PR #52920 and PR #52938.
Full Changelog​
https://github.com/BerriAI/litellm/compare/v1.110.1...v1.110.2