OCR uses Rust by default starting with v1.102.0-rc.1
Starting with v1.102.0-rc.1, OCR calls use the Rust implementation by default while preserving the existing API.
Rust Migration
LiteLLM is moving to Rust 🦀
Read the latest engineering updates, benchmarks, and rollout notes.
Benchmarking the LiteLLM Rust AI Gateway: Overhead, Memory, and Cost
AIGatewayBench measures the overhead an AI gateway adds on top of the upstream model, isolated against a deterministic mock, across LiteLLM (Rust), LiteLLM (Python v1), Portkey, and Bifrost. The LiteLLM Rust gateway has the lowest overhead and memory footprint of the four.
Migrating LiteLLM to Rust - Building the Fastest and Litest AI Gateway
LiteLLM is moving its AI gateway to Rust: 15x throughput, 11x less memory, and sub-1ms per-request overhead. No v2, no migration, your config stays the same.