Connect corporate users and applications through Cloud VPN, Interconnect, or an approved private access service. Use private DNS and a regional internal Application Load Balancer.
Create a private frontend subnet and a separate proxy-only subnet with purpose REGIONAL_MANAGED_PROXY and role ACTIVE. Allocate the frontend IP from the frontend subnet. See internal load balancer setup.
Corporate VPN tunnels or Interconnect attachments must connect in the load balancer's region unless you enable global access. Add private DNS and routes for the frontend IP. See internal load balancer client access.
Corporate clients use an internal load balancer. Cloud Run stays outside the customer VPC. Its connector reaches private data stores and Cloud NAT.
Architecture details and references
Required module changes
The module creates a global external load balancer. Replace it with regional INTERNAL_MANAGED resources, a private frontend address, and a REGIONAL_MANAGED_PROXY subnet. Keep the three serverless NEGs and path routes. Use regional TLS certificates. Set every Cloud Run service to INGRESS_TRAFFIC_INTERNAL_ONLY. The cloud guide lists the resource changes.
Corporate access
Connect corporate clients through Cloud VPN, Interconnect, or an approved private access service. Publish private DNS. Keep regional clients in the load balancer region or configure global access. Add identity checks and keep LiteLLM key permissions. Remove the public load balancer and unused Cloud Run domain mappings.
One image
The gateway, backend, UI, and migration job use the same image with a different startup argument for each component.
Separate ingress and egress
Serverless NEGs carry inbound requests through Google-managed routes. They do not use the VPC connector. Gateway, backend, and migration egress uses the connector. Cloud Run stays outside the customer VPC. Cloud SQL and Memorystore use Private Services Access to reach Google-managed producer networks.
NAT egress changes
The module does not create Cloud NAT or Cloud Router and sets PRIVATE_RANGES_ONLY. Create Cloud Router and Cloud NAT for the connector subnet. Set connector egress to ALL_TRAFFIC. Cloud Router configures NAT. Traffic does not pass through Cloud Router.
The source module creates a global external load balancer. It has no input to select an internal frontend. Make these changes to the module before deployment:
Replace the global address with a regional internal address. Use google_compute_region_backend_service, google_compute_region_url_map, google_compute_region_target_https_proxy, and google_compute_forwarding_rule. Set the backend services and forwarding rule to INTERNAL_MANAGED. Set the forwarding rule's network, subnet, private IP, and port 443. Keep all three serverless NEGs and existing path routes.
TLS certificate
Replace the global managed certificate with a supported regional certificate. Use a regional Compute Engine SSL certificate or a regional Certificate Manager certificate.
Set all three services to INGRESS_TRAFFIC_INTERNAL_ONLY. Set connector egress to ALL_TRAFFIC for gateway, backend, and migrations. Remove unused public domain mappings.
Update lb_ip and lb_url to use the regional internal address.
Remove the old public forwarding rules and address. Cloud Run stays outside the customer subnet. The inbound serverless NEG path does not use the outbound VPC connector. See Cloud Run ingress and regional TLS support.
If you use IAP, grant Cloud Run Invoker to the IAP service agent and remove the module's allUsers invoker grants. Keep the Invoker IAM check enabled. Configure IAP on the load balancer or directly on Cloud Run, but not both. See Google Cloud IAP.
Corporate clients use an internal load balancer. GKE workloads reach private data stores through peering. Cloud NAT handles public provider requests.
Architecture details and references
Corporate access
Use Cloud VPN, Interconnect, or an approved private access service and private DNS. Create a regional internal Application Load Balancer with a private frontend and proxy-only subnet. Restrict access to corporate clients. Configure global access if clients connect from another region.
One Helm chart
The litellm chart uses one image for all components. This layout uses monolith.enabled=false.
GKE ingress
Use internal GKE Ingress with the gce-internal class or the gke-l7-rilb GatewayClass. The controller creates zonal NEG backends. Allow the required proxy-only subnet and health-check traffic to pods. Keep nodes and the cluster endpoint private. Create these resources around the Helm workloads.
Private node egress
Configure Cloud NAT for the node and pod ranges that need public provider APIs or registries. Private Google Access serves supported Google APIs. Keep private database routes separate from NAT.
Terraform data stores
Set create_runtime=false to create data stores with the Google module. This skips Cloud Run, its load balancer, VPC connector, and migration job. Create GKE and configure its network separately.
Create a private GKE cluster and the network resources above. Use gce-internal Ingress or the gke-l7-rilb GatewayClass. Allow traffic from the proxy-only subnet and required health-check ranges to the pod backends. Configure routes for gateway, backend, and UI. Keep the cluster endpoint private. See GKE internal Ingress and Gateway setup.
Use Cloud NAT for node and pod ranges that need public model APIs or registries. Keep Private Services Access for Cloud SQL and Memorystore. The Terraform module's create_runtime = false mode can create the data stores without Cloud Run or a public load balancer.
