Skip to main content

Web Search

Use web search with litellm

FeatureDetails
Supported Endpoints- /chat/completions
- /responses
- /images/generations (Gemini image models only)
Supported Providersopenai, xai, vertex_ai, anthropic, gemini, perplexity
LiteLLM Cost Tracking✅ Supported
LiteLLM Versionv1.71.0+

Which Search Engine is Used?​

Each provider uses their own search backend:

ProviderSearch EngineNotes
OpenAI (gpt-5-search-api, gpt-4o-search-preview, gpt-4o-mini-search-preview)OpenAI's internal searchReal-time web data
xAI (grok-3)xAI's search + X/TwitterReal-time social media data
Google AI/Vertex (gemini-2.0-flash)Google SearchUses actual Google search results
Anthropic (claude-3-5-sonnet)Anthropic's web searchReal-time web data
PerplexityPerplexity's search engineAI-powered search and reasoning
Important: Only Search Models Support web_search_options

For OpenAI, only dedicated search models support the web_search_options parameter:

  • gpt-4o-search-preview
  • gpt-4o-mini-search-preview
  • gpt-5-search-api

Regular models like gpt-6.1-sol and gpt-6-luna do not support web_search_options

The web_search_options parameter is optional

Search models (like gpt-4o-search-preview) automatically search the web even without the web_search_options parameter.

Use web_search_options when you need to:

  • Adjust search_context_size ("low", "medium", "high")
  • Specify user_location for localized results
info

Anthropic Web Search Models: Claude models that support web search: claude-3-5-sonnet-latest, claude-3-5-sonnet-20241022, claude-3-5-haiku-latest, claude-3-5-haiku-20241022, claude-3-7-sonnet-20250219

OpenAI Web Search: Two Approaches​

OpenAI offers two distinct ways to use web search depending on the endpoint and model:

ApproachEndpointModelsHow to enable
Search Models/chat/completionsgpt-5-search-api, gpt-4o-search-preview, gpt-4o-mini-search-previewPass web_search_options parameter
Web Search Tool/responsesgpt-5, gpt-4.1, gpt-4o, and other regular modelsPass web_search_preview tool
Search models search automatically

Search models like gpt-5-search-api automatically search the web even without the web_search_options parameter. Use web_search_options to set search_context_size ("low", "medium", "high") or specify user_location for localized results.

/chat/completions (litellm.completion)​

Quick Start​

from litellm import completion

response = completion(
    model="openai/gpt-5-search-api",
    messages=[
        {
            "role": "user",
            "content": "What was a positive news story from today?",
        }
    ],
    web_search_options={
        "search_context_size": "medium"  # Options: "low", "medium", "high"
    }
)

Search context size​

OpenAI (using web_search_options)

from litellm import completion

# Customize search context size
response = completion(
    model="openai/gpt-5-search-api",
    messages=[
        {
            "role": "user",
            "content": "What was a positive news story from today?",
        }
    ],
    web_search_options={
        "search_context_size": "low"  # Options: "low", "medium" (default), "high"
    }
)

xAI (using web_search_options)

xAI no longer runs web search on its own /chat/completions, so when web_search_options is set LiteLLM sends the request to xAI's Responses API with a web_search tool and converts the result back to a chat completion. xAI does not support search_context_size, so it is dropped

from litellm import completion

response = completion(
    model="xai/grok-3",
    messages=[
        {
            "role": "user",
            "content": "What was a positive news story from today?",
        }
    ],
    web_search_options={}  # search_context_size is ignored for xAI
)

Anthropic (using web_search_options)

from litellm import completion

# Customize search context size for Anthropic
response = completion(
    model="anthropic/claude-sonnet-5",
    messages=[
        {
            "role": "user",
            "content": "What was a positive news story from today?",
        }
    ],
    web_search_options={
        "search_context_size": "medium",  # Options: "low", "medium" (default), "high"
        "user_location": {
            "type": "approximate",
            "approximate": {
                "city": "San Francisco",
            },
        }
    }
)

VertexAI/Gemini (using web_search_options)

from litellm import completion

# Customize search context size for Gemini
response = completion(
    model="gemini-3.8-flash",
    messages=[
        {
            "role": "user",
            "content": "What was a positive news story from today?",
        }
    ],
    web_search_options={
        "search_context_size": "low"  # Options: "low", "medium" (default), "high"
    }
)

Gemini image generation (using web_search_options on /images/generations)

from litellm import image_generation

response = image_generation(
    model="gemini/gemini-3.1-flash-image-preview",
    prompt="Generate an image of the latest iPhone design",
    web_search_options={},
)

Works with vertex_ai/gemini-3.1-flash-image-preview and other Gemini image models. See Vertex AI Image Generation and Google AI Studio Image Generation.

/responses (litellm.responses)​

Use the web_search_preview tool with models like gpt-6.1-sol, gpt-6-luna, etc.

info

Search-dedicated models like gpt-5-search-api and gpt-4o-search-preview do not support the /responses endpoint. Use them with /chat/completions + web_search_options instead (see above).

Quick Start​

from litellm import responses

response = responses(
    model="openai/gpt-6.1-sol",
    input="What is the capital of France?",
    tools=[{
        "type": "web_search_preview"  # enables web search with default medium context size
    }]
)

Search context size​

from litellm import responses

# Customize search context size
response = responses(
    model="openai/gpt-6.1-sol",
    input="What is the capital of France?",
    tools=[{
        "type": "web_search_preview",
        "search_context_size": "low"  # Options: "low", "medium" (default), "high"
    }]
)

Configuring Web Search in config.yaml​

You can set default web search options directly in your proxy config file:

model_list:
  # Enable web search by default for all requests to this model
  - model_name: grok-3
    litellm_params:
      model: xai/grok-3
      api_key: os.environ/XAI_API_KEY
      web_search_options: {}  # Enables web search with default settings

Advanced​

You can configure LiteLLM's router to optionally drop models that do not support WebSearch, for example

  - model_name: gpt-6.1-sol
    litellm_params:
      model: openai/gpt-6.1-sol
  - model_name: gpt-6.1-sol
    litellm_params:
      model: azure/gpt-6.1-sol
      api_base: "x.openai.azure.com/"
      api_version: 2025-03-01-preview
    model_info:
      supports_web_search: False <---- KEY CHANGE!

In this example, LiteLLM will still route LLM requests to both deployments, but for WebSearch, will solely route to OpenAI.

Note: When web_search_options is set in the config, it applies to all requests to that model. Users can still override these settings by passing web_search_options in their API requests.

Use litellm.supports_web_search(model="model_name") -> returns True if model can perform web searches

# Check OpenAI models
assert litellm.supports_web_search(model="openai/gpt-5-search-api") == True
assert litellm.supports_web_search(model="openai/gpt-4o-search-preview") == True

# Check xAI models
assert litellm.supports_web_search(model="xai/grok-3") == True

# Check Anthropic models
assert litellm.supports_web_search(model="anthropic/claude-sonnet-5") == True

# Check VertexAI models
assert litellm.supports_web_search(model="gemini-3.8-flash") == True

# Check Google AI Studio models
assert litellm.supports_web_search(model="gemini/gemini-3.8-flash") == True

Web Search Cost Tracking​

LiteLLM tracks web search costs automatically based on provider-specific billing models. The cost is added on top of the standard token-based pricing.

ProviderBilling UnitHow it works
Gemini 3.x (3-flash, 3-pro, 3.1-*)Per search queryEach internal search query is billed individually. One prompt may trigger multiple queries.
Gemini 2.x (2.0-flash, 2.5-flash, 2.5-pro)Per grounded promptFlat fee per API call that uses grounding, regardless of how many queries are executed internally.
OpenAI (gpt-4o-search, gpt-5-search)Per search context sizeCost varies by search_context_size (low, medium, high).
Anthropic (Claude with web search)Per search requestFixed cost per web search tool invocation.
Perplexity (sonar, sonar-pro)Per search context sizeCost varies by search_context_size.

Pricing configuration​

Web search costs are defined in model_prices_and_context_window.json using two fields:

  • search_context_cost_per_query: the cost per billable unit (per search context size tier).
  • web_search_billing_unit (on Gemini models): "per_query" (each search query is billed individually) or "per_prompt" (default; flat fee per API call that uses search).
{
    "gemini/gemini-3.8-flash": {
        "web_search_billing_unit": "per_query",
        "search_context_cost_per_query": {
            "search_context_size_low": 0.014,
            "search_context_size_medium": 0.014,
            "search_context_size_high": 0.014
        }
    },
    "gemini/gemini-2.5-flash": {
        "search_context_cost_per_query": {
            "search_context_size_low": 0.035,
            "search_context_size_medium": 0.035,
            "search_context_size_high": 0.035
        }
    }
}
info

Models without web_search_billing_unit default to "per_prompt": one flat charge per API call that uses web search, regardless of how many internal queries the model executes.

You can override these in your proxy config using model_info:

model_list:
  - model_name: gemini-3.8-flash
    litellm_params:
      model: gemini/gemini-3.8-flash
    model_info:
      web_search_billing_unit: per_query
      search_context_cost_per_query:
        search_context_size_low: 0.014
        search_context_size_medium: 0.014
        search_context_size_high: 0.014

Google's free grounding allowance​

Google gives every billing account a free allowance of Google Search grounding before the per-query price applies: 5,000 queries per month shared across the Gemini 3 models, 1,500 grounded prompts per day shared across Gemini 2.0 Flash, 2.5 Flash and 2.5 Flash-Lite, and 10,000 grounded prompts per day for Gemini 2.5 Pro. By default LiteLLM bills grounding from the first query, so logged grounding spend is an upper bound on the Google invoice.

The cost map describes each allowance with search_free_requests on the Gemini rows. count is the free units per period, period is "day" or "month", and models with the same shared_group draw from one pool, counted in the row's web_search_billing_unit.

{
    "gemini/gemini-3.8-flash": {
        "web_search_billing_unit": "per_query",
        "search_free_requests": {"count": 5000, "period": "month", "shared_group": "gemini-3"}
    }
}

The field does nothing on its own. To have the proxy log grounding at $0 until the shared allowance is used up, opt in with apply_search_free_tier. The proxy keeps one counter per shared_group and UTC period in its Redis, the same Redis it uses for budgets, so every proxy instance draws from the same pool and a restart does not reset it. A request that crosses the allowance is split: the queries still inside it are free and the rest are billed at search_context_cost_per_query. The spend row keeps the full price in metadata.cost_breakdown.original_cost and the freed amount in discount_amount.

litellm_settings:
  apply_search_free_tier: true   # default false
  cache: true
  cache_params:
    type: redis
    host: os.environ/REDIS_HOST
    port: os.environ/REDIS_PORT

Two things to know before turning it on. The allowance is per Google billing account and LiteLLM only sees its own traffic, so if anything else on the account also grounds, LiteLLM frees queries that Google billed and the logged spend becomes a lower bound inside the allowance window; leave the setting off if other workloads share the account. Google does not publish the time zone its allowance resets in, so LiteLLM resets on the UTC calendar day and month. Without Redis, or when the counter cannot be reached, the request is billed at full price and a warning is logged. The x-litellm-response-cost header and logging callbacks still report the pre-allowance cost; only the spend row, budgets and spend counters see the reduced spend.

How LiteLLM tracks search usage​

The number of web search requests is stored in usage.prompt_tokens_details.web_search_requests. LiteLLM extracts this from each provider's response:

  • Gemini: Extracted from groundingMetadata.webSearchQueries in the response. For Gemini 2.x, clamped to 1 (per-prompt billing).
  • OpenAI: Reported directly in the usage metadata.
  • Anthropic: Reported via server_tool_use.web_search_requests.
  • xAI: Reported in usage.server_side_tool_usage_details.web_search_calls rather than web_search_requests. num_sources_used is not read. When xAI returns usage.cost, LiteLLM uses it as the response cost, which already includes the search charge
response = litellm.completion(
    model="gemini/gemini-3.8-flash",
    messages=[{"role": "user", "content": "Latest tech news?"}],
    web_search_options={"search_context_size": "medium"},
)

# Check web search usage
print(response.usage.prompt_tokens_details.web_search_requests)  # e.g., 3

# Get total cost (includes token cost + web search cost)
cost = litellm.completion_cost(completion_response=response)
print(f"Total cost: ${cost}")
LiteLLM Enterprise
SSO/SAML, audit logs, spend tracking, multi-team management, and guardrails, built for production.
Learn more →