Tool Search
Tool search enables Claude to dynamically discover and load tools on-demand from large tool catalogs (10,000+ tools). Instead of loading all tool definitions into the context window upfront, Claude searches your tool catalog and loads only the tools it needs.
Supported Providers​
| Provider | Chat Completions API | Messages API |
|---|---|---|
| Anthropic API | ✅ | ✅ |
| Azure Anthropic (Microsoft Foundry) | ✅ | ✅ |
| Google Cloud Vertex AI | ✅ | ✅ |
| Amazon Bedrock | ✅ (Invoke API only, Opus 4.5 only) | ✅ (Invoke API only, Opus 4.5 only) |
Benefits​
- Context efficiency: Avoid consuming massive portions of your context window with tool definitions
- Better tool selection: Claude's tool selection accuracy degrades with more than 30-50 tools. Tool search maintains accuracy even with thousands of tools
- On-demand loading: Tools are only loaded when Claude needs them
Tool Search Variants​
LiteLLM supports both tool search variants:
1. Regex Tool Search (tool_search_tool_regex_20251119)​
Claude constructs regex patterns to search for tools. Best for exact pattern matching (faster).
2. BM25 Tool Search (tool_search_tool_bm25_20251119)​
Claude uses natural language queries to search for tools using the BM25 algorithm. Best for natural language semantic search.
Note: BM25 variant is not supported on Bedrock.
Chat Completions API​
SDK Usage​
Basic Example with Regex Tool Search​
Basic Tool Search Example
import litellm
response = litellm.completion(
model="anthropic/claude-sonnet-4-5-20250929",
messages=[
{"role": "user", "content": "What is the weather in San Francisco?"}
],
tools=[
# Tool search tool (regex variant)
{
"type": "tool_search_tool_regex_20251119",
"name": "tool_search_tool_regex"
},
# Deferred tool - will be loaded on-demand
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the weather at a specific location",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string"},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"]
}
},
"required": ["location"]
}
},
"defer_loading": True # Mark for deferred loading
}
]
)
print(response.choices[0].message.content)
BM25 Tool Search Example​
BM25 Tool Search
import litellm
response = litellm.completion(
model="anthropic/claude-sonnet-4-5-20250929",
messages=[
{"role": "user", "content": "Search for Python files containing 'authentication'"}
],
tools=[
# Tool search tool (BM25 variant)
{
"type": "tool_search_tool_bm25_20251119",
"name": "tool_search_tool_bm25"
},
# Deferred tools...
{
"type": "function",
"function": {
"name": "search_codebase",
"description": "Search through codebase files by content and filename",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string"},
"file_pattern": {"type": "string"}
},
"required": ["query"]
}
},
"defer_loading": True
}
]
)
Azure Anthropic Example​
Azure Anthropic Tool Search
import litellm
response = litellm.completion(
model="azure_anthropic/claude-sonnet-4-5",
api_base="https://<your-resource>.services.ai.azure.com/anthropic",
api_key="your-azure-api-key",
messages=[
{"role": "user", "content": "What's the weather like?"}
],
tools=[
{
"type": "tool_search_tool_regex_20251119",
"name": "tool_search_tool_regex"
},
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string"}
},
"required": ["location"]
}
},
"defer_loading": True
}
]
)
Vertex AI Example​
Vertex AI Tool Search
import litellm
response = litellm.completion(
model="vertex_ai/claude-sonnet-4-5",
vertex_project="your-project-id",
vertex_location="us-central1",
messages=[
{"role": "user", "content": "Search my documents"}
],
tools=[
{
"type": "tool_search_tool_bm25_20251119",
"name": "tool_search_tool_bm25"
},
# Your deferred tools...
]
)
Streaming Support​
Streaming with Tool Search
import litellm
response = litellm.completion(
model="anthropic/claude-sonnet-4-5-20250929",
messages=[
{"role": "user", "content": "Get the weather"}
],
tools=[
{
"type": "tool_search_tool_regex_20251119",
"name": "tool_search_tool_regex"
},
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get weather information",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string"}
},
"required": ["location"]
}
},
"defer_loading": True
}
],
stream=True
)
for chunk in response:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")
AI Gateway Usage​
Tool search works automatically through the LiteLLM proxy.
Proxy Configuration​
config.yaml
model_list:
- model_name: claude-sonnet
litellm_params:
model: anthropic/claude-sonnet-4-5-20250929
api_key: os.environ/ANTHROPIC_API_KEY
Client Request​
Client Request via Proxy
from anthropic import Anthropic
client = Anthropic(
api_key="your-litellm-proxy-key",
base_url="http://0.0.0.0:4000"
)
response = client.messages.create(
model="claude-sonnet",
max_tokens=1024,
messages=[
{"role": "user", "content": "What's the weather?"}
],
tools=[
{
"type": "tool_search_tool_regex_20251119",
"name": "tool_search_tool_regex"
},
{
"name": "get_weather",
"description": "Get weather information",
"input_schema": {
"type": "object",
"properties": {
"location": {"type": "string"}
},
"required": ["location"]
},
"defer_loading": True
}
]
)
Messages API​
The Messages API provides native Anthropic-style tool search support via the litellm.anthropic.messages interface.
SDK Usage​
Basic Example​
Messages API - Basic Tool Search
import litellm
response = await litellm.anthropic.messages.acreate(
model="anthropic/claude-sonnet-4-20250514",
messages=[
{
"role": "user",
"content": "What's the weather in San Francisco?"
}
],
tools=[
{
"type": "tool_search_tool_regex_20251119",
"name": "tool_search_tool_regex"
},
{
"name": "get_weather",
"description": "Get the current weather for a location",
"input_schema": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
}
},
"required": ["location"]
},
"defer_loading": True
}
],
max_tokens=1024,
extra_headers={"anthropic-beta": "advanced-tool-use-2025-11-20"}
)
print(response)
Azure Anthropic Messages Example​
Azure Anthropic Messages API
import litellm
response = await litellm.anthropic.messages.acreate(
model="azure_anthropic/claude-sonnet-4-20250514",
messages=[
{
"role": "user",
"content": "What's the stock price of Apple?"
}
],
tools=[
{
"type": "tool_search_tool_regex_20251119",
"name": "tool_search_tool_regex"
},
{
"name": "get_stock_price",
"description": "Get the current stock price for a ticker symbol",
"input_schema": {
"type": "object",
"properties": {
"ticker": {
"type": "string",
"description": "The stock ticker symbol, e.g. AAPL"
}
},
"required": ["ticker"]
},
"defer_loading": True
}
],
max_tokens=1024,
extra_headers={"anthropic-beta": "advanced-tool-use-2025-11-20"}
)
Vertex AI Messages Example​
Vertex AI Messages API
import litellm
response = await litellm.anthropic.messages.acreate(
model="vertex_ai/claude-sonnet-4@20250514",
messages=[
{
"role": "user",
"content": "Search the web for information about AI"
}
],
tools=[
{
"type": "tool_search_tool_bm25_20251119",
"name": "tool_search_tool_bm25"
},
{
"name": "search_web",
"description": "Search the web for information",
"input_schema": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "The search query"
}
},
"required": ["query"]
},
"defer_loading": True
}
],
max_tokens=1024,
extra_headers={"anthropic-beta": "tool-search-tool-2025-10-19"}
)
Bedrock Messages Example​
Bedrock Messages API (Invoke)
import litellm
response = await litellm.anthropic.messages.acreate(
model="bedrock/invoke/anthropic.claude-opus-4-20250514-v1:0",
messages=[
{
"role": "user",
"content": "What's the weather?"
}
],
tools=[
{
"type": "tool_search_tool_regex_20251119",
"name": "tool_search_tool_regex"
},
{
"name": "get_weather",
"description": "Get weather information",
"input_schema": {
"type": "object",
"properties": {
"location": {"type": "string"}
},
"required": ["location"]
},
"defer_loading": True
}
],
max_tokens=1024,
extra_headers={"anthropic-beta": "tool-search-tool-2025-10-19"}
)
Streaming Support​
Messages API - Streaming
import litellm
import json
response = await litellm.anthropic.messages.acreate(
model="anthropic/claude-sonnet-4-20250514",
messages=[
{
"role": "user",
"content": "What's the weather in Tokyo?"
}
],
tools=[
{
"type": "tool_search_tool_regex_20251119",
"name": "tool_search_tool_regex"
},
{
"name": "get_weather",
"description": "Get weather information",
"input_schema": {
"type": "object",
"properties": {
"location": {"type": "string"}
},
"required": ["location"]
},
"defer_loading": True
}
],
max_tokens=1024,
stream=True,
extra_headers={"anthropic-beta": "advanced-tool-use-2025-11-20"}
)
async for chunk in response:
if isinstance(chunk, bytes):
chunk_str = chunk.decode("utf-8")
for line in chunk_str.split("\n"):
if line.startswith("data: "):
try:
json_data = json.loads(line[6:])
print(json_data)
except json.JSONDecodeError:
pass
AI Gateway Usage​
Configure the proxy to use Messages API endpoints.
Proxy Configuration​
config.yaml
model_list:
- model_name: claude-sonnet-messages
litellm_params:
model: anthropic/claude-sonnet-4-20250514
api_key: os.environ/ANTHROPIC_API_KEY
Client Request​
Client Request via Proxy (Messages API)
from anthropic import Anthropic
client = Anthropic(
api_key="your-litellm-proxy-key",
base_url="http://0.0.0.0:4000"
)
response = client.messages.create(
model="claude-sonnet-messages",
max_tokens=1024,
messages=[
{
"role": "user",
"content": "What's the weather?"
}
],
tools=[
{
"type": "tool_search_tool_regex_20251119",
"name": "tool_search_tool_regex"
},
{
"name": "get_weather",
"description": "Get weather information",
"input_schema": {
"type": "object",
"properties": {
"location": {"type": "string"}
},
"required": ["location"]
},
"defer_loading": True
}
],
extra_headers={"anthropic-beta": "advanced-tool-use-2025-11-20"}
)
print(response)