Send text to vLLM or Watsonx through one LiteLLM tokenization interface. Your app passes a model name, such as llama-3.1-8b, and a prompt to /tokenize. Configure the provider URL and credentials in your gateway.
Choose another configured model by changing model; keep the same endpoint and JSON fields. Read the token count, numeric IDs and token strings where the provider supplies them. LiteLLM runs the provider integration in Rust and reuses shared HTTP clients.
To run the examples, use a build containing PR #48079 and its compiled Rust extension. Use token counting for context limits and input cost estimates. Choose /tokenize to inspect a text string’s token list or compare tokenizer settings across deployments.
OCTOBER 2026/tokenize
One API for
vLLM + Watsonx
Choose a model. Inspect your prompt’s tokens through the same API for vLLM and Watsonx.{
"model": "meta-llama/Llama-3.1-8B-Instruct",
"count": 2,
"token_ids": [101, 102],
"tokens": ["Hello", " world"]
}token_ids: null.
/tokenize vs. token counting
- Check prompt size. Use
/utils/token_counterto check context limits or estimate input cost. - See how your text splits. Use
/tokenizeto inspect token text and available IDs. Find why a URL takes so many tokens, or check for tokenizer changes after a vLLM rollout.
Send a prompt and read the tokens
Pick a model below and watch the model field change. Send the same JSON fields to the same endpoint for either provider.
POST /tokenize{
"model": "llama-3.1-8b",
"prompt": "Hello world"
}
model matches model_name in your gateway config. This request calls vLLM.
{
"model": "meta-llama/Llama-3.1-8B-Instruct",
"count": 2,
"token_ids": [101, 102],
"tokens": ["Hello", " world"]
}
model to choose another configured model. Keep the same endpoint and JSON fields across providers.llama-3.1-8b on vLLM selected. The request model is llama-3.1-8b.To choose between models on each call, give each one a different model_name in your gateway config, then send that name in model. Use a name your API key can access. To change providers without a client update, keep the alias and change its provider configuration.
A token ID is a vocabulary entry number
A tokenizer splits text into pieces called tokens. Each piece has an entry in that tokenizer’s vocabulary, with a number called its token ID. Click either piece below to follow the lookup.
The second piece includes the space before “world”.
| Token text | Token ID |
|---|---|
| "Hello" | 101 |
| " world" | 102 |
Count: 2 tokens. Count the entries in the list. The numbers 101 and 102 identify those entries.
You selected "Hello". Look up that piece in the vocabulary to get ID 101, then put it in position 1 of the output list.
Why does a model need the number?
Models calculate with numbers. When you send a prompt for inference, the model uses each token ID to look up a learned vector: a list of numbers that represents that token. It processes those vectors to predict the next token. A /tokenize call stops at the token list; it does not generate a reply.
These IDs are made up to show the lookup. Your tokenizer can split the text differently and use different numbers. An ID only makes sense with its matching vocabulary. vLLM returns IDs; Watsonx returns token text with token_ids: null.
Built in Rust
Rust handles authentication, provider requests and response translation. We measured the SDK’s native request path against a local test server.
- 95th percentile
- 0.61 ms
- Throughput
- 1,870requests / sec
Animation illustrates the path, not request timing.
Public async SDK calls through the vLLM adapter with a fixed two-token response. Includes the Python/Rust bridge, loopback HTTP and response translation. Excludes real token computation, internet latency and the proxy server.
Test conditions and raw measurements
Apple M4 Max, macOS 15.5, Python 3.12.13. Optimized Rust release build at 5f68bf4.
4,096 requests per trial, 3 trials per concurrency level, after 256 warmup calls. We show the trial with median throughput and that trial’s latency percentiles. Workers start their next call after the previous call completes.
Read the raw measurementsCall /tokenize
Choose a provider and copy the setup. Use a gateway alias to keep provider configuration out of your client.
Use a model available on your vLLM deployment and set the environment variables below; omit api_key if your vLLM server does not require it.
model_list:
- model_name: llama-3.1-8b
litellm_params:
model: hosted_vllm/meta-llama/Llama-3.1-8B-Instruct
api_base: os.environ/HOSTED_VLLM_API_BASE
api_key: os.environ/HOSTED_VLLM_API_KEY
general_settings:
master_key: os.environ/LITELLM_MASTER_KEY
curl http://localhost:4000/tokenize \
--fail-with-body \
-H "Authorization: Bearer $LITELLM_MASTER_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "llama-3.1-8b",
"prompt": "Hello world"
}'
Start with litellm --config config.yaml --port 4000. Change the provider configuration and restart to use the same alias with another provider.
Supported providers and output fields
Keep your parsing code when you switch providers. Token values and counts depend on the model; numeric IDs do not transfer between tokenizers.
| Provider | Count | Numeric token IDs | Token strings |
|---|---|---|---|
| vLLM | Returned | Returned | When supplied by the server |
| Watsonx | Returned | null | Returned by default |
Both /tokenize and /v1/tokenize return model, count, token_ids and tokens. Send one text prompt per call. For count-only workflows, see token counting.