GradientAI

https://digitalocean.com/products/gradientai

LiteLLM provides native support for GradientAI models. To use a GradientAI model, specify it as gradient_ai/<model-name> in your LiteLLM requests.

API Key & Endpoint

Set your API key as an environment variable. Requests go to the serverless inference endpoint https://inference.do-ai.run/v1/chat/completions by default, so no endpoint variable is needed:

import os
os.environ['GRADIENT_AI_API_KEY'] = "your-api-key"

To call a GradientAI agent instead, set GRADIENT_AI_AGENT_ENDPOINT (or pass api_base) to the agent's base URL. LiteLLM appends /api/v1/chat/completions to it, so do not include a path:

os.environ['GRADIENT_AI_AGENT_ENDPOINT'] = "https://<agent-id>.agents.do-ai.run"  # optional, agents only

Sample Usage

from litellm import completion
import os

os.environ['GRADIENT_AI_API_KEY'] = "your-api-key"
response = completion(
    model="gradient_ai/model-name",
    messages=[
        {"role": "user", "content": "Hello, how are you?"}
    ],
)
print(response.choices[0].message.content)

Streaming Example

from litellm import completion
import os

os.environ['GRADIENT_AI_API_KEY'] = "your-api-key"
response = completion(
    model="gradient_ai/model-name",
    messages=[
        {"role": "user", "content": "Write a story about a robot learning to love"}
    ],
    stream=True,
)

for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")

Supported Parameters

Parameter	Type	Description
temperature	float	Controls randomness (0.0-2.0)
top_p	float	Nucleus sampling parameter (0.0-1.0)
max_tokens	int	Maximum tokens to generate
max_completion_tokens	int	Alternative to max_tokens
stream	bool	Whether to stream the response
k	int	Top results to return from knowledge bases
retrieval_method	string	Retrieval strategy (rewrite/step_back/sub_queries/none)
frequency_penalty	float	Penalizes repeated tokens (-2.0 to 2.0)
presence_penalty	float	Penalizes tokens based on presence (-2.0 to 2.0)
stop	string/list	Sequences to stop generation
kb_filters	List[Dict]	Filters for knowledge base retrieval
instruction_override	string	Override agent's default instruction
include_retrieval_info	bool	Include document retrieval metadata
include_guardrails_info	bool	Include guardrail trigger metadata
provide_citations	bool	Include citations in response

---

For more details, see DigitalOcean GradientAI documentation.