---
title: "CompatCanary"
url: "/docs/projects/CompatCanary"
canonical_url: "https://docs.litellm.ai/docs/projects/CompatCanary"
type: "docs"
last_updated: "2026-10-09"
summary: "CompatCanary is an open-source conformance scanner for OpenAI-compatible endpoints. It sends deterministic probes through a LiteLLM proxy and records observed behavior for chat completions, streaming, tool calls, structured outputs, and..."
related:
  - "/docs/projects/Harbor"
  - "/docs/projects/GraphRAG"
---
# CompatCanary

> Index of all LiteLLM docs: https://docs.litellm.ai/llms.txt


[CompatCanary](https://github.com/CognizenOrg/compatcanary) is an open-source conformance scanner for OpenAI-compatible endpoints. It sends deterministic probes through a LiteLLM proxy and records observed behavior for chat completions, streaming, tool calls, structured outputs, and the Responses API.

Start with the Chat Completions surface exposed by a configured LiteLLM model:

```bash
export OPENAI_API_KEY="your-litellm-key"
npx --yes compatcanary@latest scan \
  --base-url http://localhost:4000/v1 \
  --model your-model \
  --profile chat
```

Use `--profile modern` to include Responses API probes, and `--format markdown` or `--format json` to save reviewable evidence for local debugging or CI.

See the [project repository](https://github.com/CognizenOrg/compatcanary) and a [reproducible LiteLLM setup with reviewed reports](https://github.com/CognizenOrg/compatcanary/blob/main/evidence/setups/litellm-1.91.1-github-models.md).

## Related pages

- [Harbor](https://docs.litellm.ai/docs/projects/Harbor.md)
- [Microsoft GraphRAG](https://docs.litellm.ai/docs/projects/GraphRAG.md)