Inspect AI alternatives
Inspect AI covers evaluation and starts at Free; its headline PyPI package had 907,448 downloads in the latest measured week (rank 8). 52 other listed tools share at least one category; 27 of them can be self-hosted on any plan and 40 ingest OpenTelemetry natively. The table is ordered by PyPI package downloads, not by payment.
Week 2026-W40 (2026-09-28 to 2026-10-04, UTC), measured 2026-10-07 20:16 UTC.
Same categories as Inspect AI
| # | Tool | Package | PyPI downloads (week) | 12 weeks | vs prior week | Licence | Self-host | OTel | Entry price |
|---|---|---|---|---|---|---|---|---|---|
| 2 | LangSmithTracing & observability | langsmithbundled | 18,866,051 | +8.1% | Proprietary | Enterprise only | Native (OTLP) | $39/seat/mo | |
| 3 | LangfuseTracing & observability | langfuse | 6,306,471 | +12.7% | MIT (open core) | Yes | Native (OTLP) | $29/mo | |
| 4 | MLflowTracing & observability | mlflow-tracingbundled | 3,616,315 | +3.7% | Apache-2.0 | Yes | Native (OTLP) | Free | |
| 5 | Pydantic LogfireTracing & observability | logfirebundled | 3,488,038 | +14.3% | Proprietary | Enterprise only | Native (OTLP) | $49/mo | |
| 6 | Pydantic EvalsEvaluation | pydantic-evalsbundled | 2,121,173 | +10.5% | MIT | Yes | Native (OTLP) | Free | |
| 7 | BraintrustTracing & observability | braintrust | 1,650,096 | 0.0% | Proprietary | Enterprise only | Native (OTLP) | $249/mo | |
| 10 | DeepEval / Confident AIEvaluation | deepeval | 686,214 | +9.7% | Apache-2.0 (open core) | Enterprise only | Native (OTLP) | $200/mo | |
| 11 | Traceloop / OpenLLMetryTracing & observability | traceloop-sdk | 455,110 | +7.6% | Apache-2.0 (open core) | Enterprise only | Native (OTLP) | Contact sales | |
| 12 | OpikTracing & observability | opik | 437,574 | +2.0% | Apache-2.0 | Yes | Native (OTLP) | $19/mo | |
| 13 | RagasEvaluation | ragas | 279,757 | +9.5% | Apache-2.0 | Yes | Unknown | Free | |
| 14 | lm-evaluation-harnessEvaluation | lm-eval | 270,051 | +10.3% | MIT | Yes | Unknown | Free | |
| 15 | W&B WeaveTracing & observability | weave | 239,276 | +28.3% | Apache-2.0 (open core) | Yes | Native (OTLP) | $60/mo | |
| 16 | Evidently AIEvaluation | evidently | 198,068 | +7.3% | Apache-2.0 (open core) | Yes | Native (OTLP) | Free | |
| 17 | Arize PhoenixTracing & observability | arize-phoenix | 161,687 | +13.6% | Elastic-2.0 (source-available) | Yes | Native (OTLP) | Free | |
| 19 | Arize AXTracing & observability | arize-otel | 84,266 | -5.1% | Proprietary | Enterprise only | Native (OTLP) | $50/mo | |
| 20 | OpenLITTracing & observability | openlit | 72,503 | +14.5% | Apache-2.0 | Yes | Native (OTLP) | Free | |
| 21 | LunaryTracing & observability | lunary | 49,317 | +22.9% | Unknown | Enterprise only | Native (OTLP) | $20/seat/mo | |
| 22 | OpenlayerEvaluation | openlayer | 37,719 | -16.7% | Proprietary | Enterprise only | Native (OTLP) | Contact sales | |
| 23 | Azure AI Evaluation SDK (Microsoft Foundry)previewEvaluation | azure-ai-evaluation | 33,482 | -4.5% | MIT | Yes | Unknown | not published in USD | |
| 24 | Orq.aiGateways with observability | orq-ai-sdk | 30,379 | +14.1% | Proprietary | Enterprise only | Native (OTLP) | not published in USD | |
| 26 | PromptLayerPrompt management | promptlayer | 23,137 | +10.5% | Proprietary | Enterprise only | Native (OTLP) | $49/mo | |
| 27 | Judgment Labs (Judgeval)Tracing & observability | judgeval | 19,443 | +39.3% | Apache-2.0 (open core) | No | Native (OTLP) | Contact sales | |
| 28 | garakEvaluation | garak | 11,417 | +25.8% | Apache-2.0 | Yes | Unknown | Free | |
| 29 | AgentaPrompt management | agenta | 10,423 | +38.6% | MIT (open core) | Yes | Native (OTLP) | $29/mo | |
| 30 | GalileoEvaluation | galileo | 9,821 | +46.8% | Proprietary | Enterprise only | Native (OTLP) | Contact sales | |
| 31 | TruLensEvaluation | trulens | 6,198 | +28.0% | MIT | Yes | Native (OTLP) | Free | |
| 32 | GiskardEvaluation | giskard | 5,667 | +93.3% | Apache-2.0 (open core) | Yes | Unknown | Contact sales | |
| 33 | HeliconeTracing & observability | helicone-helpers | 4,840 | +20.7% | Apache-2.0 | Yes | Unknown | $79/mo | |
| 34 | PromptfooEvaluation | promptfoo | 3,451 | +13.7% | MIT (open core) | Yes | Native (OTLP) | Contact sales | |
| 35 | HoneyHiveTracing & observability | honeyhive | 3,245 | +38.9% | Proprietary | Enterprise only | Native (OTLP) | Contact sales | |
| 36 | LatitudeTracing & observability | latitude-telemetry | 2,379 | -18.5% | MIT | Yes | Native (OTLP) | $99/mo | |
| 37 | GalteaEvaluation | galtea | 2,373 | +23.0% | Proprietary | Enterprise only | Native (OTLP) | Contact sales | |
| 38 | OkareoEvaluation | okareo | 2,051 | +0.3% | Proprietary | Enterprise only | Native (OTLP) | $199/mo | |
| 41 | Fiddler AITracing & observability | fiddler-otel | 791 | -21.0% | Proprietary | Enterprise only | Native (OTLP) | not published in USD | |
| 42 | ScorecardEvaluation | scorecard-ai | 621 | +3.8% | Proprietary | Unknown | Native (OTLP) | $299/mo | |
| 43 | UpTrainEvaluation | uptrain | 423 | +46.4% | Apache-2.0 | Yes | Unknown | Contact sales | |
| 44 | AI Verify MoonshotbetaEvaluation | aiverify-moonshot | 370 | +65.9% | Apache-2.0 | Yes | Unknown | Free | |
| 45 | Scorable (formerly Root Signals)Evaluation | scorable | 118 | +57.3% | Proprietary | Enterprise only | Native (OTLP) | $19/seat/mo | |
| – | AgentEvalsEvaluation | – | SDK not mapped | MIT | Yes | Unknown | Free | ||
| – | Arthur EngineEvaluation | – | SDK not mapped | MIT (open core) | Yes | Native (OTLP) | $60/mo | ||
| – | Coralogix AI Observability (AI Center)Tracing & observability | – | SDK-less | Proprietary | Unknown | Native (OTLP) | not published in USD | ||
| – | Datadog LLM ObservabilityTracing & observability | – | proxy-only | Proprietary | Unknown | Native (OTLP) | $160/mo | ||
| – | Deepchecks LLM EvaluationEvaluation | – | no PyPI SDK | Proprietary | Enterprise only | Unknown | Contact sales | ||
| – | Future AGIEvaluation | – | SDK not mapped | Apache-2.0 (open core) | Yes | Native (OTLP) | $250/mo | ||
| – | LaminarTracing & observability | – | SDK not mapped | Apache-2.0 (open core) | Yes | Native (OTLP) | $30/mo | ||
| – | LangWatchTracing & observability | – | SDK not mapped | Apache-2.0 (open core) | Yes | Native (OTLP) | not published in USD | ||
| – | Maxim AIEvaluation | – | SDK not mapped | Proprietary | Enterprise only | Native (OTLP) | Contact sales | ||
| – | OpenEvalsEvaluation | – | SDK not mapped | MIT | Yes | Unknown | Free | ||
| – | OpenObserve (LLM observability)Tracing & observability | – | SDK-less | AGPL-3.0 (open core) | Yes | Native (OTLP) | not published in USD | ||
| – | Patronus AIEvaluation | – | no PyPI SDK | Proprietary | Enterprise only | Unknown | not published in USD | ||
| – | RaindropTracing & observability | – | SDK not mapped | Proprietary | Unknown | Native (OTLP) | $299/mo | ||
| – | RespanTracing & observability | – | SDK not mapped | Proprietary | Enterprise only | Native (OTLP) | $199/mo |
Other categories
- LiteLLM (Gateways with observability)
- Portkey (Gateways with observability)
- NeMo Guardrails (Guardrails & safety)
- AgentOps (Tracing & observability)
- LlamaFirewall (Guardrails & safety)
- Last9 (AI / LLM observability) (Tracing & observability)
- Cloudflare AI Gateway (Gateways with observability)
- Guardrails AI (Guardrails & safety)
- Invariant Guardrails (Guardrails & safety)
- Lakera Guard (Guardrails & safety)
- SigNoz (LLM observability) (Tracing & observability)