Pydantic Evals
Code-first Python framework for defining datasets, cases and evaluators to test LLM calls and agents, from the Pydantic AI project.
By Pydantic Services Inc. · pydantic.dev/docs/ai/evals/evals
Package download activity
Each package is shown on its own; packages that can be installed together are never added up. Week 2026-W40 (2026-09-28 to 2026-10-04).
PyPIpydantic-evals · headline (ranked)
2,121,173 downloads in the week, +10.5% vs prior week · rank 6 on PyPI
bundledBundled by pydantic-ai (via pydantic-ai-slim[evals]): installed automatically as a required dependency, so these downloads include installs of pydantic-ai. source
Reproduce this number (ClickHouse SQL) · registry metadata (mapping checked 2026-10-07)
GitHub stars
No public repository is linked from Pydantic Evals's own site. This is not a zero.
Facts
| Categories | Evaluation | |
|---|---|---|
| Status | Generally available | |
| Licence | MIT | source, checked 2026-10-07 |
| Self-host | Yes | source, checked 2026-10-07 |
| OpenTelemetry | Native (OTLP) | source, checked 2026-10-07 |
| Pricing model | free | |
| Entry price | Free; Open-source library; results can optionally be viewed in Pydantic Logfire (separate product, pricing not checked here) | source, checked 2026-10-07 |
| Free tier | Yes | |
| Measurability | Measured (mapped SDK packages) |
Facts last verified 2026-10-07 on first-party pages. Prices as published, taxes excluded; non-USD prices are not converted.
Compare
Spotted an outdated fact? The methodology page explains how facts are checked; corrections are applied at the next weekly build.