Measure how models think,
not just what they answer.
ReasonEval is an open suite of trace-level reasoning benchmarks — math, code, logic and long-horizon planning — graded on process quality, calibration and trust, not final answers alone.
Six suites, one question:
can the model actually reason?
Every suite grades the full reasoning trace — not just the final answer — under adversarial perturbation and calibration probes.
ReasonBench · Math
Competition and formal-verified mathematics. Answers are checked by CAS verifiers; proofs are graded step-by-step.
ReasonBench · Code
Algorithmic and agentic coding tasks scored against executable test suites in sandboxed runtimes.
ReasonBench · Logic
Formal logic, constraint satisfaction and proof completion with machine-checkable ground truth.
ReasonBench · MultiStep
Long-horizon planning tasks requiring 15+ chained tool calls, state tracking and recovery from dead ends.
ReasonBench · Calib
Epistemic calibration: separable confidence on knowns, honest uncertainty on unknowns and traps.
ReasonBench · Agent
Multi-turn agentic evals in sandboxed environments — browsing, file systems and APIs with replayable logs.
The current reasoning frontier
Composite R-score across all six suites, weighted 40/25/20/15 for correctness, process, calibration and robustness.
| # | Model | Organization | Overall R-score | Math | Code |
|---|---|---|---|---|---|
| 1 | Claude Opus 4.5 | Anthropic | 89.2 | 91.4 | 90.1 |
| 2 | GPT-5.2 | OpenAI | 88.6 | 92.0 | 88.9 |
| 3 | Gemini 3 Pro | 87.9 | 89.8 | 87.2 | |
| 5 | DeepSeek-R2Open | DeepSeek | 84.7 | 87.9 | 86.4 |
| 6 | GLM-5Open | Z.ai | 84.0 | 85.2 | 85.8 |
Showing 5 of 27 tracked models. View the full leaderboard →
Grading the reasoning, not the result
Trace-level grading
Every step of the chain-of-thought is scored against rubrics and formal verifiers — lucky guesses don't pass, sound reasoning that trips on arithmetic doesn't crash.
Adversarial perturbation
Each problem ships with rewritten variants — reordered facts, changed constants, added distractors — to detect memorization and score genuine robustness.
Calibrated trust
We reward models for saying "I don't know" when they shouldn't know. Overconfidence is penalized; honest uncertainty is scored as a feature.
Start evaluating in minutes
Point the CLI at any OpenAI-compatible endpoint and get a full trace-graded report — per-step scores, calibration curves and failure taxonomies.
Get started$ pip install reasoneval $ reasoneval run --model "my-model" --suite all ▸ math 87.9 (±1.2) 1,842 problems ▸ code 86.4 (±1.4) 1,655 problems ▸ logic 82.1 (±1.6) 1,208 problems ▸ R-score 84.7 (±0.9) ✓ report.html written