Results and exports
Check that a run finished and how much it covered, then read its scores or export a versioned report.
Read a result#
evalrouter runs get RUN_ID --json
evalrouter runs results RUN_ID --jsonA result records the benchmark version, completed and missing work, scores, errors and billing state. A sample is not a full benchmark result: mark partial and failed runs as such in comparisons. For a run still in progress, use evalrouter .
In a terminal, the results command lists failed tasks under Why tasks failed, using each sample's error code from GET/v1. A task without a code has no class. The terminal shows plain metric names (Accuracy for acc). JSON and exports keep the original keys and error codes.
If a benchmark offers output formats, the run's selection and each sample record the chosen one. Compare only matching formats. Otherwise a comparison returns output_format_differs with no delta, and leaderboard protocol hashes differ.
Compare runs in the browser#
Open Evaluations › Group runs. A group holds models started together or runs grouped later. Use Runs for its members and Comparison to read their results together.
- Create group from existing runs takes two to four finished results: one baseline and up to three candidates. It saves frozen result references (version, checksum and model index) and never reruns a model. Pass/fail checks are optional. Without them the comparison makes no pass/fail decision. Incompatible pairs explain their differences and show no direct score deltas or release conclusions.
- New evaluation › Run a benchmark starts one to eight managed models. Two to eight start a group on the same selection, whose Comparison tab reads each member's latest result without running inference or saving another comparison. It says when results are pending or unavailable. Comparing a resumed group isn't available in the browser yet.
Save a versioned report#
evalrouter runs export RUN_ID --version RESULT_VERSION --file-format json --output result.json
evalrouter runs export RUN_ID --version RESULT_VERSION --file-format csv --output result.csv
evalrouter runs export RUN_ID --version RESULT_VERSION --file-format html --output result.htmlUse the integer version the result returns. Exports need a destination and refuse to overwrite a file by default. Keep the version and checksum with any downstream report. Workspace permissions and the evaluation's rights govern what you can export. If a download fails, retry the same run and version.
Metrics in HTML reports
HTML single-run bundles and CLI comparisons show All benchmark metrics, with each recorded value and its available uncertainty. Hover a metric name for its definition and exact key, or expand the glossary. Weighted variants stay separate. SOB source scores appear only when the export includes them. Source coverage alone is not a score. Missing values are never inferred from pooled averages. An upgrade of the CLI doesn't change an existing archive. Generate a new report instead. When the saved data lacks the source metrics, request a new API export.