Model API reference
Discover models, quote a run, start it once and read results over HTTP. The CLI and SDKs use this API. The API schema has every field, limit and pagination rule.
On this page
The flow has five steps:
- Discover a benchmark profile and a model route.
- Create a quote.
- Start it once with an idempotency key.
- Poll the run.
- Read the results and exports.
The schema describes the release source, not every operation deployed to your workspace. Availability, permissions and feature flags still apply. Optional legacy response fields don't expand the public offering.
Authentication and workspace#
Send requests to https://api.evalrouter.ai with a workspace API key (create an account first):
Authorization: Bearer YOUR_WORKSPACE_KEY
X-Workspace-ID: YOUR_WORKSPACE_ID
Accept: application/jsonAdd Content-Type: for JSON bodies. Load credentials from a secret manager or your client's secure configuration, and keep keys out of URLs, source files, shared commands and logs.
Discover model routes and benchmark profiles#
| Method and path | Purpose |
|---|---|
GET/v1 | Discover active benchmarks. Listing alone doesn't mean a profile is runnable. |
GET/v1 | Read the benchmark and its profiles. Choose a profile with quote_availability.status equal to ready_for_quote. |
GET/v1 | Discover model routes and their availability. |
GET/v1 | Read available execution choices. |
Paginated endpoints accept cursor and limit where the schema declares them. Follow the returned cursor. A single page may not be complete. The quote remains the final compatibility and cost check.
Model connections#
| Method and path | Purpose |
|---|---|
GET/v1 | List workspace model connections. |
POST/v1 | Create a supported endpoint connection. |
PATCH/v1 | Update a connection. |
POST/v1 | Test a connection before selecting it. |
DELETE/v1 | Disable a connection. |
A check may make a small model request, which your provider may bill separately. Keep provider credentials private.
Quote a model evaluation#
Send POST/v1 with identifiers from discovery:
{
"model": {"kind": "managed", "route_id": "MODEL_ROUTE_ID"},
"selection": {"profile_ids": ["BENCHMARK_PROFILE_ID"]},
"coverage": {"mode": "sample", "sample_count": 3, "seed": 42},
"max_charge_microusd": "1000000"
}- For a checked connection,
modelis{"kind":"connection","connection_id":"CHECKED_CONNECTION_ID"}. Full coverage is{"mode":"full"}. - The cap is a decimal string in micro-USD (
1000000is $1). A quote starts no paid work and doesn't guarantee the run fits the example cap. Paid runs need available credit. - The response is a quote record (HTTP 201). Review compatibility, counts, warnings, expiry and cost components, and keep its
idand expiry.GET/v1reads it again./quotes /{quote_id}
Submit once and follow status#
Persist an operation key first, then send the reviewed quote to POST/v1 with it as Idempotency-Key:
{"quote_id":"REVIEWED_QUOTE_ID","name":"Model evaluation"}The HTTP 202 response includes the run ID.
| Method and path | Purpose |
|---|---|
GET/v1 | List workspace runs. |
GET/v1 | Read status, progress counts and accounting state. |
POST/v1 | Request cancellation explicitly. |
Poll with a bounded local timeout. Completed and error counts describe processed work. A percentage isn't a score. A local timeout or a disconnected client doesn't cancel the run, and cancellation is a request, not proof that work or accounting has stopped.
Results and exports#
| Method and path | Purpose |
|---|---|
GET/v1 | Read results. Keep the returned result version. |
GET/v1 | Download a report as JSON, CSV or HTML. Add the integer version for a repeatable export. |
Results and exports explains coverage and partial runs.
Errors and recovery#
A rejected request starts no evaluation. Check the HTTP status and the structured error body. On an authentication failure, check the key and the workspace. Otherwise, fix the request fields or unavailable choices before you submit again. Keep request IDs for support. After an uncertain submission, recover with the saved quote and operation key. Every error body includes error.feedback, which names the endpoint for reporting a suspected defect.