Model API reference
Discover models, quote a run, start it once and read results over HTTP. The CLI and SDKs use this API. The API schema has every field, limit and pagination rule.
On this page
The flow has five steps:
- Discover a benchmark profile and a model route.
- Create a quote.
- Start it once with an idempotency key.
- Poll the run.
- Read the results and exports.
The schema describes the release source, not every operation deployed to your workspace. Availability, permissions and feature flags still apply. Optional legacy response fields don't expand the public offering.
Authentication and workspace#
Send requests to https://api.evalrouter.ai with a workspace API key (create an account first):
Authorization: Bearer YOUR_WORKSPACE_KEY
X-Workspace-ID: YOUR_WORKSPACE_ID
Accept: application/jsonAdd Content-Type: for JSON bodies. Load credentials from a secret manager or your client's secure configuration, and keep keys out of URLs, source files, shared commands and logs.
Discover model routes and benchmark profiles#
| Method and path | Purpose |
|---|---|
GET/v1 | Discover active benchmarks. Listing alone doesn't mean a profile is runnable. |
GET/v1 | Read the benchmark and its profiles. Choose a profile with quote_availability.status equal to ready_for_quote. |
GET/v1 | Discover model routes and their availability. |
GET/v1 | Read available execution choices. |
Paginated endpoints accept cursor and limit where the schema declares them. Follow the returned cursor. A single page may not be complete. The quote remains the final compatibility and cost check.
Model connections#
| Method and path | Purpose |
|---|---|
GET/v1 | List workspace model connections. |
POST/v1 | Create a supported endpoint connection. |
PATCH/v1 | Update a connection. |
POST/v1 | Test a connection before selecting it. |
DELETE/v1 | Disable a connection. |
A check may make a small model request, which your provider may bill separately. Keep provider credentials private.
Quote a model evaluation#
Send POST/v1 with identifiers from discovery:
{
"model": {"kind": "managed", "route_id": "MODEL_ROUTE_ID"},
"selection": {"profile_ids": ["BENCHMARK_PROFILE_ID"]},
"coverage": {"mode": "sample", "sample_count": 3, "seed": 42},
"max_charge_microusd": "1000000"
}- For a checked connection,
modelis{"kind":"connection","connection_id":"CHECKED_CONNECTION_ID"}. Full coverage is{"mode":"full"}. - The cap is a decimal string in micro-USD (
1000000is $1). A quote starts no paid work and doesn't guarantee the run fits the example cap. Paid runs need available credit. - The response is a quote record (HTTP 201). Review compatibility, counts, warnings, expiry and cost components, and keep its
idand expiry.GET/v1reads it again./quotes /{quote_id}
Submit once and follow status#
Send the reviewed quote to POST/v1:
{"quote_id":"REVIEWED_QUOTE_ID","name":"Model evaluation"}A quote starts at most one run. After a lost response, send the same quote again: you get the same run back, not a second charge. A new quote can start separate, separately charged work.
The response is the run record with its ID: HTTP 201 when the run is created, HTTP 200 when the quote already started it.
| Method and path | Purpose |
|---|---|
GET/v1 | List workspace runs. |
GET/v1 | Read status, progress counts and accounting state. |
POST/v1 | Request cancellation explicitly. |
Poll with a bounded local timeout. Completed and error counts describe processed work. A percentage isn't a score. A local timeout or a disconnected client doesn't cancel the run, and cancellation is a request, not proof that work or accounting has stopped.
Results and exports#
| Method and path | Purpose |
|---|---|
GET/v1 | Read results. Keep the returned result version. |
GET/v1 | Download a report as JSON, CSV or HTML. Add the integer version for a repeatable export. |
Results and exports explains coverage and partial runs.
Errors and recovery#
A rejected request starts no evaluation. Check the HTTP status and the structured error body. On an authentication failure, check the key and the workspace. Otherwise, fix the request fields or unavailable choices before you submit again. Keep request IDs for support. After an uncertain submission, send the same quote again. Every error body includes error.feedback, which names the endpoint for reporting a suspected defect.