Model API reference
The CLI uses the model API. You can also integrate through HTTP using the supported model-evaluation subset documented here. This is not the complete service contract. The OpenAPI download describes these operations, request bounds, response fields and pagination. It is projected from the existing Production contract; optional legacy response fields do not expand the public offering.
Authentication and workspace
Create an account and a workspace API key. Send requests to https://api.evalrouter.ai with these headers:
Authorization: Bearer YOUR_WORKSPACE_KEY
X-Workspace-ID: YOUR_WORKSPACE_ID
Accept: application/jsonFor JSON bodies, also send Content-Type: application/json. Inject credentials through your secret manager or HTTP client's secure configuration. Never put real keys into URLs, source files, shared commands or logs. Workspace permissions and available credit still apply. Browser signup and account management are separate from API-key authentication.
Discover model routes and benchmark profiles
| Method and path | Purpose |
|---|---|
GET /v1/catalog/benchmarks?status=active | Discover active benchmarks. Listing alone does not mean a profile is runnable. |
GET /v1/catalog/benchmarks/{slug} | Read the benchmark and its profiles. Choose a profile with quote_availability.status equal to ready_for_quote. |
GET /v1/catalog/models | Discover model routes and their availability. |
GET /v1/catalog/execution-options | Read available execution choices. |
Paginated endpoints accept cursor and limit where declared in the schema. Follow the returned cursor; do not assume a single page is complete. A quote remains the final compatibility and cost check for your exact selection.
Model connections
| Method and path | Purpose |
|---|---|
GET /v1/connections | List workspace model connections. |
POST /v1/connections | Create a supported endpoint connection. |
PATCH /v1/connections/{connection_id} | Update a connection. |
POST /v1/connections/{connection_id}/check | Test a connection before selecting it. |
DELETE /v1/connections/{connection_id} | Disable a connection. |
Use the exact request fields in the schema. Keep provider credentials private. A connection check may make a small model request; your provider may bill model usage separately.
Quote a model evaluation
Send POST /v1/quotes with exact identifiers returned by discovery:
{
"model": {"kind": "managed", "route_id": "MODEL_ROUTE_ID"},
"selection": {"profile_ids": ["BENCHMARK_PROFILE_ID"]},
"coverage": {"mode": "sample", "sample_count": 3, "seed": 42},
"max_charge_microusd": "1000000"
}For a checked connection, replace model with {"kind":"connection","connection_id":"CHECKED_CONNECTION_ID"}. Full coverage uses {"mode":"full"}. Review compatibility, counts, warnings, expiry and cost components before submitting. The cap is expressed as a decimal string in micro-USD: 1000000 is $1. A quote starts no paid work and does not guarantee that an evaluation fits this example cap. Paid runs require available credit.
The response is a quote record (HTTP 201). Preserve its id and expiry. GET /v1/quotes/{quote_id} retrieves a saved quote.
Submit once and follow status
For POST /v1/runs, persist an operation key first and send it as Idempotency-Key. Submit the reviewed quote:
{"quote_id":"REVIEWED_QUOTE_ID","name":"Model evaluation"}The HTTP 202 response includes the run ID. Reuse the same quote and same operation key after a lost submission response. A new operation key can start separate work.
| Method and path | Purpose |
|---|---|
GET /v1/runs | List workspace runs. |
GET /v1/runs/{run_id} | Read status, progress counts and accounting state. |
POST /v1/runs/{run_id}/cancel | Request cancellation explicitly. |
Poll status with a bounded local timeout. Completed and error counts describe processed work; a percentage is not a score. A local timeout or disconnected client does not cancel the server run. Cancellation is a request, not proof that all work or accounting has already stopped.
Results and exports
Read GET /v1/runs/{run_id}/results. Preserve the returned result version. Read GET /v1/runs/{run_id}/export?format=json to download a report; csv and html are also supported. Use the integer version query parameter for a repeatable export of an existing version.
Inspect terminal status, coverage, errors and billing alongside scores. Partial, failed and cancelled work remains distinct from a complete result; a sample is not a full-benchmark score. See results and exports.
Errors and recovery
A rejected request starts no successful evaluation. Check the HTTP status and structured error body: verify the key and workspace for authentication failures, and correct request fields or unavailable choices before submitting again. Preserve request IDs for support. After an uncertain run submission, recover with the saved quote and operation key instead of creating another operation. Follow the schema for validation responses and the returned error code for runtime failures.