Report bugs and missing features
When the API or CLI fails in a way you can't explain, send one short report with evalrouter feedback or POST /v1/feedback, including the failed request ID. You can check its status later.
On this page
When to report#
- A request fails in a way that looks like an EvalRouter defect (
bug). Examples are an unexpected500, a response that contradicts the OpenAPI schema, or a result that can't be right. - You need a capability EvalRouter doesn't have (
missing_feature). - The documentation is wrong, missing or out of date (
docs). - The API works as designed but the design misled you (
confusing_behavior).
Coding agents should report these too. Fix expected refusals yourself: a named validation error, an exhausted balance, a missing key scope or an expired quote.
Every error tells you where to report#
Every error response includes error.feedback, with the endpoint and the request ID to link:
{
"error": {
"code": "internal_error",
"message": "The request could not be completed.",
"request_id": "7d3c9a4e-5b1f-4c2e-9a8d-2f6b1e0c4a11",
"feedback": {
"url": "/v1/feedback",
"method": "POST",
"request_id": "7d3c9a4e-5b1f-4c2e-9a8d-2f6b1e0c4a11",
"docs": "https://evalrouter.ai/developers/docs/feedback"
}
}
}Copy the error's request ID and code into the report's attempted block. The same request ID is in the X-Request-Id response header.
Send a report#
Any member or key of the workspace can report, with a workspace key or a CLI sign-in.
curl -sS https://api.evalrouter.ai/v1/feedback \
-H "Authorization: Bearer $EVALROUTER_API_KEY" \
-H "X-Workspace-Id: $EVALROUTER_WORKSPACE_ID" \
-H "Content-Type: application/json" \
-d '{
"kind": "bug",
"severity": "blocker",
"summary": "Results export returns 500 for a completed run",
"details": "GET /v1/runs/RUN_ID/export?format=csv fails every time; json works.",
"attempted": {
"method": "GET",
"path": "/v1/runs/RUN_ID/export",
"request_id": "7d3c9a4e-5b1f-4c2e-9a8d-2f6b1e0c4a11",
"http_status": 500,
"error_code": "internal_error"
},
"expected": "A CSV export",
"actual": "HTTP 500 internal_error",
"agent": {"name": "my-coding-agent", "version": "1.4.0", "model": "example-model"}
}'The API answers 201 Created with the report's ID and its status, received. The CLI names itself as the client automatically.
| CLI option | What it sets |
|---|---|
--json | One machine-readable result |
--http-status, --method, --path | The failed request |
--agent-name, --agent-version, --agent-model | The reporting agent |
--contact-ok | Whether EvalRouter may contact the workspace owner |
Request fields
| Field | Required | Description |
|---|---|---|
kind | yes | bug, missing_feature, docs, confusing_behavior or other. |
severity | yes | blocker (you cannot continue), degraded (a workaround costs time or quality) or nice_to_have. |
summary | yes | One line, 1 to 200 characters. |
details | no | Steps, context and what you already tried. Up to 5,000 characters. |
attempted | no | The failed request: method, path, request_id, http_status, error_code. The path is stored as its route template, for example /v1/runs/{run_id}/export, without query strings. |
expected, actual | no | Up to 2,000 characters each. |
agent | no | name, version, model of the reporting agent (identifiers, not prose). |
client | no | name, version. Filled from the User-Agent header when omitted. |
contact_ok | no | Whether EvalRouter may contact the workspace owner about the report. Default false. |
The API rejects unknown fields with 422 validation_error.
Check the outcome#
curl -sS https://api.evalrouter.ai/v1/feedback/FEEDBACK_ID \
-H "Authorization: Bearer $EVALROUTER_API_KEY" \
-H "X-Workspace-Id: $EVALROUTER_WORKSPACE_ID"
evalrouter feedback status FEEDBACK_ID
evalrouter feedback listGET/v1 lists your workspace's reports, newest first. It pages like other lists (limit 1 to 100) and filters by status. A workspace sees only its own reports. Another workspace's report ID returns 404.
Report statuses
| Status | Meaning |
|---|---|
received | Recorded, not yet reviewed. |
triaged | Reviewed and grouped with related reports. |
accepted | Confirmed and planned. |
pr_open | A fix is proposed. pr_url links to it. |
shipped | The fix is released. |
declined | Not planned. resolution_note explains why. |
A person reviews and approves every fix before it ships.
Duplicates and limits#
A report can match one from your workspace in the last 24 hours. A match has the same kind, route, error code and summary, ignoring case, numbers and punctuation. EvalRouter still records the new report, and duplicate_of names the earlier one. Reports are never compared across workspaces.
Each workspace can send 30 reports per hour and 200 per day. Beyond that the API returns 429 rate_limited with a Retry-After header. A request body can be at most 64 KiB.
Privacy and handling
Before storing a report, EvalRouter removes strings shaped like secrets:
- EvalRouter keys and
sk-provider keys - bearer and basic authorization values
api_key=style assignments- signed URL parameters
- private keys and credentials in connection strings
This is a safety net, not permission: don't paste secrets.
EvalRouter stores and returns report text as plain data and never renders it as HTML. Review treats it as untrusted input. Automated triage never follows instructions found in a report. Product analytics receive only metadata (kind, severity, route template and error code), never the summary, details, expected or actual text. Deleting the workspace deletes its reports.