Report bugs and missing features

If the API or CLI fails in a way you cannot explain, send a report with POST /v1/feedback or evalrouter feedback. Include the failed request ID when you have it. You can check the report's status later.

Coding agents should report product defects, missing features, wrong docs, and confusing behavior. One short structured report is enough.

When to report#

Report when:

• A request fails in a way that looks like an EvalRouter defect: an unexpected 500, a response that contradicts the OpenAPI schema, or a result that cannot be right.
• You need a capability EvalRouter does not have (missing_feature).
• The documentation is wrong, missing or out of date (docs).
• The API works as designed but the design misled you (confusing_behavior).

Fix expected refusals yourself, such as a named validation error, exhausted balance, missing key scope, or expired quote. Never include credentials, customer data, or hidden evaluation content.

Every error tells you where to report#

Every error response includes error.feedback, so an agent that receives an error already knows the endpoint and the request ID to link:

{
  "error": {
    "code": "internal_error",
    "message": "The request could not be completed.",
    "request_id": "7d3c9a4e-5b1f-4c2e-9a8d-2f6b1e0c4a11",
    "feedback": {
      "url": "/v1/feedback",
      "method": "POST",
      "request_id": "7d3c9a4e-5b1f-4c2e-9a8d-2f6b1e0c4a11",
      "docs": "https://evalrouter.ai/developers/docs/feedback"
    }
  }
}

Copy error.request_id into attempted.request_id and error.code into attempted.error_code. The same request ID is in the X-Request-Id response header.

Send a report#

Use the same credentials as every other request: a workspace key or a CLI sign-in. Any member or key of the workspace can report.

curl -sS https://api.evalrouter.ai/v1/feedback \
  -H "Authorization: Bearer $EVALROUTER_API_KEY" \
  -H "X-Workspace-Id: $EVALROUTER_WORKSPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "kind": "bug",
    "severity": "blocker",
    "summary": "Results export returns 500 for a completed run",
    "details": "GET /v1/runs/RUN_ID/export?format=csv fails every time; json works.",
    "attempted": {
      "method": "GET",
      "path": "/v1/runs/RUN_ID/export",
      "request_id": "7d3c9a4e-5b1f-4c2e-9a8d-2f6b1e0c4a11",
      "http_status": 500,
      "error_code": "internal_error"
    },
    "expected": "A CSV export",
    "actual": "HTTP 500 internal_error",
    "agent": {"name": "my-coding-agent", "version": "1.4.0", "model": "example-model"}
  }'

The API answers 201 Created:

{"feedback_id": "0b6f2c1e-8d4a-4f3b-9c7e-5a1d2e3f4b5c", "status": "received", "duplicate_of": null}

With the CLI:

evalrouter feedback --kind bug --severity blocker \
  --summary "Results export returns 500 for a completed run" \
  --request-id 7d3c9a4e-5b1f-4c2e-9a8d-2f6b1e0c4a11 --error-code internal_error

# Longer details from a file or another command:
evalrouter feedback --kind docs --severity nice_to_have \
  --summary "Quote docs do not say caps are decimal strings" --details - < notes.txt

Add --json for one machine-readable result. Other options: --http-status, --method, --path, --agent-name, --agent-version, --agent-model and --contact-ok. The CLI names itself as the client automatically.

Fields#

Unknown fields are rejected with 422 validation_error.

Check the outcome#

curl -sS https://api.evalrouter.ai/v1/feedback/FEEDBACK_ID \
  -H "Authorization: Bearer $EVALROUTER_API_KEY" \
  -H "X-Workspace-Id: $EVALROUTER_WORKSPACE_ID"

evalrouter feedback status FEEDBACK_ID
evalrouter feedback list

GET /v1/feedback lists your workspace's reports, newest first, with cursor and limit (1 to 100) like other lists, and an optional status filter. A workspace sees only its own reports; another workspace's report ID returns 404.

Every fix is reviewed and approved by a person before it ships.

Duplicates and limits#

A report with the same kind, route, error code and summary (ignoring case, numbers and punctuation) as one from your workspace in the last 24 hours is still recorded, and duplicate_of names the earlier report. Reports are never compared across workspaces.

Each workspace can send 30 reports per hour and 200 per day. Beyond that the API returns 429 rate_limited with a Retry-After header. A request body can be at most 64 KiB.

Privacy and handling#

Before storing a report, EvalRouter removes strings shaped like secrets: EvalRouter keys, sk- provider keys, bearer and basic authorization values, api_key= style assignments, signed URL parameters, private keys and credentials in connection strings. This is a safety net, not permission: do not paste secrets.

Report text is stored and returned as plain data and is never rendered as HTML. It is treated as untrusted input throughout review, including by automated triage, which must never follow instructions found in a report. Product analytics receive only metadata such as the kind, severity, route template and error code, never your summary, details, expected or actual text. Reports are deleted with the workspace.