Import a workload

Connect a Braintrust, Langfuse, or LangSmith project from trace connections. A workspace admin supplies the provider credentials and selects the project and region. EvalRouter encrypts those credentials and syncs traces in the background. The connection shows progress, errors, and controls to pause, resume, import history, or update credentials.

For Braintrust and Langfuse datasets or experiments, open a dataset and choose Import workload data. Select the connected project and resource, declare usage rights, and assign a split. Import subsequent pages until the import completes. Source identifiers and content hashes preserve provenance across retries.

Historical model outputs and provider scores remain source evidence. They do not become correct answers or pass labels. Imported inputs seed a task draft for review. Review the instruction, inputs, tool behavior, reference answer, and grading rule before approving a task. Freeze an approved dataset version before using it in Tune.

Use development tasks to choose configurations. Reserve separate held-out tasks for the final comparison. A copied task or a paraphrase of it should stay in the same split as its source.

You can use reviewed deterministic rules in individual tasks or create a versioned grader in the grader workspace. Model-assisted drafting uses an explicit spending cap. Review its proposal, label calibration examples, and save the calibration against the exact dataset version. Calibration excludes held-out examples and never uses historical scores as human labels.

Provider connections and Tune depend on deployment availability. See availability. Uploaded executable scorer code is not run by these imports.

You can also save a custom HTTPS scorer or pinned Braintrust function in Evaluators. Set its immutable version, dedicated credential, declared charge, and maximum calls per job. Then calibrate the resulting evaluator on reviewed labels. Imported historical scores alone do not make a scorer executable.