Evals before the system

Definition. Evals are repeatable checks of how an AI system handles the work it is meant to do. “Evals before the system” means agreeing on those checks before tuning the system around them.

Without a shared definition of good work, a team can argue over each new model or prompt without learning much. A useful eval set makes the disagreement concrete: here is the request, here is the information available at the time, and here is what an acceptable response must do.

Build a first set of cases

Choose a narrow decision type with a process owner. Gather real examples from the current workflow, with appropriate handling of personal or confidential information. Include common requests, expensive mistakes, ambiguous situations and cases that should be escalated. The set should reflect the decisions the system will actually face, not just the ones that are easy to grade.

For each case, record the input, relevant context, expected behaviour and reason it matters. Some cases have one correct answer. Others have several acceptable answers and a list of failures to avoid. A domain expert should review the criteria; an engineer can make the run repeatable.

Run the current process or an early prototype against the cases to establish a baseline. The baseline is the result you observe, not an assumed zero. Keep part of the set stable for comparison across versions. Keep separate cases for new problems discovered in live work, and review them before adding them to the set. Otherwise one mistaken human correction can become a permanent “correct” answer.

Judge the result in context

Report more than a single pass rate. Separate routine cases from exceptions, and distinguish harmless differences in wording from errors that change a decision. For a customer-facing response, check factual accuracy, whether the system respected a boundary, and whether the customer could use the answer. For a routing task, inspect wrong destinations and missed escalations.

Some work has no single reference answer. Writing, research and other open-ended tasks can still be evaluated: use a clear rubric, samples reviewed by qualified people and checks for factual or policy failures. The uncertainty should be visible rather than hidden inside a precise score.

Pair the evals with measures from live work. A higher score on a fixed set is encouraging, but it may reflect tuning to those cases. Watch actual corrections, outcomes and the cost of the process as well. If the test score improves while live performance does not, examine whether the set still resembles current work. The six metrics article offers options for that operational view.

Keep the set useful

Name someone who can approve changes to cases and grading criteria. Keep versions so the team can tell whether a score changed because the system improved or because the test changed. After a release or a process change, review failures and add cases that represent new work. Retire cases only with a recorded reason.

Evals help the process owner decide what to improve and what to test next. They do not replace human approval, execution controls or the separate decision about autonomy. A system can pass its tests and still be wrong on a case the team has never seen.