Six Metrics for an AI Process

Definition. A set of measures that can help a team judge the cost, quality and operating behaviour of a process using AI.

Counting automated tasks tells you how often the system runs. It does not tell you whether customers get a better result or whether people spend less time fixing mistakes. Choose the measures that answer the process owner's questions. The six below are options, not a required scorecard.

Six measures to consider

1. Cost-per-Outcome

Divide the full cost of the process by the number of completed results in the same period. Define a result carefully: a support request resolved without a repeat contact is different from a ticket marked closed. Include people, software, infrastructure, transition work and the cost of errors. See Cost-per-Outcome for the calculation. Watch a quality measure beside it; lower cost is not progress if customers need more help afterwards.

2. Human Intervention Rate (HIR)

Count the AI-assisted cases in which a person corrects, takes over or escalates the work, divided by all AI-assisted cases in the same scope. Define what counts as intervention before tracking it. A fall may mean the system is improving, or that people have stopped reviewing it. Pair the rate with an error measure and check whether the autonomy level has changed.

3. Cycle time

Measure elapsed time from a request arriving to a usable outcome. Include waiting, hand-offs and rework, not just the seconds spent generating an answer. If the AI step becomes faster but total cycle time stays flat, look for a bottleneck elsewhere in the process. Check completeness or repeat-contact rate so work is not closed quickly at the expense of finishing it.

4. Feedback coverage

For cases where an outcome can be observed, measure how many have a recorded outcome and how many reviewed corrections become a test, rule or approved example. A large log is not the same as a learning process. This measure shows whether the team has evidence it can use for the next change. See the Data Flywheel for how that feedback can become an improvement.

5. Automated share

Divide the number of decisions the system is permitted to complete without individual approval by all decisions of the same type. Keep the scope consistent: including difficult cases in one period and excluding them in another makes the trend meaningless. A higher share is useful only if error rates, customer outcomes and escalation handling remain acceptable. It is a description of the workflow, not a target by itself.

6. Revenue per Employee

Company revenue divided by headcount can be useful context for leaders reviewing many initiatives. It is too broad to prove that one AI process worked. Pricing, sales, hiring and market conditions also move it. Show revenue and headcount separately, and use process-level measures to judge the change the team actually made.

Build a dashboard people will use

Start with the outcome the process exists to deliver and the failure the team most wants to avoid. Choose a small set of measures that covers both. For a support process, that might be cost per resolved request, repeat-contact rate, cycle time and the rate of cases people must correct. For another process, a different combination may be better.

Write down each measure's definition, data source, reporting period and owner. Establish a baseline before changing the workflow where possible. Segment the results when routine and unusual cases behave differently; an aggregate can conceal a worsening experience for one group of customers.

The process owner should review the dashboard often enough to respond while a problem is still actionable. An engineer or analyst may collect the data, but someone who owns the work must decide what a change means. When a number moves, inspect actual cases before changing the model or the rules. Data can tell the team where to look; it rarely explains the cause on its own.