Choose metrics and rubrics that reward the right AI behaviour

A metric is a compressed statement of what you value. The benchmark pillar places scoring in the wider model-selection workflow. If it rewards the wrong shortcut, a model can improve its score while making the product worse. Before selecting a number, define what a successful outcome looks like and what errors matter to the user.

This guide helps practitioners match scoring methods to tasks. It connects to the cluster’s task-design guide and human-review guide.

Start with observable outcomes

For structured extraction, compare fields to labelled values and decide how to treat normalisation. “$1,200.00” and “1200 AUD” may be equivalent if currency is explicit; “1200” without a currency may not be. Report field-level and record-level results because a record with one wrong critical field may count as a failure even when three fields are correct.

For classification, use a confusion matrix and per-class measures, not only overall accuracy. If 95 per cent of messages are routine, a system that always predicts routine can look impressive on accuracy while missing every urgent report. Precision answers how often predicted positives are correct; recall answers how many actual positives are found. Choose based on the operational cost of false alarms versus misses, and show both when the trade-off matters.

For generated text, exact match is often too strict and semantic similarity can be too permissive. A reference answer may be one of many acceptable phrasings. Define criteria such as factual correctness, completeness, grounding, relevance, instruction following and readability, then give examples of passing, borderline and failing outputs. If a criterion is a hard requirement, make it a gate rather than hiding it inside a weighted average.

Separate dimensions before combining them

A scorecard should expose its parts: task success, format validity, groundedness, safety, latency, cost and escalation behaviour may each matter. A composite can be useful for sorting candidates, but it makes trade-offs harder to see. If you create one, publish the formula, units, weights and threshold. Run a sensitivity check: would small reasonable changes in weights reverse the ordering?

Prefer a hierarchy when risks are asymmetric. First require non-negotiable criteria, such as correct handling of a prohibited request or an acceptable critical-error rate. Then compare candidates that pass on softer preferences such as brevity or cost. This prevents a low cost score from numerically compensating for a dangerous failure.

Avoid metrics that are easy to optimise but weakly related to user value. Token overlap, response length and generic sentiment scores can be diagnostic, but they rarely capture whether a support answer solved the user’s problem. A public evaluation guide similarly warns against relying on generic academic metrics alone and recommends task-specific evaluation tied to intended behaviour (OpenAI evaluation best practices). Treat this as design guidance rather than proof that any particular metric is valid for your application.

For the standard precision and recall definitions, see Manning, Raghavan and Schütze’s evaluation chapter. Map the positive class to your task before using these retrieval or classification measures.

Write rubrics like executable specifications

A usable rubric has a named criterion, a scale, observable anchors and examples. “Helpful” is not enough. For a policy answer, one dimension could be “groundedness”: 0 means at least one material claim is unsupported or contradicted by the supplied policy; 1 means every material claim is supported, but at least one relevant minor claim is unsupported; 2 means all relevant claims, material and minor, are supported, with no contradiction. An otherwise correct refund answer that invents a non-material processing detail scores 1, not 2. If that detail changes a customer’s eligibility or promised payment date, treat it as material and score 0. Assess missing required information separately under completeness. Reviewers should be able to point to evidence in the answer and the source.

Keep dimensions as independent as possible. If “accuracy” includes completeness and tone, reviewers cannot tell why a score changed. If two criteria overlap, write a rule for double-counting. Define how to handle missing, malformed, refused or truncated outputs. A blank response should not quietly disappear from the denominator.

For ordinal scales, do not imply that the gap from 1 to 2 equals the gap from 4 to 5 unless that assumption is justified. A 1–5 rubric can help structure judgement, but its mean may hide whether outputs are mostly acceptable or polarising. Include score distributions and pass rates at meaningful thresholds. Use written rationale on a sample so the numeric ratings remain interpretable.

Validate automated metrics

For deterministic checks, test the checker itself. A JSON validator should reject missing fields and wrong types, but permit only the variations your product accepts. Build positive and negative fixtures, including whitespace, Unicode, empty values and malformed output. When comparing answers to references, normalise only what should be irrelevant; aggressive normalisation can erase meaningful errors.

For semantic similarity or model-based graders, create a calibration set with clear examples and human labels. Examine false positives and false negatives. Test adversarial cases that meet superficial wording requirements while failing the task. OpenAI’s grader guidance describes exact-string, similarity, code and model-scoring approaches, and recommends testing a model grader against examples with known quality ordering (Graders guide). The underlying principle is general: a grader is another model or program whose behaviour needs evaluation.

If a score is used as a release gate, test whether the gate can be gamed. A summary graded for key-term inclusion may stuff terms without conveying meaning. A model trained to maximise rubric points may produce verbose answers that mention every criterion. Inspect outputs and keep a human audit path for high-impact decisions.

Match the unit of analysis

State what one observation is. It might be a user request, a document, a conversation, a tool-use episode or a whole task trajectory. If each conversation produces many scored turns, treating every turn as independent can understate uncertainty. Group or aggregate at the level that matches deployment and report the number of independent units.

Use both macro and weighted summaries when class balance matters. A macro average gives each class equal weight; a prevalence-weighted measure reflects the observed distribution. Neither is universally correct. The former can surface poor minority-class performance; the latter can estimate average traffic outcomes if the sample really reflects traffic. Report slice counts so readers can judge stability.

For selective systems, measure coverage and quality together. A model that abstains on every difficult request may have excellent accuracy among answered cases but provide little value. Report answer rate, escalation rate and error rate conditional on answering. Make sure abstention labels are appropriate, not just a way to hide mistakes.

Example scorecard for invoice extraction

The following is hypothetical. For each invoice, the system returns supplier, date, currency, total and a confidence flag. A useful scorecard might include:

• exact-match rate by field after documented normalisation;
• proportion of invoices with every critical field correct;
• rate of unsupported values when the source is unreadable;
• rate of correct abstention or human escalation;
• malformed structured-output rate;
• p50 and p95 response latency;
• cost per successful invoice, including retries.

The team could set a non-inferiority gate for critical-field correctness relative to its current system, then choose the least costly candidate that passes. The margin and thresholds must come from business risk, not be copied from this example. A specialist might review all high-risk misses and a random sample of apparent successes to see whether automated checks miss subtle errors.

Report uncertainty and failure cost

A percentage without a denominator is difficult to interpret. “96 per cent accuracy” could mean 24 of 25 or 9,600 of 10,000. Include counts and uncertainty intervals where appropriate. Break down critical failures by type and show examples in a privacy-safe way. The repeatability guide explains why intervals and repeated calls answer different questions.

Not all errors carry equal cost. A simple weighted error score can help prioritise, but the weights are assumptions. Show raw error counts beside the weighted total and perform a sensitivity check. For regulated or safety-relevant workflows, a threshold based on the worst slice or a required human review may be more defensible than expected average loss alone.

A short metric review

Before running candidates, ask: Does every metric map to a user outcome? Are failure costs explicit? Is the denominator correct? Can the score be gamed? Are subjective criteria anchored with examples? Do the metrics cover format, safety and operational constraints where needed? Can a reviewer trace a score back to the output?

Metrics do not eliminate judgement; they make some judgement explicit and repeatable. Use a small scorecard that you can explain, show its components and revise it when the task changes. When the desired quality is contextual, combine automation with calibrated human evaluation, and avoid turning a diagnostic number into a claim about overall model intelligence.