Build an evaluation dataset that resists leakage and contamination

An evaluation set can look realistic and still give you the wrong answer. A conversation may appear in both development and test data. A prompt may include a field recorded only after the decision. A team may repeatedly tune against its supposedly final holdout.

Protecting the dataset means controlling how information moves through the whole evaluation process. File names and random splits are not enough. The recommendations below use a hypothetical support assistant; no private customer data or observed model results are involved.

Distinguish three routes to misleading evidence

Development leakage occurs when information from evaluation data influences the configuration you are evaluating. That can happen through prompt examples, label-derived features, repeated inspection of failures or a retrieval index that accidentally contains answers.

Scikit-learn's documentation describes leakage as using information unavailable at prediction time when building a model and recommends splitting before fitting data transformations. The same timing question is useful when assembling LLM context: would this information have existed at the moment of the request? Common pitfalls.

Contamination usually refers here to evaluation material appearing in a model's training or adaptation data. This can be difficult to investigate for a model whose full training data are unavailable. Research on pretraining-data detection explicitly studies that limited-access problem. It does not give an application team a universal certificate that its tests are unseen. Detecting Pretraining Data from Large Language Models.

A third problem is mismatch: a clean dataset may simply represent the wrong work. Leakage controls do not establish representativeness. Keep all three questions visible in the dataset record.

Document provenance before splitting

For each case, record its origin, collection date, relevant time period, group identifiers, transformations and label evidence. State whether it came from permitted operational records, expert construction, a public dataset or synthetic generation.

A dataset-level note should explain intended use, composition, collection method, known gaps and maintenance ownership. Datasheets for Datasets proposes this kind of documentation to improve transparency between creators and users. Your implementation can be compact while retaining those decisions. Datasheets paper.

For the hypothetical assistant, retain a policy version and the order-record snapshot available at the scored turn. Store eventual resolution separately as reviewer evidence. If a delivery date was entered after the assistant responded, including it as input would change the task.

Minimise identifying content and record permissions for the intended use. If you redact or substitute details, check that the transformation preserves the task's causal structure. Removing a date needed to determine eligibility can turn a straightforward case into a missing-information case.

Split according to the generalisation you need

Randomly separating rows is only appropriate when rows are a reasonable independent unit for the intended question. Multiple messages from one conversation are usually a poor basis for claiming performance on new conversations.

Group related items before assigning splits. Keep a source document and its paraphrases together. Keep variants of one scenario together. Depending on your objective, you may also need to group by customer, document template or repository. The correct boundary comes from the task specification.

A time split can help answer whether a configuration works on later work, but it introduces other questions. Did policy, product mix or labelling practice change across the boundary? Record those changes so a decline is not automatically attributed to model behaviour.

Use a development set for building prompts and scorers, a validation process for choosing among candidates, and a final holdout for the declared decision. These are roles, not mandatory percentages. Assign enough independent cases to important slices and disclose thin coverage.

Protect the holdout from repeated decisions

Restrict access to final inputs and labels where practical. Separate the people or process doing prompt optimisation from the final scoring step. Log which configurations were evaluated and who inspected the results.

Suppose a team runs a final holdout, reads every failure, rewrites the prompt and runs it again. The second result may show useful improvement on known problems. It should not be described as an untouched estimate of performance on unseen cases.

Promote exposed cases into development or regression use and reserve fresh cases for the next consequential decision. Preserve the original results and explain the transition. Do not erase the first run because the prompt was “nearly finished”.

You can still maintain a stable regression set that everyone sees. Its job is to detect recurring failures, not to provide unlimited fresh evidence about generalisation. Separating those purposes reduces pressure to make one dataset serve incompatible roles.

Audit overlap at several levels

Start with exact duplicates after documented normalisation, then inspect near-duplicates and shared templates. Compare stable source IDs as well as text. Different wording can conceal the same underlying case, while identical boilerplate can appear in genuinely different tasks.

Keep an overlap report with the rule used and the disposition of flagged cases. Do not automatically delete everything with a high similarity score. A reviewer should decide whether the shared material leaks an answer or merely reflects the normal workload.

Audit context assembly too. Search indexes, example libraries, cached tool outputs and generated summaries can carry reference material across boundaries. A clean input CSV does not guarantee a clean final request.

Black-box contamination research includes a statistical test based on the likelihood of canonical versus shuffled benchmark ordering, under an exchangeability assumption. That is a specific method with specific access requirements, not a generic inference from high accuracy. Proving Test Set Contamination in Black Box Language Models.

If you cannot establish training-data exposure, say it is unknown. Do not claim “contamination-free” merely because a web search found no identical sentence.

Treat synthetic cases as designed coverage

Synthetic cases are useful for constructing missing-information scenarios, contradictions and boundary conditions. Have a knowledgeable reviewer validate the input, expected behaviour and difficulty. A generated reference can be wrong or can make the answer unnaturally obvious.

Record the construction method and keep variants grouped. Do not count a hundred paraphrases of one scenario as a hundred independent business situations. They can measure robustness to wording, which is valuable, but that is a narrower claim.

Separate synthetic challenge results from traffic-based estimates unless you have a defensible weighting scheme. A collection generated from your own checklist reflects the checklist's assumptions. It cannot establish that the checklist captured every important production failure.

Refresh without rewriting the past

Assign immutable versions to released datasets and their labels. When correcting a reference, record the case ID, old and new interpretation, evidence and affected runs. Re-score both candidates when the comparison is affected, rather than correcting only one result.

Keep a stable core for trend analysis and a fresh sample for current workload coverage. Report both when policy or traffic changes. A growing suite needs a maintenance owner, not just another directory of examples.

Finish the dataset handoff with provenance, grouping rules, split assignments, access history, overlap findings and unresolved exposure questions. Then carry the independent grouping units into the uncertainty analysis. The benchmarking guide shows how these controls support a decision whose limits can be explained.