Contamination, Leakage and Holdouts: Why Your Benchmark Score May Be Inflated

Suppose a model scores 91% on your internal evaluation, up from 84%. The improvement is large, the demo looks good, and the team is ready to ship. Then a customer sends a document format that was not in the test set, and the new model fails in a way the old one did not.

One explanation is ordinary distribution shift. Another is that the evaluation was never as clean as it looked. The test items may have appeared in training data, been paraphrased into it, leaked through a prompt, or been used so often for tuning that they stopped being a test.

Contamination is not a single bug but a family of ways the measurement becomes entangled with the thing it measures. This article separates the mechanisms, summarises what the evidence shows, and sets out controls that survive realistic adversaries — including your own future optimisation.

Four different problems

Training-data contamination. Evaluation items, or close variants, appear in the data used to train or fine-tune a model. The model may memorise the answer or learn a shortcut tied to the item's surface form.

Prompt or answer leakage. The expected answer is present in the context, the file name, the system prompt or a retrieved document. The model appears competent because it can copy.

Holdout reuse. The same test set is used repeatedly for prompt selection, threshold tuning or model selection. It becomes a development set in practice, even if it is still called a holdout.

System-level benchmark overfitting. The whole system — prompts, tools, retries, scaffold, router — is optimised against the benchmark. The score improves without a corresponding improvement on new tasks. This form affects teams that are not training a model at all, and agent benchmarks in particular often lack adequate holdouts (Kapoor et al., 2024).

They have different symptoms and controls; no single decontamination step addresses all four.

Why contamination inflates scores

Benchmarks are supposed to estimate performance on unseen tasks drawn from a population. Contamination breaks the "unseen" part. If the model has seen the item, the score measures recognition as well as capability. If the model has seen many paraphrases, it may learn the label distribution rather than the task.

The effect can be large. Yang and colleagues showed that string-matching decontamination is insufficient: paraphrasing or translating test data can bypass n-gram filters. They found 8–18% HumanEval overlap in the RedPajama-Data-1T and StarCoder-Data pretraining sets, and showed that a 13B model could overfit a benchmark using such variants and reach performance on par with much larger models (Yang et al., 2023).

Deng and colleagues proposed Testset Slot Guessing, which masks a wrong answer in a multiple-choice question and asks the model to fill the gap. On MMLU, ChatGPT and GPT-4 achieved exact-match rates of 52% and 57% respectively when guessing the missing option (Deng et al., 2024). That is not proof that either model was trained on MMLU, but it is evidence that a high MMLU score cannot be interpreted as pure reasoning on unseen items.

A survey of the field categorises detection into white-box, grey-box and black-box methods and notes that contamination-free evaluation often relies on data updating, data rewriting or prevention (Cheng et al., 2025). No single test proves a benchmark is clean.

Temporal and dynamic benchmarks

One response is to make the test newer than the model. LiveCodeBench continuously collects new coding problems, enabling evaluation on a selected post-cutoff window where the model’s training cutoff is known. Such a window reduces, but does not eliminate, contamination risk. The later v2 paper snapshot reports 400 problems through May 2024; the v1 snapshot ends in February 2024, and its analysis included evidence of contamination effects on older code benchmarks (Jain et al., v1; v2).

Date-stamped tasks are powerful because they create a natural barrier: if the model could not have seen the item, memorisation is less plausible. The trade-off is comparability. A fresh test cannot be compared directly with last year's score on a different fresh test. You need an anchor set that stays fixed, plus a rolling fresh set, and you need to report which is which.

System-level overfitting

Even with a clean dataset, a team can overfit the system to the test. Common patterns:

• running dozens of prompt variants and choosing the one with the best test score;
• tuning retry counts or timeouts until failures disappear on the test;
• adding tool descriptions that mention the test's specific entities;
• selecting a model based on the test result and then reporting that result as if it were an independent confirmation;
• using an LLM judge that has been optimised against the same outputs.

This is why the holdout must be sealed and why the decision run should happen once. If you would not accept a pharmaceutical trial that kept peeking at the unblinded data, do not accept an evaluation that keeps tuning against the holdout.

The Leaderboard Illusion analysis shows how selection effects distort a ranking: private testing, selective disclosure and unequal sampling change what the leaderboard appears to show (Singh et al., 2025). The lesson for an internal evaluation is the same: access and iteration rules are part of the measurement.

Controls that actually help

A layered defence is more realistic than a single clean dataset.

Seal the holdout. Store it separately, restrict access, log every read, and require a written decision question before a run. Do not include it in the repository that prompt engineers use daily.

Collect fresh, date-stamped tasks. Add new cases on a schedule. Keep an anchor set for comparability and a fresh set for validity.

Use several decontamination methods. Exact and n-gram matching, embedding similarity, and LLM-based semantic comparison. Each catches different paraphrases. Report which methods you used and what they found.

Perturb the test. Change names, numbers, formats and irrelevant details. A model that is robust to perturbation is more likely to be using the task structure. A model that collapses when a name changes may have memorised the item.

Hold back answers. Do not put the expected output in the prompt, the file name or the metadata the model can see. Check retrieved context for answer leakage.

Rotate and retire. After a set has been used for a decision, move it to the regression suite and add fresh cases for the next decision. A seen test is still useful for regression, but weaker evidence for generalisation.

Audit the system, not just the data. Log prompt versions, tool versions and retry policies alongside scores. If a score jumps after a scaffold change, check whether the scaffold is exploiting the grader.

What to do when you suspect contamination

Start with a fresh set of tasks collected after the model's training cut-off. A sharp drop warrants investigation, but does not identify its cause. Check task difficulty, scoring and configuration differences alongside contamination and distribution shift. Then:

1. Compare performance by task age. A steep decline for newer items is a signal.
2. Paraphrase the test items and re-run. A large drop suggests surface memorisation.
3. Inspect reasoning traces for verbatim recall of test-specific details.
4. Check whether failures cluster on items that are publicly available online.
5. Ask the provider what decontamination it performs. If the answer is unclear, record exposure and filtering as unknown, then test relevant fresh cases. Unknown filtering does not establish an upper bound on deployment performance.

None of these tests proves contamination, but together they can justify a cautious conclusion.

Limits

You cannot prove a benchmark is uncontaminated, especially for a closed model. You cannot inspect the training data, and providers may not disclose their filtering. A private holdout reduces the risk; it does not eliminate the possibility that the provider trained on data that resembles your holdout.

State the limitation in the report. "We used a sealed 200-task holdout collected after the model's published cut-off; we cannot rule out contamination in the provider's training data" is honest and useful. "The model scores 91%, therefore it generalises" is not.

A holdout policy template

A short policy is better than a vague intention:

• Owner: one named person accountable for access and rotation.
• Access: read-only for the evaluation runner; no bulk export; every access logged.
• Use: one predeclared decision cycle, with all candidates and repeated trials specified before the sealed run; no prompt tuning against the set.
• Rotation: replace a third of the set every quarter, or after any decision that used it.
• Retirement: after the planned decision cycle, move the set to regression and mark it as seen; do not use it to tune and then claim a fresh selection result.
• Breach: if the set is exposed, retire it and record the exposure in the next report.

The point

Contamination is not a reason to abandon benchmarks. It is a reason to stop treating a score as a clean measurement by default. Assume some entanglement, design controls that make it harder, and describe the residual risk. For context, see the pillar guide, public benchmarks and designing your task set; maintenance covers rotation. The next article, on metrics and rubrics, assumes you have a task set worth scoring; this one is about keeping it worth scoring.

References

• Cheng, Y., Chang, Y. and Wu, Y. (2025). A Survey on Data Contamination for Large Language Models. Read source
• Deng, C., Zhao, Y., Tang, X., Gerstein, M. and Cohan, A. (2024). Investigating Data Contamination in Modern Benchmarks for Large Language Models. NAACL 2024. Read source
• Jain, N., Han, K., Gu, A., Li, W.-D., Yan, F., Zhang, T., Wang, S., Solar-Lezama, A., Sen, K. and Stoica, I. (2024). LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code. Read source
• Kapoor, S., Stroebl, B., Siegel, Z. S., Nadgir, N. and Narayanan, A. (2024). AI Agents That Matter. Read source
• Singh, S., Nan, Y., Wang, A., D'Souza, D., Kapoor, S., Üstün, A., Koyejo, S., Deng, Y., Longpre, S., Smith, N. A., Ermis, B., Fadaee, M. and Hooker, S. (2025). The Leaderboard Illusion. Read source
• Yang, S., Chiang, W.-L., Zheng, L., Gonzalez, J. E. and Stoica, I. (2023). Rethinking Benchmark and Contamination for Language Models with Rephrased Samples. Read source