# Buying evaluation datasets: what makes a good AI eval set

URL: https://sellmydatatoai.net/guides/evaluation-datasets-for-ai
Reviewed: 2026-11-08

A good AI evaluation set is private, expert-checked and uncontaminated: its tasks have never appeared in any model's training data, each answer has been verified by a second expert and the set is refreshed or retired before it leaks. Buy it with exclusivity, a ban on reuse by the seller and a remedy if tasks surface in public, because a leaked eval set measures memory, not ability.

## Key takeaways

- Contamination is the main risk: Zhang and colleagues found accuracy drops of up to 8% when leading models met fresh problems mirroring the public GSM8k benchmark.
- Exact-match checks miss paraphrases, and Yang and colleagues found 8% to 18% of HumanEval overlapping two public pre-training sets.
- Refresh or retire sets on a schedule, as LiveBench does with monthly question updates.
- Public benchmarks such as Humanity's Last Exam keep a private held-out split to detect overfitting, which is the model to copy.
- License eval sets exclusively, for evaluation only, with leak notification and replacement terms.

An evaluation set is the one dataset you buy hoping no model ever learns from it. Its value is the gap between what a model has memorized and what it can actually do, and that gap closes the moment the tasks leak.

This guide is for teams buying private evals for model development, vendor selection or regression testing. Our [evaluation datasets](https://sellmydatatoai.net/buy-ai-training-data/evaluation-datasets) page covers formats and delivery, so this guide focuses on contamination, exclusivity, refresh cycles and private benchmarks.

## What makes a good AI evaluation set?

A good AI evaluation set has tasks that match your deployment, answers checked by experts, scoring rules you can reproduce and a documented history showing nobody else has trained on it. Difficulty matters too: a set every model passes tells you nothing.

Public benchmarks show the effort involved. OpenAI's SWE-bench Verified, published in August 2024, used 93 software developers experienced in Python to screen tasks and kept 500 samples verified as non-problematic. Humanity's Last Exam accepted only questions that made frontier models fail, then refined each one with between one and three expert peer reviews.

| Element | What good looks like |
| --- | --- |
| Task | Self-contained, with everything needed to answer it |
| Reference answer | Verified by a second expert, or a rubric where answers vary |
| Scoring rule | Automatic check or written rubric that two graders apply consistently |
| Metadata | Stable ID, author, creation date, difficulty and domain tags |
| Provenance | Who wrote it, from what source and who has seen it since |

## Why does contamination ruin an evaluation set?

Contamination ruins an evaluation set because a model that has seen the tasks during training scores well by recall. The score then measures exposure, and you can no longer compare models or track progress.

The effect is measurable. Hugh Zhang and colleagues built GSM1k in 2024 to mirror the public GSM8k math benchmark and observed accuracy drops of up to 8% for leading models, with several model families showing evidence of systematic overfitting.

Contamination also hides well. Shuo Yang and colleagues showed in 2023 that paraphrased or translated test items pass n-gram decontamination, that a 13 billion parameter model trained on such variants could reach scores on par with GPT-4, and that synthetic datasets generated by GPT-3.5 and GPT-4 also contained benchmark overlap.

## How do you check a purchased eval set for contamination?

Check a purchased eval set for contamination by tracing each task's origin, then searching for it and its paraphrases in public benchmarks, the open web and your own training corpora. Do this on a sample before you sign and on the full set at delivery.

1. **Trace provenance**: Ask for each task's author, creation date and source, and whether the seller has shown it to any other buyer.
2. **Search public benchmarks**: Compare tasks against well-known public sets in the same domain.
3. **Search the web**: Run sampled tasks through web search to catch tasks copied from forums, textbooks or exams.
4. **Check paraphrases**: Use a model-based similarity check, since Yang and colleagues showed exact matching misses rephrased items.
5. **Check your own corpora**: Search your training data, including synthetic data, for overlap before the first evaluation run.
6. **Use time cutoffs**: Prefer tasks created after your models' training cutoff, the approach LiveCodeBench takes by collecting new contest problems over time.

## Should you buy an evaluation set exclusively?

You should usually buy an evaluation set exclusively, at least for a fixed term, because every additional licensee is another route for tasks to reach training data. Exclusivity costs more, so match its length to how long you expect to use the set.

- Exclusive use for a stated term, with no sale of the same tasks or close variants to anyone else
- No reuse by the seller of tasks, answers or rubrics in its own training or other products
- No publication of the tasks, including in papers or marketing
- Leak notification within a set period if any task appears in public
- Replacement of leaked tasks at the seller's cost, or a partial refund

Our guide to [exclusive vs non-exclusive data licenses](https://sellmydatatoai.net/guides/exclusive-vs-non-exclusive-data-licenses) explains term, field and territory limits that can lower the price of exclusivity.

## How often should an evaluation set be refreshed?

Refresh an evaluation set on a schedule set by how quickly your domain changes and how widely the set is used. Published benchmarks give three models: rolling updates, a private held-out split and versioned retirement.

| Model | Example | How it works | Fits when |
| --- | --- | --- | --- |
| Rolling updates | LiveBench | Questions added and updated monthly from recent sources such as math competitions, arXiv papers and news | The domain changes quickly |
| Time-windowed collection | LiveCodeBench | New problems collected over time from LeetCode, AtCoder and CodeForces, so models can be scored only on problems after their cutoff | Tasks carry a natural release date |
| Private held-out split | Humanity's Last Exam | Public questions released while a private held-out set is kept to assess overfitting | You need a public score and a check on gaming |
| Unpublished set | FrontierMath | New, unpublished problems with automated verification | Tasks are expensive to write and must stay secret |

For a purchased set, write the refresh into the contract: how many new tasks per period, how retired tasks are handled and whether retired tasks may then be used for training.

## What are private benchmarks and who runs them?

Private benchmarks are evaluation sets whose tasks are withheld from the public so that model builders cannot train on them. Several US data vendors now run their own and sell evaluation work alongside training data.

| Publisher | Benchmark | What it publishes about the set |
| --- | --- | --- |
| Scale AI | SEAL leaderboards | Curated evaluation sets combining private datasets to prevent overfitting with open-source datasets |
| Mercor | Harvey LAB, with Harvey | Attorney-built benchmark of 1,749 real legal tasks across 24 practice areas, graded on finished work products |
| Mercor | APEX-Accounting | Long-horizon accounting tasks across accounting software, spreadsheets and PDFs |
| Surge AI | DAYJOB: Tax and others | Lists benchmarks including DAYJOB: Tax, a benchmark for AI tax agents |
| Epoch AI | FrontierMath | Several hundred unpublished, expert-written mathematics problems |

Ask who funded a benchmark and who can see its tasks. The FrontierMath paper, for example, acknowledges OpenAI for its support in creating the benchmark. Funding does not imply access, but a buyer comparing vendors' models should know the relationships.

## What does a private evaluation set cost?

Private evaluation sets are usually priced per task or per set, with exclusivity priced higher. We found no published per-task buyer prices from the vendors named in this guide in November 2026, so expect quotes rather than list prices.

The cost drivers are the expert time to write and verify each task, the rubric work for open-ended answers and the length of exclusivity. Our guide to [sourcing expert and human-generated data](https://sellmydatatoai.net/guides/sourcing-expert-and-human-data) lists the contributor pay rates vendors publish, which set a floor on task cost.

## What should an evaluation set license include?

An evaluation set license should limit use to evaluation, define exclusivity and set out leak remedies and refresh deliveries. It should also stop your own teams from training on the set by mistake.

- Use limited to evaluation, with training on the tasks prohibited for both parties
- Access controls: named teams, logged access and storage away from training pipelines
- Ownership of scoring code and any rubric changes you make
- Refresh deliveries and retirement rules
- Leak notification, replacement and refund terms
- Provenance records for each task, kept for audit

Company records can also become evaluation tasks. A resolved support ticket or a closed legal matter pairs a problem with a verified outcome. If you license records for that purpose, check that the seller has not licensed the same records to anyone for training, a point our [due diligence checklist](https://sellmydatatoai.net/guides/ai-training-data-due-diligence-checklist) covers.

## How can Sell My Data To AI help you source eval sets?

Sell My Data To AI shortlists vetted sellers whose records can be turned into private evaluation tasks, such as professional firms and software companies. Every seller passes our eight-point vetting standard, and you sign the license, including exclusivity, directly with the seller.

To start, [submit a data requirement](https://sellmydatatoai.net/request-data) or [book a buyer consultation](https://sellmydatatoai.net/book-a-consultation?side=buyer). If you plan to use the eval set to judge a pilot, read our guide to [training data pilot design](https://sellmydatatoai.net/guides/training-data-pilot-design).

## Frequently asked questions

### What is evaluation data contamination?

Evaluation data contamination is when tasks from a test set, or close paraphrases of them, appear in a model's training data. The model then scores well by recall rather than ability. Zhang and colleagues' 2024 GSM1k study found accuracy drops of up to 8% when leading models met fresh problems built to mirror the public GSM8k benchmark.

### Is n-gram matching enough to decontaminate an eval set?

N-gram matching is not enough on its own. Yang and colleagues showed in 2023 that paraphrased or translated test items pass n-gram checks, and they found 8% to 18% of the HumanEval benchmark overlapping the RedPajama-Data-1T and StarCoder-Data pre-training sets. Add a model-based similarity check and trace each task's origin.

### How long does a private eval set stay useful?

A private eval set stays useful until its tasks leak or models saturate it. Use and sharing speed up both. Published benchmarks manage this with refresh cycles, such as LiveBench's monthly question updates, or with a private held-out split, as Humanity's Last Exam keeps. Write a refresh schedule into any license you sign.

### Can I use company records as evaluation data?

Company records can make strong evaluation data when each one pairs a problem with a verified outcome, such as a resolved support ticket or a closed legal matter. Each task must be rewritten to stand alone and given a scoring rule. Check that the records have not been licensed to anyone else for training, or the set may already be contaminated.

### Should evaluation data be exclusive?

Evaluation data should usually be exclusive, at least for a fixed term, because each extra licensee raises the chance that tasks reach someone's training data. Agree that the seller will not reuse or publish the tasks, will notify you of any leak and will replace leaked tasks at its own cost.

## Sources

- [A careful examination of large language model performance on grade school arithmetic](https://arxiv.org/abs/2405.00332), arXiv (Zhang et al., 2024), accessed November 2026
- [Rethinking benchmark and contamination for language models with rephrased samples](https://arxiv.org/abs/2311.04850), arXiv (Yang et al., 2023), accessed November 2026
- [LiveBench: A challenging, contamination-limited LLM benchmark](https://arxiv.org/abs/2406.19314), arXiv (White et al., 2024), accessed November 2026
- [LiveCodeBench: Holistic and contamination free evaluation of large language models for code](https://arxiv.org/abs/2403.07974), arXiv (Jain et al., 2024), accessed November 2026
- [Humanity's Last Exam](https://arxiv.org/abs/2501.14249), arXiv (Phan et al., 2025), accessed November 2026
- [FrontierMath: A benchmark for evaluating advanced mathematical reasoning in AI](https://arxiv.org/abs/2411.04872), arXiv (Glazer et al., Epoch AI, 2024), accessed November 2026
- [Introducing SWE-bench Verified](https://openai.com/index/introducing-swe-bench-verified/), OpenAI, accessed November 2026
- [SEAL leaderboards](https://labs.scale.com/leaderboard), Scale AI, accessed November 2026
- [Mercor: Organizing human intelligence to power the AI economy](https://www.mercor.com/), Mercor, accessed November 2026
- [Surge AI: Human intelligence for AGI](https://www.surgehq.ai/), Surge AI, accessed November 2026

---

Book a free consultation: https://sellmydatatoai.net/book-a-consultation