Data types

Agent trajectories and RL environments explained

By Gareth HoyleLast updated November 8, 20266 minute readHow we research

In short

An agent trajectory is the step-by-step record of an AI agent or a person completing a task in software: each observation, click, tool call and result. An RL environment is the replayable sandbox where those tasks run and are scored. AI labs buy both to train agents with reinforcement learning, and they value realistic tasks whose outcomes can be checked automatically.

Key takeaways

A trajectory records the steps of a task, while an environment lets a buyer rerun the task and generate new trajectories.
TechCrunch reported in September 2025 that Anthropic leaders had discussed spending more than $1 billion on RL environments over the following year, citing The Information.
Benchmarks such as SWE-bench, OSWorld and tau-bench show what good tasks look like: real software, a clear goal and automatic scoring.
Buyers must check personal data in logs and screenshots, the rights to the software inside the environment and whether the scoring can be gamed.

AI labs are moving from models that answer questions to agents that complete tasks in software. Training those agents needs a different kind of data: records of tasks being done, and sandboxes where tasks can be done again and scored.

What is an agent trajectory?

An agent trajectory is the record of an agent working through a task, step by step. DataVendor describes it as each click, tool call, observation and final result in a workflow. The agent may be an AI model or a person whose actions are logged.

What a single trajectory step usually records
Field	What it holds
Task	The goal, such as resolving a support ticket or fixing a failing test
Observation	What the agent could see, such as a screenshot, page structure or tool output
Action	What the agent did, such as a click, keystroke, API call or code edit
Reasoning	Notes or model output explaining the action, where captured
Result	The new state after the action
Outcome	Whether the task succeeded, and how that was judged

Trajectories that fail are useful too, provided the outcome is recorded honestly. See the agent trajectory glossary entry for a short definition.

What is an RL environment?

An RL environment is a controlled copy of a piece of software where an agent can attempt a task, be scored and start again. TechCrunch describes environments as training grounds that simulate what an AI agent would be doing in a real software application, such as a simulated browser where an agent is graded on completing a purchase.

An initial state that can be reset, such as a configured desktop, repository or database
A task description
The actions and tools available to the agent
A verifier or scoring script that judges the outcome

DataVendor explains the difference for buyers: a replayable environment adds the software or sandbox needed to rerun a task, so buyers can generate fresh trajectories and train models against verified outcomes. See the RL environment glossary entry.

Why are AI labs buying trajectories and environments?

AI labs are buying trajectories and environments because reinforcement learning on checkable tasks has become a main route to better agents. The Allen Institute for AI's Tulu 3 paper, submitted to arXiv in November 2024, introduced the term Reinforcement Learning with Verifiable Rewards for training against outcomes that can be checked automatically.

Spending has followed. TechCrunch reported in September 2025 that Anthropic leaders had discussed spending more than $1 billion on RL environments over the following year, citing The Information. The same article named Scale AI, Surge and Mercor among data companies moving into environments, alongside start-ups such as Mechanize and Prime Intellect.

“All the big AI labs are building RL environments in-house”
Jennifer Li, Andreessen Horowitz, quoted by TechCrunch, September 2025

Not everyone is convinced. Ross Taylor of General Reasoning told TechCrunch that people are underestimating how difficult it is to scale environments. For buyers, that difficulty is the reason well-built environments and real-world trajectories carry value.

What do benchmarks show about good agent tasks?

Benchmarks show that good agent tasks use real software, have a clear goal and are scored automatically. Three widely cited benchmarks illustrate the pattern.

Three agent benchmarks and how they are built
Benchmark	Tasks	Environment	Scoring	Result at publication
SWE-bench (2023)	2,294 real GitHub issues from 12 popular Python repositories	The repository codebase and the issue text	Whether the edit resolves the issue	Best model resolved 1.96% of issues
OSWorld (2024)	369 computer tasks	Real Ubuntu, Windows and macOS environments	A custom evaluation script per task	Humans 72.36%, best model 12.24%
Tau-bench (2024)	Customer service tasks with a simulated user	Domain API tools and policy rules	Final database state against the goal state	GPT-4o under 50%, pass^8 under 25% in retail

Each one is built from the same parts a buyer should look for: a realistic starting state, a task drawn from real work and an objective check. SWE-bench uses real issues and pull requests. OSWorld gives each task an initial-state configuration and an evaluation script. Tau-bench adds a pass^k metric to measure reliability over repeated trials.

What makes trajectory data valuable?

Trajectory data is valuable when it is complete, real and checkable. A log of clicks with no task or outcome attached is hard to train on.

Every step is captured, including what the agent saw before acting
The task and the outcome are recorded, including failures
The work happened in software people actually use, not a toy simulation
The operators were skilled at the task, which matters for expert domains
An environment exists to replay the task or generate new attempts
Personal data and credentials have been removed from screenshots and logs

DataVendor notes that on its marketplace, raw trajectories, internal support workflows and software-use logs can be packaged as raw data ZIPs, while environments cannot be listed on their own and are sold with tasksets. Our agent trajectory data page describes formats buyers ask for.

Where does trajectory data come from?

Trajectory data comes from three main places: companies that log how staff use software, AI product companies that record agent runs and specialist vendors that commission experts to perform tasks.

Support and operations teams, whose ticket workflows show multi-step problem solving
Software companies, whose product event data and internal tooling logs show real use
AI product companies, whose agent run traces, user feedback and ratings show where agents succeed and fail
Data vendors, who pay experts to complete tasks in prepared environments

If you hold this kind of data, our pages for software companies and call centers explain what buyers ask for.

What should buyers check before licensing trajectories or environments?

Before licensing trajectories or environments, buyers should check rights, privacy and scoring quality. Each creates a different kind of risk.

Checks for trajectory and environment purchases
Risk	What to check
Personal data	Screenshots, page captures and logs often show names, emails and account details
Credentials	Tokens, passwords and API keys captured in tool calls or terminal output
Third-party software	Whether the licenses of applications inside the environment allow this use
Employer and client rights	Whether staff activity logs and client systems can be licensed, and under which privacy notice
Scoring quality	Whether the verifier can be passed without completing the task
Reproducibility	Whether the environment resets to the same state and gives the same score
Contamination	Whether tasks overlap with public benchmarks you report on
Derived rights	Whether you may generate new trajectories, modify the environment and keep trained policies

Test a sample of tasks yourself before buying a full set. How to evaluate a dataset sample covers the method, and evaluation datasets covers task sets used for scoring rather than training.

How are trajectories and environments priced?

Trajectories and environments are usually priced per task or per task set rather than per record. AIData.Broker lists the task among the standard pricing units, as quoted by ddiy.co, and DataVendor sets task and environment rates across each listing's tasksets.

Public price ranges for this data are scarce. Treat any quote as specific to the task difficulty, the expertise needed and whether an environment is included. AI training data pricing models explains the structures in use.

How does the EU AI Act treat this data?

The EU AI Act treats human trajectory and reinforcement learning data as a training source that providers must describe. The Commission's training content template names datasets labeled by humans commissioned by the provider and human-generated reinforcement learning data in Section 2.6, while AI feedback used in reinforcement learning falls under synthetic data in Section 2.5.

Keep a record of where each task set and trajectory came from so you can complete those sections. Our guide to the EU AI Act and training data transparency explains the full template. To source trajectories or environments, book a buyer consultation.

Frequently asked questions

What is the difference between a trajectory and an environment?

A trajectory is a recording of one attempt at a task, while an environment is the sandbox where tasks can be attempted again and scored. Trajectories suit supervised training on demonstrated behavior. Environments suit reinforcement learning, because a model can generate many new attempts and learn from the scores.

Can a company sell logs of how its staff use software?

A company can sometimes sell logs of how its staff use software, but it needs to check its privacy notices, employment terms and the licenses of the software being used. Screenshots and logs often contain personal data, client information and credentials, which must be removed or the records excluded. Take independent legal advice before licensing staff activity data.

Why do verifiable outcomes matter?

Verifiable outcomes matter because reinforcement learning needs a reliable score for each attempt. The Tulu 3 paper from November 2024 named the approach Reinforcement Learning with Verifiable Rewards. If the score can be passed without doing the task, the model learns to pass the check rather than do the work, so buyers test verifiers before buying.

Are public benchmarks like SWE-bench available to train on?

Public benchmarks like SWE-bench are designed for evaluation, and training on them makes your reported scores unreliable. They are useful as a model of what good tasks look like: real issues, a realistic codebase and an objective check. Buyers looking for training data usually want private tasks built the same way but kept out of public benchmarks.

Sources

AI training data marketplaces, DataVendor, accessed November 2026
Silicon Valley bets big on environments to train AI agents, TechCrunch, accessed November 2026
Tulu 3: Pushing frontiers in open language model post-training, arXiv (Lambert et al.), accessed November 2026
SWE-bench: Can language models resolve real-world GitHub issues?, arXiv (Jimenez et al., ICLR 2024), accessed November 2026
OSWorld: Benchmarking multimodal agents for open-ended tasks in real computer environments, arXiv (Xie et al.), accessed November 2026
Tau-bench: A benchmark for tool-agent-user interaction in real-world domains, arXiv (Yao et al.), accessed November 2026
How much is company data worth to AI?, ddiy.co, accessed November 2026
Explanatory notice and template for the public summary of training content for general-purpose AI models, European Commission, Shaping Europe's digital future, accessed November 2026