There is no posttraining data commons

The AI industry’s shift toward posttraining—especially agentic reinforcement learning—is constrained by the absence of a shared, ready-made data commons comparable to the web corpora that fueled pretraining, according to a new analysis. Pretraining drew on roughly 10 petabytes of Common Crawl archives plus Wikipedia, ArXiv, Stack Exchange, Project Gutenberg and open-source code, with GPT-3 trained on about 60% Common Crawl-derived data and the original Llama mix almost entirely web-sourced. Posttraining instead consumes tasks rather than documents, and those tasks must largely be built and curated rather than harvested, making data the binding constraint as frontier labs pour compute into agentic research. The piece notes that incumbent labs now run custom crawlers—Cloudflare reports over half of crawler traffic is AI-related—while open models such as Nemotron still depend on the commons for most pretraining data. The takeaway for the sector is that continued open access to web archives underpins open LLMs, and the next watch item is how task-generation and curation pipelines scale to meet posttraining demand.