The data efficiency gap remains unexplained, posing a challenge for AI architects and cognitive scientists.
Children still outperform the most sophisticated language models when it comes to learning efficiency, and researchers do not know why. A new look at the data efficiency gap shows that large language models consume vastly more text than a child hears in a lifetime.
Stanford cognitive scientist Michael C. Frank and Georgetown linguist Ethan Gotlieb Wilcox study the gap. Meta’s Llama 3.1 model used 15 trillion tokens in pretraining, and frontier models may use ten times more. A preteen raised in a linguistically rich home may hear around 100 million words.
For builders, data is a finite resource, and the internet may run dry for pretraining as early as the 2030s. The path forward lies in sample efficiency, not just scale. Teams should track research that mimics child learning, because it could unlock models that train on far fewer tokens.
Researchers will keep probing why children learn language so efficiently, and that work could reshape AI pretraining. Watch for new benchmarks that measure data efficiency rather than raw scale. Cognitive science may soon influence model architectures in concrete ways.
What matters
- A modern LLM can require a hundred thousand times more language data than a child.
- For builders, the scarcity of training data will push innovation in data efficiency.
- Watch for new methods mimicking child learning to shrink AI’s massive data appetite.
Why it matters
Watch for new methods mimicking child learning to shrink AI’s massive data appetite.
This GenAI News article was prepared in original wording using reporting and materials published by MIT Technology Review AI. Source reference: https://www.technologyreview.com/2026/08/24/1141740/kids-machines-language-learning/.
Drafted by the GenAI News review pipeline.