This is a special post for quick takes by lilkim2025. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
To what extent does use of user data for training enable dominance lock-in for AI companies?
The current paradigm seems to be dominated by RLVR - that's when the recent spike in capabilities started, and it makes sense that training LLMs to get to the right answer for complex problems is more useful than training them to speak a certain way (RLHF) or to superficially emulate the training data (pre-RLHF).
Still, user data, properly filtered, strikes me as a powerful source of hard problems to train on, provided you filter for the users who ask the LLM for help on PhD-level research rather than help deciding what to eat for lunch. That data seems entirely exclusive to the companies currently running LLMs that the best researchers are using.
OpenAI wasn't able to lock in its dominance this way, but Anthropic getting in on enterprise coding early seems to have gotten them, in reputation, at least, a decisive advantage on practical coding tasks.
Assuming I woke up with ten billion dollars tomorrow, would I be able to get to parity by [1] scraping respected Github repos with quantitative metrics, booting up AutoResearch, and rewarding runs that improve the metrics, or is entry permanently out of reach to new players?