I wonder if we'll shortly be seeing mathematicians, and possibly other intellectual workers, "go dark," limiting how much they share of their results, or obfuscating them, in order to avoid getting scooped by AI. As I understand it, that's how mathematicians operated in the past.

The "scooped by AI" story in the NY Times today indicates that the AI startup that scooped the mathematicians relied on a proof roadmap they'd freely shared with the startup. My guess is that such free sharing will rapidly come to an end.

Running out of fresh training data is one thing. It's curable by paying people to mechanically generate more of it, or by using non-language inputs, and it's treatable by getting more leverage out of what already exists. But running out of open problems and roadmaps is a very specific kind of training data that's not nearly so easy to just mechanically regenerate, and that by it's nature requires reasoning and judgment, not just measuring stuff.

It'll be interesting to see if AI can generate new open problems in mathematics and new roadmaps for solving them, or whether it's in fact mostly dependent on humans for such steering.