An algorithm for solving PSPACE complete problems in polynomial time would probably get you a good chunk of the way there, although there's no particular reason to believe this is possible other than the fact that nobody has yet proven it to be impossible.
I think the optimistic case might be that in order to get the AGI to do anything useful at all you have to get at least part-way to a solution to the alignment problem, because otherwise its outputs will include many that will be so obviously "wrong" that you'd never actually let it do anything in which being wrong mattered.
I linked to it because it seemed like Robin Hanson was saying something close to the opposite of this.
Right now the bottleneck for becoming able to legally practice medicine as a doctor in the US is the number or residency positions for training medical school graduates, not the number of people graduating from medical schools.
Maybe imagine a dog?
My impression as an outsider (I met him once and heard and read some things people were saying about him) was that he seemed smart but also seemed like kind of a kook...
As the joke goes, there's nothing crazy about talking to dead people. When dead people respond, then you start worrying.
Which is why I specified "an algorithm" and not "a proof".
Also, if my understanding is correct, simulating quantum systems is in PSPACE, so one thing this would do is make nanotechnology much easier to develop...