Epistemic status: Probably worth thinking about as a problem, but possibly not worth posting public solutions to
This seems to be a known question in AI alignment, I just thought it would popularising and getting more discussion on. (Those who are already deep into AI alignment may or may not find this a promising direction to work on. I just felt that it's a problem sufficiently abstracted from AI alignment stuff, that even people don't know too much AI background could contribute.)
Provide a set of mathematically expressible problems whose solutions would allow you personally (or a small group of scientists with less than $10B in funding) to establish significant totalitarian control over the world. These problems should contain as little information about the real world as possible.
A mathematical statement of Goldbach's conjecture, along with the base set of axioms contains very little information about the real world. A formal description of protein folding, along with descriptions of how proteins work and interact, along with what proteins are made up of and how those work, contains a lot of information about the real world.
You can consider the Kolmogrow complexity of your problem statement as a possible indicator for how much information it exposes about the real world. (Although beyond a certain point, this may not be a good indicator.)
One of the proposals to the AI alignment problem is a pivotal act that prevents anyone else from building unaligned AGI. One way of doing this is setting up your own personal global dictatorship (singleton) that monitors and enforces this. One way of getting there is to use a Task AI that is superhuman at only one narrow field of study, that does not contain too much information about the real world in it. For instance, number theory or more broadly, proving math theorems.
Not containing information about the real world is important because it reduces the odds the AI tries to discover or discovers strategies like "figure out if you're in a box or simulation, break out of it, eliminate civilisation and get back to math problem with more compute power". The first of these steps is important here, "figure out if you're in a box or simulation, and if there's a real world outside". If it has no real world information or any priors that a "real world" exists, the AI may be less likely to decide this as a problem worth spending compute power on, as opposed to directly attacking the math problem. It is still an open question whether that would actually happen, or whether "figuring out if you're in a simulation" is so instrumentally convergent that there can be no guard against it. But for now, assume this whole thing is a viable strategy.
The problem now is that any set of problems whose solutions let you take over the (real) world, containing too much information about the world. "Solve Goldbach's conjecture" contains a lot less information about the world than "solve protein folding, given information about what proteins are and how they work". Real world strategies to establish a dictatorship usually require a lot of consideration of information about the real world, rather than abstract math problems. But it is possible the hardest challenges are in fact reducible. "Establish dictatorship" may reduce to "build one particular form of nanotech" which may reduce to "solve protein folding" which may reduce to "solve this complex differential equation" which may reduce to "solve this broader more abstract math theorem that contains even lesser real world information than the differential equation".
This research might go in two directions. First direction is reducing the information exposed by human solution. Problem_A(x1,y1,z1) can be reducible to Problem_B(x1,z1) if humans themselves know how to convert a solution to latter into a solution to the former. The latter contains less real world information. For instance, how does nanotech translate into establishing world dictatorship could be a problem that only humans are pointed at, not AI. The other direction is generalising to reduce information exposed. Problem_A(x1,y1,z1) could be reducible to Problem_A(x,y,z) for all x,y,z in X, Y, Z. So you don't expose data about what specific values the problem has to be instatiated with to be relevant to the real world. For instance, instead of asking about electron interactions given some constants like electron mass and permittivity, and an inverse square law - ask the AI to make progress on all differential equations that contain polynomials. This avoids revealing the fact that inverse square laws are specifically important to the real world. The human can then pick out whatever progress the AI makes on inverse square laws.
The querying could possibly be interactive. For instance you might want to solve one problem then react to the solution to that problem to figure out what the next problem worth solving is and how to cleverly pose it to the AI. Depending on how safe or unsafe the AI, you can take that many steps to interactively enhance your powers. But for now, I guess you can try to find key steps such that you can large progress using fewer number of them.
Lastly, even if you don't become a world dictator yourself - having any significant power could be enough to get the powers-that-be to atleast take you seriously. Maybe you can negotiate existing powers to into a place where the agents they are composed of, gain more than they lose by co-operating and establishing this monitoring regime.
Solving this problem might pose an infohazard. My thinking on this so far is that all of us on this forum are aligned enough that even if someone* besides me uses my idea to become a world dictator, it may still be a better world than one where US and China remain in control and neither of them has singleton and neither of them thinks AI alignment is as important as people here. That being said, I would not be surprised by a different set of deranged behaviours from people here either. (I'm pretty sure I qualify by some people's definitions of deranged simply for thinking about this problem.) I haven't thought this through though - so if you have strong reason to not post solution attempts or discourage others from posting them - I'd be keen to know about it. Private channels to communicate solutions may be worthwhile.
*I guess it also depends on who the someone is, is it a random person on this forum or MIRI or the US govt? And how democratic a structure they wish to set up. Just because you can be a world dictator in all respects doesn't mean you want to. Maybe you just setup a strong social or military institution that does monitoring and enforcement of AI, but does not otherwise interfere in global governance affairs.
To make this less of an infohazard, maybe I should explicitly ask for solutions that enable monitoring and enforcement of AI, without also enabling totalitarianism in other respects. This seems to me to be much harder, but I'm happy to be proven wrong.