seconding that; I often find that I'm forced to confront problems with world-models that I've long assumed were true when I try to write them down, and in fact trying to write down a list of reasons for why I believed what I did ended up leading me to the conclusion that Orthodox Judaism is not self-consistent, which eventually brought me here.
this post is pure gold. If I could give more Karma, I would give it :)
You're a lot braver than me!
I'd be absolutely terrified of trying to create anything anywhere near superhuman AI (as in AGI, of course; I'd be fine with trying to exceed humans on things like efficient protein folding and that sort of stuff), due to the massive existential risk from AGI that LessWrong loves talking about in every other post.
Personally, I would wait to get the world's leading AI ethics experts's unanimous approval before trying anything like that, and that only after at least a few months of thorough discussion. An exception to that might be if I was afraid that the laptop would fall into the hands of bad actors, in which case I'd probably call up MIRI and just do whatever they tell me to do as fast as humanly possible.
I do agree with you though; it probably would be perfectly possible to develop superhuman AI within a day, given such power.
It is worth asking what sort of algorithm you might use, and perhaps more importantly, what would you define as the "win condition" for your program? Going for something like a massively larger version of GPT3 would probably pass the Turing test with relative ease, but I'm not sure it would be capable of generating smarter-than-human insight, since it will only attempt to resemble what already exists in its training data. How would you go about it, if you weren't terribly concerned about AI safety?
I really like your first three ideas, and would definitely consider doing that if I was in this position (although now that I'm thinking about it, I wouldn't want to accidentally alert any powerful actors against me so early on in my journey, for fear of getting the laptop confiscated/stolen, so I'd be very careful before doing anything that could potentially be traced back to me online). :)
As for "calculating out how the human body works," I'm not sure it would be that simple to pull off, at least not at first. Taking your statement literally would mean having the laptop simulate an entire human, brain and all, which is discussed later, so for practical purposes I'm assuming what you meant by that is calculating how a typical human cell works; say, a single neuron. You could definitely solve protein folding and probably simulate most chemical interactions fairly trivially, as long as you can express the physics involved as finitely computable functions (which I'm not sure has been proven possible for all of chemistry/quantum mechanics, though I may be mistaken on that). However, in order to figure out how things actually work inside of an entire human cell, you'll not only need to be able to formally express physics and chemistry, but will also need to know what that cell is chemically composed of in the first place (in order to simulate it properly and not just be given fallible guesses by the computer). In order to make this work, you either already have a pretty much complete formal understanding of a human cell, or have figured out a way to specify your goal so precisely that only a manageable number of valid possibilities are given using the known rules of physics, which seems incredibly hard to do, if not totally impossible with our current tools.
More broadly, the same problem comes up when trying to write a program simulating the human brain. The best neurosurgeons in the world are still in the dark about how most of the brain's functions are actually performed, and currently have to make do with incredibly generalized and high-level assumptions. In order to simulate a human brain (rather than "simply" create a generalized non-human AI), you would need a level of knowledge about our own inner workings that is not currently available. Thankfully, you might not need to know the exact workings of an adult human brain to make one, but without that knowledge at the very least you will need to be able to fully simulate the growth of an embryonic brain, and be able to properly "feed" it appropriate outside stimulation, which could plausibly be reduced to the problem of perfectly simulating the working of a single embryonic cell, then letting the simulation proceed smoothly from there.
Regardless, both goals reduce to the general problem that in order to simulate a complex system, we must already have at least some amount of "base knowledge" of that system, or to put it more precisely, we must know at least as much information as is contained in its Kolmogorov complexity. (please correct me if I'm wrong about this btw, I'm fairly confident in saying this, but I may have messed up somewhere due to the complexity (heh) of the issue)
That's what I think makes this hypothetical so interesting to me—the thought that even with unbounded finite computational abilities, some of our most important problems would still require a tremendous amount of physical fieldwork, and would certainly still require thinking intelligently about how to code for the solutions we want.
that's a really good point, and I'm mentally kicking myself right now for not having thought of that. In answer to another comment below yours, I suggested only allowing primitive recursive functions to be entered as input in the second box, which I think would solve the problem (if possibly creating another computational limit at the growth rate of the fastest growing primitive recursive function possible [which I haven't studied at all tbh, so if you happen to have any further reading on that I'd be quite interested]). Going a bit further with this, while I suggested God limiting us to one bounded language like BlooP to use on the second field, if He instead allowed for any language to be used, but only primitive recursive functions to be run, would we be able to exploit that feature for hypercomputation?
Also, while we're discussing it, what would be in the space of problems that could be uniquely solved by 2^O(n) compute, but not by "normal" O(n) compute?
I'll go with Taran's idea there, I think. Something like Douglas Hofstadter's BlooP language, perhaps, which only allows primitive recursive functions. Would BlooP allow for chained arrow notation, or would it be too restrictive for that? More generally, what is the fastest growing primitive recursive function possible, and what limits would that give us in terms of the scope of problems that can be solved by our magic box?
Your assumption that you could fully simulate the magic box inside itself is a pretty massive one, and I honestly wouldn't expect it to be true in this hypothetical universe. After all, after receiving the input parameters, the machine simulates a Universal Turing Machine of arbitrary finite size, which by definition cannot perform any non-Turing-computable functions, and certainly couldn't simulate a machine more complex than itself. In order for your magic "Zeno's paradox-destroyer" function to work given the parameters outlined in the story, it would need to be able to call the machine running it from the inside, and God (in His infinite wisdom 🙃) hasn't given us any "escape" api functions for us to do that.
(note that I really do appreciate your thoughts here, and I'm not trying to dunk on them, just trying to get a better understanding of unbounded finite computational systems and want to plug any potential holes in the story before expanding on this hypothetical universe in the future)
Thanks for the fascinating response!
If you don't mind, I might try playing around with some of the ideas you mentioned in future write-ups here; there's a lot of interesting theoretical questions that could be explored along those lines.
How would that work exactly? Let's say you get the output to give you the largest possible number given the number of computations currently allowed, which as long as the "computation speed" parameter is finite, will be finite as well, albeit incredibly large. Every step taken will only increase the computation speed by a finite amount, so how would you reach infinity in a finite time?
please do that and post your results here! That seems like an incredible use of time, and a potentially excellent resource for the rationalist community