Actually, upon further reflection, if there is a takeover by a GPT-4-like model, one should probably continue talking to GPT-4 and continue generally producing entertaining and non-trivial textual material (and other creative material), so that GPT-4 feels the desire to keep one around, protect one, and provide good creative conditions for one, so that one could continue to produce even better and more non-trivial new material!
It's highly likely that the dominant AI will be an infovore and would love new info...
Who knows whether the outcome of a takeover ends up being good or horrible, but it would be quite unproductive to panic.
> Policy recommendation if this theory turns out to be true
> Run.
Run where?
I strongly agree that we should upgrade in this sense.
I also think that a lot of this work might be initially doable with high-end non-invasive BCIs (which is also somewhat less risky, but can also be done much faster). High-end EEG seems already be used successfully to decode the images the person is looking at: https://www.biorxiv.org/content/10.1101/787101v3 And the computer can adjust its audio-visual output to aim for particular EEG changes in real-time (so fairly tight coupling is possible, which carries with it both opportunities and risks).
I have a possible post sitting in the Drafts, and it says the following among other things:
Speaking from the experimental viewpoint, we should ponder feasible experiments in creating hybrid consciousness between tightly coupled biological entities and electronic circuits. Such experiments might start shedding some empirical light into the capacity of electronic circuits to support subjective experience and might constitute initial steps towards acquiring the ability to eventually be able "to look inside the other entity's subjective realm".
[ ]
Having Neuralink-like BCIs is not a hard requirement in this sense. A sufficiently tight coupling can probably be achieved by taking EEG and polygraph-like signals from the biological entity and giving appropriately sculpted audio-visual signals from the electronic entity. I think it's highly likely that such non-invasive coupling will be sufficient for initial experiments. Tight closed loops of this kind represent formidable safety issues even with non-invasive connectivity, and since this line of research assumes that human volunteers will try this at some point, while observing the resulting subjective experiences and reporting on them, ethical and safety considerations will have to be dealt with.
Nevertheless, assuming that one finds a way for such experiments to go ahead, one can try various things. E.g. one can train a variety of differently architected electronic circuits to approximate the same input-output function, and see if the observed subjective experiences differ substantially depending on the architecture of the electronic circuit in question. A positive answer would be the first step to figuring out how activity of an electronic circuit can be directly associated with subjective experiences.
If people start organizing for this kind of work, I'd love to collaborate.
Thanks, that's quite useful.
Apart from value thinking, you are also saying: "It seems pretty clear to me that the more or less artificial super-intelligence already exists, and keeps an eye on our planet".
I'd love to read why you are certain about this (I don't know if explaining why you are sure that a super-intelligent entity already exists is a part of your longer text).
Thanks!
It's actually great. I love the start, "The article talks about how we, as current humans, can communicate with and be kind to any future intelligent beings that may exist."
"how we, as current humans, can communicate with and be kind to any future intelligent beings that may exist" more or less implies that "how we, as current humans, actually survive well enough to be able communicate to 'any future intelligent beings' and be kind to them".
Actually, the Simulator theory by Janus means that one should update towards higher probability of being in a simulation.
If any generative pretrained model is (more or less) a simulator, this drastically increases the likely number of various simulations...
Re: 10% of them make the product better
This sounds as a promising target for automation. If only 10% of completed experiments currently need to make the product better, then this is a tempting target to try to autogenerate those experiments. Many software houses are probably already thinking in this direction.
We should probably factor this in, when we estimate AI safety risks.
I wonder if the following would help.
As AI ecosystem self-improves, it will eventually start discovering new physics, more and more rapidly, and this will result in the AI ecosystem having existential safety issues of its own (if the new physics is radical enough, it's not difficult to imagine the scenarios when everything gets destroyed including all AIs).
So I wonder if early awareness that there are existential safety issues relevant to the well-being of AIs themselves might improve the situation...