Philip Niewold

Excellent from-the-heart post. Predictability and stability is a great good, and if you have a large imagination and good intellect, you can become lost in your own projections easily. I know I do.

You have just realized that just working towards some future is not a viable path to living. This is a lesson most people take decades to discover. Perhaps you look happier because your mind was forced to live more and the here and now and less in the future, and living in the here and now is really.

It is hard to both grasp and let go. But that is really the only option we have.

You make a number of interesting points. 

Interpretabilty would certainly help a lot, but I am worried about our inability to recognize (or even agree) to leaving local optima that we believe are 'aligned'. 

Like when we force a child to go to bed by his parents when it doesn't want to, because the parents know it is better for the child in the long run, but the child is still unable to extend his understanding to this wider dimension.

Ar some point, we might experience what we think of as misaligned behaviour when the A.I. is trying to push us out ... (read more)

While I acknowledge this is important, it is a truly hard problem, as it often involves looking not just at first-order consequences, but also at second-order consequences and so on. People are notoriously bad at predicting, let alone managing side-effects.

Besides, if you look at it more fundamentally, human natures and technological progress in a broad sense has many of these side effects, where you basically need to combat human nature itself to have people take into account the side-effects.

We are still struggling coming to terms and accurately classify... (read more)

An LLM it a tool of communicative expression, but so it the written or spoken word, music etc. It is a medium throug which the intent travels. As a Dutchman, I have a preference of being direct and clear, but the impact of my words sometimes have the opposite effect, as my listeners do not have my context and can react emotionally to a worded message that is meant factually. If an LLM can help me translate such expression to a language that is better for my target audience to understand, then it is similar to translating into another language.

Still, the wr... (read more)

Our exploration system is very useful, but it takes a lot of energy (and anxiety), because of the inherent cost of failure which genetics baked into our brain. Hence, doing something new everyday in a society as complex and everchanging as our own is very useful, but very hard with our outdated brain hardware and software.

Add to that the distractions that hijack our outdated brain mechanisms: we have gotten better and better and such hijacking, creating an additional barrier. Doing this is comparably difficult to keeping to a strict diet and exercise regim... (read more)

I think your claim the rudimentary abilities arrive before transformational ones cannot be applied to A.I. the same as human intelligence. While humans might have taken millennia to go from caveman painting to our current ability to produce artistic images, it is clear that A.I. became transformational very quickly in that particular field. You see the same transformational abilities in text writing, music and video too and software development is getting there.

Some of the more artistic of these abilities don't have a clear benchmark, but even with more fu... (read more)

Your working paper, "Open Global Investment as a Governance Model for AGI." It provides a clear, pragmatic, and much-needed baseline for discussion by grounding a potential governance model in existing legal and economic structures. The argument that OGI is more incentive-compatible and achievable in the short term than more idealistic international proposals is a compelling one.

However, I wish to offer a critique based on the concern that the OGI model, by its very nature, may be fundamentally misaligned with the scale and type of challenge that AGI prese... (read more)

I don't think people in general react well to societal existential risks, regardless how well or courageous the message is framed. These are abstract concerns. The fact that we are talking about AI (an abstract thing in itself) makes it even worse.

I'm also a very big opponent of arguing by authority (I really don't care how many nobel laureates are of the opinion of something, it is the content of their argument I care about, now how many authorities are saying it). That is simply that I cannot determine the motives of these authorities and hence their opi... (read more)

Social messaging is fine balancing act: people like to offload responsibility and effort, especially if it doesn't come at the cost of status. And, to be honest, you don't know if your question would impose upon the other (in terms of cognitive load, social pressure or responsibility), so you it is smart to start your social bid low and see if the other wants to raise the price. Sometimes they work, creating a feedback loop similar to how superstitions evolve: if it is minimal effort and sometimes it is effective, better continue using it.

As a child, I des... (read more)

Of course it is perfectly rational to do so, but only from a wider context. From the context of the equilibrium it isn't. The rationality your example is found because you are able to adjudicate your lifetime and the game is given in 10 second intervals. Suppose you don't know how long you have to live, or, in fact, now that you only have 30 seconden more to live. What would you choose?

This information is not given by the game, even though it impacts the decision, since the given game does rely on real-world equivalency to give it weight and impact. 

Any Nash Equilibrium can be a local optimum. This example merely demonstrates that not all local optima are desirable if you are able to view the game from a broader context. Incidentally, evolution has provided us with some means to try and get out of these local optima. Usually by breaking the rules of the game or leaving the game or seemingly not acting rationally from the perspective of the local optimum.

Please keep in mind that the Chat technology is an desired-answer-predicter. If you are looking for weird response, the AI can see that in your questioning style. It has millions of examples of people trying to trigger certain responses in fora etc, en will quickly recognize what you really are looking for, even if your literal words might not exactly request it.

If you are a Flat Earther, the AI will do its best to accomodate your views about the shape of the earth and answer in a manner that you would like your answer to be, even though the developers of ... (read more)