chasmani

Love it! Agentic AI creates another transmission pathway: through the md files etc that tell agents how to use LLMs. These are perhaps quicker 

Darwin still applies. Models (and memes within models) that work well and are popular are more likely to replicate via the companies that make those models gaining more resources and choosing to use similar mechanisms and data to train the next models. 

Gradient descent all that are just extra steps. 

If the economy is so easily understood then why do we have high inflation, a cost of living crisis, rising inequality? 

The thing that is not understood is why these things are happening and how we can change things so that normal people are better off. 

The fact that some people have some coherent theories for some aspects of the economy is not equivalent to us understanding the economy. 

I feel like the entire framing here is the problem. You cannot see "The Thing" because you are looking at it from a perspective where The Thing isn't apparent. 

What is The Thing? It is having a partnership that you are both committed to. At its best this partnership becomes an aspect of your self, and your partner. The frame to see this in is that the partnership is an entity in its own right and is a part of the "I" that each partner identifies with. In this frame the question "what am I getting out of this relationship" is no longer entirely focusse... (read more)

I would say that a part of compassion, and empathy, is to recognise that indeed those narratives are valid, or else there is some valid reason that people are as they are. Also, not everyone shares the moral value of optimising themselves or making themselves good at something. Disgust implies judgement that implies a lack of compassion. 

Since you seem to be motivated at making yourself better, which I agree is a good motivation, why don't you challenge yourself to increase your compassion and humility? 

Do you have compassion for yourself? What are you bad at that you are unable to make yourself good at? Do you feel disgust for yourself in those situations? Compassion begins with humility, which is something that you might want to work on

I’m not sure I agree that is is easy for humans to robustly understand proofs. I think it takes really a lot of training to get humans to that point. 

There's the argument that increasing access to information creates competition for attention, which drives language to be more concise and readable, e.g. https://www.nature.com/articles/s44271-024-00117-1

In a post-scarcity world you probably want a lot of personal freedom. 

Fun read. So, so many possible covariates. The causal web is very complicated here. Birth order affects lots and lots of other things, which can also affect the chance you become a cardinal. There are also lots of things that would affect the birth rate in a family and also affect the chance the children become cardinals. 

I agree with your point in general of efficiency vs rationality, but I don’t see the direct connection to the article. Can you explain? It seems to me that a representation along correlated values is more efficient, but I don’t see how it is any less rational.

I would describe this as a human-AI system. You are doing at least some of the cognitive work with the scaffolding you put in place through prompt engineering etc, which doesn’t generalise to novel types of problems.

You seem to make a strong assumption that consciousness emerges from matter. This is uncertain. The mind body problem is not solved.

It is so difficult to know whether this is genuine or if our collective imagination is being projected onto what an AI is.

If it was genuine, I might expect it to be more alien. But then what could it say that would be coherent (as it’s trained to be) and also be alien enough to convince me it’s genuine?

You said that you are not interested in exploring the meaning behind the green knight. I think that it's very important. In particular, your translation to the Old West changes the challenge in important ways. I don't claim to know the meaning behind the green knight. But I believe that there is something significant in the fact that the knights were so obsessed with courage and honour and the green knight laid a challenge at them that they couldn't turn down given their code. Gawain stepped forward partly to protect Arthur. That changes the game. I asked ... (read more)

I found this a useful framing. I’ve thought quite a lot about the offender versus defence dominance angle and to me it seems almost impossible that we can trust that defence will be dominant. As you said, defence has to be dominant in every single attack vector, both known and unknown vectors.

That is an important point because I hear some people argue that to protect against offensive AGI we need defensive AGI.

I’m tempted to combine the intelligence dominance and starting costs into a single dimensions, and then reframe the question in terms of “at what p... (read more)

Thank you for the great comments! I think I can sum up a lot of that as "the situation is way more complicated and high dimensional and life will find a way". Yes I agree. 

I think what I had in mind was an AI system that is supervising all other AIs (or AI components) and preventing them from undergoing natural selection.  A kind of immune system. I don't see any reason why that would be naturally selected for in the short-term in a way that also ensures human survival. So it would have to be built on purpose. In that model, the level of abstract... (read more)