All of Max TK's Comments + Replies

I guess this means they found my suggestion reasonable and implemented it right away :D I am impressed!

Apparently you need to have experimental features on.

I think there is an option for whether they can be promoted to front page.

Answer by Max TKAug 25, 202310

When I am writing my articles, I prefer a workflow in which I am able to show my article to selected others for discussion and review before I publish. This seems to not be possible currently without giving them co-authorship - which often is not what I want.

This could be solved for example by having one additional option that makes the article link accessible by others even while it is in draft mode.

Unless you are specifically talking about writing a wiki article (which I have 0 experience), it is possible to do this with a draft without co-authorship.
1Nathan Young3mo
I sense this is possible, though what I want is for articles to just be public but not posted to the front page.

Update: Because I want to include this helpful new paragraph in my article and I am unable to reach Will, I am now adding it anyways (it seems to me that this is in spirit of what he intended). @Will: please message me if you object

Lovely; can I add this to this article if I credit you as the author?

Just saw this, sure!
1Max TK3mo
Update: Because I want to include this helpful new paragraph in my article and I am unable to reach Will, I am now adding it anyways (it seems to me that this is in spirit of what he intended). @Will: please message me if you object

On How Yudkowsky Is Perceived by the Public

Over the recent months I have been able to gather some experience as an AI safety activist. One of my takeaways is that many people I talk to do not understand Yudkowsky's arguments very well.

I think this is for 2 reasons mainly:

  1. A lot of his reasoning requires a kind of "mathematical intuition" most people do not have. In my experience it is possible to make correct and convincing arguments that are easier to understand, or even invest more effort into explaining some of the more difficult ones.

  2. I think he i

... (read more)

I think a significant part of the problem is not the LLMs trouble of distinguishing truth from fiction, it's rather to convince it through your prompt that the output you want is the former and not the latter.

My argument does not depend on the AI being able to survive inside a bot net. I mentioned several alternatives.

You were the one who made that argument, not me. 🙄

How are inner layers relevant?

Usually between people in international forums, there is a gentlemen's agreement to not be condescending over things like language comprehension or spelling errors, and I would like to continue this tradition, even though your own paragraphs would offer wide opportunities for me to do the same.

Based on your phrasing I sense you are trying to object to something here, but it doesn't seem to have much to do with my article. Is this correct or am I just misunderstanding your point?

-4Gerald Monroe3mo
You are misunderstanding. Is English not your primary language? I think it's pretty clear. I suggest rereading the first main paragraph. The point is there, the other 2 are details.

LLMs use 1 or more inner layers, so shouldn't the proof apply to them?

what proof?

the delta for power efficiency is currently ~1000 times in favor of brains => brain: ~20 W, AGI: ~20kW, kWh in Germany: 0,33 Euro 20 kWh: ~6 Euro => running our AGI would, if we are assuming that your description of the situation is correct, cost around 6 Euros in energy per hour, which is cheaper than a human worker.

So ... while I don't assume that such estimates need to be correct or apply to an AGI (that doesn't exist yet) I don't think you are making a very convincing point so far.

2Gerald Monroe3mo
We're talking about the scenario of "the ASI wouldn't be able to afford the compute to remain in existence on stolen computers and stolen money".  There are no 20 kilowatt personal computers in existence.  Note that you cannot simply botnet them together as the activations for current neural networks require too much bandwidth between nodes for the machine to operate at useful timescales. I am assuming an ASI needs more compute and resources than merely an AGI as well.  And not linearly more, I estimate the floor between AGI -> ASI is at least 1000 times the computational resources.  This falls from how it requires logarithmically more compute for small improvements in utility in most benchmarks.   So 20 * 1000 = 20 megawatts.  So that's the technical reason.  You need large improvements in algorithmic efficiency or much more efficient and ubiquitous computers for the "escaped ASI' threat model to be valid. If you find this argument "unconvincing", please provide numerical justification.  What do you assume to be actually true?  If you believe an ASI needs linearly more compute, please provide a paper cite that demonstrates this on any AI benchmark.

I don't really know what to make of this objection, because I have never seen the stochastic parrot argument applied to a specific, limited architecture as opposed to the general category.

Edit: Maybe make a suggestion of how to rephrase to improve my argument.

Citation. Quote something somebody said.
I've never seen anything else. According to wikipedia, the term was originally applied to LLMs.

About point 1: I think you are right with that assumption, though I believe that many people repeat this argument without having really a stance on (or awareness of) brain physicalism. That's why I didn't hesitate to include it. Still, if you have a decent idea of how to improve this article for people who are sceptical of physicalism, I would like to add it.

About point 2: Yeah you might be right ... a reference to OthelloGPT would make it more convincing - I will add it later!

Edit: Still, I believe that "mashup" isn't even a strictly false characterization of concept composition. I think I might add a paragraph explicitly explaining that and how I think about it.

Isn't that a response to a completely different kind of argument? I am probably not going to discuss this here, since it seems very off-topic, but if you want I can consider putting it on my list for arguments I might discuss in this form in a future article.

Interesting insight. Sadly there isn't much to be done against the beliefs of someone who is certain that god will save us.

Maybe the following: Assuming the frame of a believer, the signs of AGI being a dangerous technology seem obvious on closer inspection. If god exists, then we should therefore assume that this is an intentional test he has placed in front of us. God has given us all the signs. God helps those who help themselves.

weakly suggested that more dimensions do reduce demon formation

This also makes a lot of sense intuitively, as it should become more difficult in higher dimensions to construct walls (hills / barriers without holes).

I am under the impression that the public attitude towards AI safety / alignment is about to change significantly.
Strategies that aim at informing parts of the public that may have been pointless in the past (abstract risks etc.) may now become more successful, because mainstream newspapers are now beginning to write about AI risks, people are beginning to be concerned. The abstract risks are becoming more concrete.

Maybe if it happens early there is a chance that it manages to become an intelligent computer virus but is not intelligent enough to further scale its capabilities or produce effective schemes likely to result in our complete destruction. I know I am grasping at straws at this point, but maybe it's not absolutely hopeless.

The result could be a corrupted infrastructure and a cultural shock strong enough for the people to burn down OpenAI's headquarters (metaphorically speaking) and AI-accelerating research to be internationally sanctioned.

In the past I have thought a lot about "early catastrophe scenarios", and while I am not convinced it seemed to me that these might be the most survivable ones.

One very problematic aspect of this view that I would like to point out is that in a sense, most 'more aligned' AGIs of otherwise equal capability level seem to be effectively 'more tied down' versions, so we should assume them to have a lower effective power level than a less aligned AGI that has a shorter list of priorities.
If we imagine both as competing players in a strategy game, it seems that the latter has to follow fewer rules.


Good addition! I even know a few of those "AI rights activists" myself.
Since this here is my first post - would it be considered bad practice to edit my post to include it?

Sorry I missed your question.  I believe it's perfectly fine to edit the post for small things like this.

I think that's not an implausible assumption.
However this could mean that some of the things I described might still be too difficult for it to pull them off successfully, so in the case of an early breakout dealing with it might be slightly less hopeless.

Even completely dumb viruses and memes have managed to propagate far. NAI could probably combine doing stuff itself and tricking/bribing/scaring people nto assist it. I suspect some crafty fellow could pull it even now, finetuning some "democratic" LLM model.

This is an important question. To what degree are both of these (naturally conflicting) goals important to you? How important is making money? How important is increasing AI-safety?

I would be the last person to dismiss the potential relevance understanding value formation and management in the human brain might have for AI alignment research, but I think there are good reasons to assume that the solutions our evolution has resulted in would be complex and not sufficiently robust.
Humans are [Mesa-Optimizers]( and the evidence is solid that as a consequence, our alignment with the implicit underlying utility function (reproductive fitness) is rather brittle (i.e. sex with contracepti... (read more)