All of Cervera's Comments + Replies

"We'll be building a cluster of around 22,000 H100s. This is approximately three times more compute than what was used to train all of GPT4. 


This bothers me. It's a naive way of seeing compute. It's like confusing Watts and Watt-hours 

22,000 H100s is three times the amount of FLOP/s than what was used to GPT-4, so you could train it in 3x less time, of with 1/3 of your cluster and the same time. 

I think this view of looking at compute helps making naive asumptions about what this compute can be used to. And FLOP/s are not a perfect unit for normal discourse when we're at x10¹⁵ scales. 

If ancestor is parent/mother/grandparent etc but nothing else. Obviously non hunters.

If we count how many people dead or alive are you related to. Farmers.

Why? This is extremely non-obvious to me. There are ~290k years between 300k years ago and the approximate start of the agricultural revolution.

This gap will only widen over time; China is failing to develop a domestic semiconductor industry, despite massive efforts to do so, and is increasingly cut off from international semiconductor supply chains.


I would say this is a falsehood 

The US export ban on Controlled GPUs has really made china push for local semiconductor manufacturing way, and accelerate their projects, they dont have 5nm TSMC quality wafers, fine, but they're developing the full stack. 

tphuang on Twitter: "Semiconductor export from Korea down 41% in April Falling SSD ... (read more)

I dont think the difficulty of the task has much with the outcome. 


I mean, I take your comment at face value and update to "it's going to get powerful faster" and not the other way around.

A "moonshot idea" I saw brought up is getting Yudkowsky's Harry Potter fanfiction translated into Chinese (please never ever do this).


Can you expand on this? Why would it be a bad idea? I have interacted with mainland chinese people (outside of china) and I'm not really making the connection. 

Let's just say that weirdness in China is very different from weirdness in the West. AI safety isn't even a weird concept here. It's something people talk about, briefly think over, then mostly forget, like Peter Thiel's new book. People are generally receptive to it. What AI safety needs to get traction in the Chinese idea sphere is to rapidly disassociate with really really weird ideas like EA. EA is like trying to shove a square peg into the round hole of Chinese psychology. It's a really bad sign that the AI Safety toehold in China is clustered around ... (read more)

How do you think discharge rates would affect the battery? Would it behave like a LFP that basically outputs mostly the same rate of Amps in the full spectrum of charge (Voltage varies little with discharge %)  

Do you think an approach like this generates bateries with long lifetimes? 

Did you expect balancing of these particular batteries to be particularly complicated? 

Basically, tell us more! 

Li-ion batteries have solid particles that Li ions migrate into and out of. This can cause particles to break up, especially at high charge/discharge rates. Because there are fewer ions to migrate, fast discharge at low charge is bad for battery lifetime and gives lower voltage. SMAC batteries have solid particles that dissolve and form as the battery is operated. It doesn't matter if those break up. To some extent, the maximum discharge rate would decrease as some smaller particles disappear during discharge. There's also some Ostwald ripening that happens, which decreases discharge rate a bit over time, until the next charge cycle, but the extent is limited. Li-ion batteries are limited largely by SEI growth from electrolyte-Li reaction. Charging and discharging accelerates SEI growth because it causes cracking in the existing SEI, especially at high rates. SMAC battery lifetime would probably be limited by water migration, with charge cycles being irrelevant and only time & temperature being important, but the long-term lifetime isn't clear at this point. Yes, there is a SEI in SMAC batteries, but it's a thin SEI that works for Na but not Li, with less surface area, so it wouldn't cause much capacity loss. The relative charge rate of Li-ion vs SMAC depends on the thickness of the electrolyte layers, which depends on the manufacturing process rather than the chemistry. The experimental data I got doesn't really indicate this because an insulating oxide layer was forming, and because the test cells used much thicker layers than commercial cells would. But I'd expect it to be similar, meaning max charge rates between 0.1C and 10C. I'm not sure why balancing would be different.

Well, Evals and that stuff OpenAI did with predicting loss could be a starting point to work in the tables. 

But we dont really know, I guess that's the point EY is trying to make. 

I was hoping that he meant some concrete examples but did not elaborate on this due this being letter in magazine and not a blog post. The only thing that comes to my mind in somehow measure unexpected behavior and if bridge some times lead people in circles then it will be definitely cause for concern and reevaluation of used technics.

Use tables for concrete loads and compare experimentally with the to be poured concrete, if a load its off, reject it.

We dont even have the tables about ML. Start making tables, dont build big bridges until you got the fucking tables right.

Enforce bridge making no larger than the Yudkowski Airstrike Threshold.

Do we have an idea of how this tables about ML should look like? I dont know about ML that much.

I appreciate the concept. 

I wonder about how the hardware overhangs would look like after the moratorium ends or somebody bypasses it.

On first thought it doesnt look like a robuts solution, I assume a 15-20% improvement on Compute access per year. Would need to plot it against the moratorium treshold and see if over time one gets closer to the Yudkowski Airstrike treshold  but I assume no, 20% vs 54%,,

I dont know! maybe this is a good idea. 

3Cleo Nardo8mo
I've added a section on hardware:

I dont think I never learned something because it would make me a better worker / provide me with more economical resources when I was a child and was in need of tutoring, I got lucky to have a somewhat curious mind and I tried to saciate it. 


Of course as an adult I choose to do things that are useful, overall, and that normally repercutes on being a human with skills that other people pay for. But the explicit bayesian calculation about knowledge and money is not one I tend to do, what interest me interest me. Of course when trying to learn som... (read more)

6Roman Leventov9mo
This is not the case. Maybe your memory capacity is naturally on the upper side of the human range (e.g., 9 pieces rather than 4) as well as IQ, which makes learning for you seem doable. The fact is, most people, no matter how hard they try, are probably incapable of learning calculus, let alone tensor algebra or something even more abstract or complex, such as the math that is needed even to begin to approach string theory. Or, something that requires keeping simultaneous track of many moving pieces. For example, it's considered that no human can properly understand how the brain works: it requires simultaneously holding in one's head dozens or hundreds of moving pieces. AI can do this, but a human can't. I saw this mistake made by David Deutsch: because he himself is a genius and can relatively easily understand anything that any other human can write, he conjectured that the "human understanding has universal reach". Stephen Wolfram, another genius, concurred.

So first thoughts while reading the research/Gpt-4 page

- ChatGPT System Prompt open soon to users, not API holders, that's going to be interesting. 

-Only trained adversarialy with 50 experts? One would think you would spend a bit more and do it with 100x more people? Or if hard to coordinate, at least 500. 

Closing in on human performance, but I would love to see numbers on the compute needed to train. They can predict loss accurately, but what that loss means seems to be mostly an open problem. 

Will need a couple days and rereads to digest this. 

I did finish it, and was going to edit the original comment, I get the purpose of what you did here, thanks for the post, it's an interesting read. 

I plan on reading the whole post, but a thing struck with me on the first paragraph. 


it's not a "her", it's a "it"

Happened yesterday too in Spanish in another forum, using feminine pronouns to refer to ChatGPT (Terrible name) I guess it's more "normal" in my native language because Intelligence has a feminine gender on my language. But seeing it in English really makes me notice. 

I think nowadays "they" is more fitting.
Yes, I used to be exactly like you :) You should definitely read the whole post to understand why I refer to her this way. This is a deliberate choice reflecting how I feel about her. I start with "it" in the first sections, very reluctantly, and then switch to the personal pronoun as the story unfolds.

I was trying to imply that the flow of water goes in a single direction. And that it's a quite irrational fear to be sprayed by others fecal matter in this case. 


I dont like the japanese style ones, at my home we have a hand mini showerhead that we direct ourselves. We clean it periodically and it rarely touch anybody's skin. All the muslim world has that. 

Would you drink water from a kitchen sink? 


In my country we have traditional bidets, they're another porcelain furniture that sits near the toilet, so you suffle to it to clean yourself. Usually with warm water and soap. 

Not all homes have it but just using paper feels like savagery, also, access to a full sized bidet like the one I describe allows you to clean yourself without showering, like, all the important parts. 

I frequently drink water from kitchen sinks, yes.
I have seen such porcelain bidets also in Germany. When I was a kid, I assumed they were footbaths, or baths for babies. 

I dont think I wrote that statement with that particular intention in mind. 

I'm not trying to imply he is wrong because he doenst know our "groupthink" I was just generally annoyed at how he started the post, so i wanted to be reasonably civil, but a bit mean. 

Thanks for noticing, I'm not convinced I should have refrained from that particular comment tho.

What would you have said? 

>What would you have said?  Your comment is stronger without this sentence.
I would suggest to remove "I dont think you are calibrated properly about the ideas that are most commonly shared in the LW community. " and present your argument, without speaking for the whole community. 

Hey, Interesting post. 

Artificial General Inteligence has nothing to do with simulating brains. 

The approaches are different, the math formulares are different, We're slowly moving to sparcity for some things (wich is similar to how a brain works) but still. 

I dont think you are calibrated properly about the ideas that are most commonly shared in the LW community. 

Nobody is saying "we will get a so good brain simulator that will kill us" That's not the point. 

The point is that we can create agents in other ways, and those Agents can still kills us, no brain simulation included. 

AGI doesn't necessarily have anything to do with simulating brains, but it would count if you could do it.
>I dont think you are calibrated properly about the ideas that are most commonly shared in the LW community.  This is chastising him for failure to abide by groupthink. The rest of your comment makes a point that is undermined by this statement.
We address this argument. AGI has a lot to do with simulating brains in our opinion, since an agent of similar or higher complexity has to be created. There can be no shortcut, in our opinion. A deep learning network with 10^7 nodes will not outperform a brain with 10^11 neurons, especially if each neuron is highly complex. We are not arguing that a brain simulation will/will not take over, but that an agent which could would have to use a similar amount of energy, or even several orders below. And that's unrealistic.

Thanks for the content. 

I woke up today wondering about fallout. Wich modern, mostly fusion weapons, it surely cannot be that bad. 

The amount of grams of plutonium used before detonating the fusion reaction should be too little to generate the mass histery about fall out, this is not the sixties. 


Epistemic status: I only have hunches and have to confirm what I just wrote with facts. 

4Martin Fell1y
For what it's worth, most modern fusion bombs actually generate most (e.g. 80%+) of their "yield" from fission - the fusion stage is surrounded by a layer of uranium which is bombarded by neutrons produced in the fusion reaction, causing fission in the uranium and magnifying the yield. So they are pretty dirty weapons. They are at least smaller than the weapons from the 50s and 60s though.
4Lao Mein1y
Fusion bombs don't produce much fallout from airbursts - the worry of fallout is the large-scale use of ground-burst bombs designed to target hardened facilities like bunkers and missile silos. Dirt would then absorb much of the resulting radiation, become radioactive, and get carried by wind. And you're welcome for reading :^).

I think there is a discussion to be had about if a country that has countless military bases of another country in their land is occupied of just an "ally" It's not clear to me there is a big difference in practical terms. 

Not every occupation is the same, but nations occupied by military force are often denied the ability to run their own affairs with regard to legal proceedings, defence, etc. In particular not being allowed to have final authority over legal matters on their own soil seems to historically be a great sticking point: see the Austro-Hungarian demands of Serbia leading to WW1.  This is one of the key domains which defines the authority of a sovereign nation, whereas it doesn't seem that uncommon in history for there to be foreign military assets in a nation as a non-occupying force that does not damage the sovereignty of that nation. Auxiliary troops, mercenaries, allied soldiers.  From this perspective, U.S. bases look like occupation insofar as they damage the sovereignty of the occupied nation, and look like anything but occupation to the degree that they protect or abide by that sovereignty. Russian propaganda would of course claim, that the former dramatically outweighs the latter.
I don't think US is occupying Japan, because US bases in Japan are there by mutual consent. Similarly for South Korea and Germany. But yes, US is occupying Okinawa. US bases in Okinawa are not there by mutual consent. Japan consents, Okinawa doesn't. Marine Corps Air Station Futenma is, like, the single biggest political issue in Okinawa. Just look at the aerial picture. It is absurd. Similarly, US was occupying Seoul, Yongsan Garrison in particular. After a long negotiation, US bases relocated to Pyeongtaek, where they are welcome. Putin does have a point here. US should do more of Pyeongtaek.

How could one end up taking the correct medication without knowing first that they're lactose intolerant? 

I don't see how that could happen. To end up taking lactose pills you need to know there is something wrong with your lactase tolerance in particular. 

I don't see how Claiming hate speech changes anything about the underlying ideas. 

I have done fasts before, but I did not partake in this year Vavilov Day, because, well, my birthday happens to fall in one of those days. 

Thinking about it afterwards, I should have. And I think this could be a positive thing to do for most people. Let me expand on that thought. 

  • First, Most people have never been in a state where their body does not have food in their stomach, and the sets of feelings that come with it are interesting and thought provoking at the least, and can be life changing at best. most humans historically have fasted, volu
... (read more)