I think that too much scafolding can obfuscate a lack of general capability, since it allows the system to simulate a much more capable agent - under narrow circumstances and assuming nothing unexpected happens.

Consider the Egyptian Army in '73. With exhaustive drill and scripting of unit movements, they were able to simulate the capabilities of an army with a competent officer corps, up until they ran out of script, upon which it reverted to a lower level of capability. This is because scripting avoids officers on the ground needing to make complex tactical decisions on the fly and communicate them to other units, all while maintaining a cohesive battle plan. If everyone sticks to the script, big holes won't open up in their defenses, and the movements of each unit will be covered by that of others. When the script ran out (I'm massively simplifying), the cohesion of the army began to break down, rendering it increasingly vulnerable to IDF counterattacks. The gains in combat effectiveness were real, but limited to the confines of the script. 

Similarly, scafolding helps the AI avoid the really hard parts of a job, at least the really hard parts for it. Designing the script for each individual task and subtask in order to make a 90% reliable AI economically valuable turns a productivity-improving tool into an economically productive agent, but only within certain parameters, and each time you encounter a new task, more scafolding will need to be built. I think some of the time the harder (in the human-intuitive sense) parts of the problem may be contained in the scafolding as opposed to the tasks the AI completes.

Thus, given the highly variable nature of LLM intelligence, "X can do Y with enough scafolding!" doesn't automatically convince me that X possesses the core capabilities to do Y and just needs a little encouragement or w/e. If may be that task Y is composed of subtasks A and B, such that X is very good and reliable at A, but utterly incapable at B (coding and debugging?). By filtering for Y with a certain easy subset of B, using a pipeline to break it down into easier subtasks with various prompts, trying many times, and finally passing off unsolved cases to humans, you can extract much economic from X doing Y, but only in a certain subset of cases, and still without X being reliably good at doing both A and B. 

You could probably do something similar with low-capability human programmers playing the role of X, but it wouldn't be economical since they cost much more than an LLM and are in some ways less predictable. 

I think a lot of economically valuable intelligence is in the ability to build the scafolding itself implicitly, which many people would call "agency". 

I'd give my right eye in exchange for the chance to live in NYC.

We plan to advance capabilities as fast as possible while making sure our safety always remains ahead.

This is the galaxy-brained plan of literally every single AI safety company of note. 

Then again, maybe only the capabilities focused ones become noteworthy. 

Electricity is ~$0.1 per kWh for industry in Texas. Running a H100 for a full year costs ~$1800 at that price. I'm not sure how much depreciation is for an H100, but 20% seems reasonable. If a H100 is $40,000, and a year of depreciation is $8000, then you would be losing $800 a year if you just had 10% idle time. 

So... maybe? But my guess is that natural gas power plants are just cheaper and more reliable - a few cloudy weeks out of a year would easily shift the equation in favor of natural gas. No idea how power cycling affects depreciation. The AI industry people aren't talking much about solar or wind, and they would be if they thought it was more cost effective. 

I don't think there will actually be an electricity shortage due to AI training - I just think industry is lobbying state and federal lawmakers very very hard to make these power plants ASAP. I think it is quite likely that various regulations will be cut though and those natural gas plants will go up faster than the 2-3 years figure I gave.

Are people actually working on human enhancement? Many talk about how it's the best chance humanity has, but I see zero visible efforts other than Neurolink. No one's even seriously trying to clone Von Neumann!

That's a pretty good point - I was mostly thinking from my position, which was the rapid end of lockdowns in China in late 2022. I had a ~100% chance of catching COVID within a month, and had no idea how prepared the hospitals were in regards to that. I ended up with a really bad cough and secondary lung infection that made me consider going to the hospital, so I guess I made the right choice? If I did develop pneumonia, I would have looked foward to a less crowded hospital with more amenities, which would have been nice. My decision was also based on the possibility of "this may be worse than expected and the hospitals will somehow fuck it up", which thankfully didn't take place.

But yeah, I didn't consider the consequences pozing yourself in 2020/2021. I agree with you that it's a bad idea in hindsight for almost everyone. I was one of the few people who could have benefited from it in theory, and all I really got was internet bragging rights.

Still very curious about those who did variolate themselves and their reasoning behind it.

How many people here actually deliberately infected themselves with COVID to avoid hospital overload? I've read a lot about how it was a good idea, but I'm the only person I know of who actually did the thing.

I'll bite the bullet for this one and suport vegan nutraloafs for all prisoners. Prison labor is a valuable source of restitution and rehabilitation, and tying it to non-terrible food in canteens seems like a good ideas.

Can you give examples of what you're looking for? Can I email you entries and expect a response?

I can confirm that my PayPal has received the $500, although it'll be frozen for a while.

Thanks! I had a lot of fun doing the research for this and I'm working on an update that'll be out in a few days.