Yes, I think this was irrational to not clean up the glass. That is the point I want to make. I don't think it is virtual to have failed in this way at all. What I want to say is: "Look I am running into failure modes because I want to work so much."

Ah! I completely missed that, that changes my interpretation significantly. Thank you for the clarification, now I'm less worried for you since it no longer sounds like you have a blindspot around it.

Not running into these failure modes is important, but these failure modes where you are working too much are much easier to handle than the failure mode of "I can't get myself to put in at least 50 hours of work per week consistently."

While I do think that it is true, I am probably very bad in general at optimizing for myself to be happy. But the thing is while I was working so hard during AISC I was most of the time very happy. The same when I made these games. Most of the time I did these things because I deeply wanted to.

It sounds right that these failure modes are easier to handle than the failure mode of not being able to do much work. 

Though working too much can lead to the failure mode of "I can't get myself put in work consistently". I'd be cautious in that it's possible to feel like you really enjoy your work... and then burn out anyway! I've heard several people report this happening to them. The way I model that is something like... there are some parts of the person that are obsessed with the work, and become really happy about being able to completely focus on the obsession. But meanwhile, that single-minded focus can lead to the person's other needs not being met, and eventually those unmet needs add up and cause a collapse.

I don't know how much you need to be worried about that, but it's at least good to be aware of.

Seems like a rational prioritization to me if they were in an important moment of thought and didn't want to disrupt it. (Noting of course that 'walking on it' was not intentional and was caused by forgetting it was there.)

This sounds like you're saying that they made a rational prioritization and then, separately from that, forgot that it was there. But those two events are not separate: the forgetting-and-then-walking-on-it was a predictable consequence of the earlier decision to ignore it and instead focus on work. I think if you model the first decision as a decision to continue working and to also take on a significant risk of hurting your feet, it doesn't seem so obviously rational anymore. (Of course it could be that the thought in question was just so important that it was worth the risk. But that seems unlikely to me.)

As the OP says, a "normal person might stop and remove all the glass splinters". Most people, in thinking whether to continue working or whether to clean up the splinters, wouldn't need to explicitly consider the possibility that they might forget about the splinters and step on them later. This would be incorporated into the decision-making process implicitly and automatically, by the presence of splinters making them feel uneasy until they were cleaned up. The fact that this didn't happen suggests that the OP might also ignore other signals relevant to their well-being. 

The fact that the OP seems to consider this event a virtue to highlight in the title of their post, is also a sign that they are systematically undervaluing their own well-being in a way that to me seems very worrying.

Also, I would feel pretty bad if someone wrote a comment like this after I posted something. (Maybe it would have been better as a PM)

Probably most people would. But I think it's also really important for there to be clear, public signals that the community wants people to take their well-being seriously and doesn't endorse people hurting themselves "for the sake of the cause". 

The EA and rationalist communities are infamous for having lots of people burning themselves out through extreme self-sacrifice. If someone makes a post where they present the act of working until their feet start bleeding as a personal virtue, and there's no public pushback to that, then that sends the implicit signal that the community endorses that reasoning. That will then contribute to unhealthy social norms that cause people to burn themselves out. The only way to counteract that is by public comments that make it clear that the community wants people to take care of themselves, even if that makes them (temporarily) less effective.

To the OP: please prioritize your well-being first. Self-preservation is one of the instrumental convergent drives; you can only continue to work if you are in good shape.

This seems similar to the thing in Why Artists Study Anatomy and Drawing Less Wrong, that in order to produce art that accurately reproduces an aspect of reality, you have to actually learn to pay attention to reality. And by learning to produce art, you start noticing all the ways in which your brain usually filters out reality.

I think this generalizes. E.g. to write good dialogue you need to pay attention to how people actually talk and what makes for an interesting conversation; to learn to dance you need to start paying attention to your body, etc.

I think this is true of an environmentalist movement that wants there to be a healthy environment for humans; I'm not sure this is true of an environmentalist movement whose main goal is to dismantle capitalism. 

FWIW, the environmentalist movement that I'm most familiar with from Finland (which is somewhat partisan but much less so than the US one) is neither of these. There's some element of "wants there to be a healthy environment for humans" but mostly it's "wants to preserve the environment for its own sake". 

E.g. ecosystems being devastated is clearly depicted as being intrinsically bad, regardless of its effect on humans. When "this is how humans would be affected" arguments are brought in, they feel like they're being used as a motte.

EDIT: I guess climate change stuff is much more human-focused; it being so big is a more recent development, so I didn't happen to think of it when considering my prototypical sense of "environmentalism". (It also feels like a more general concern, with "environmentalism" connoting a more narrowly-held concern to me.)

Dang. I did post the chapter online, and linked to it from e.g. this post summarizing it.

I really like this post for being an appropriately nuanced look at both how to strive for honesty and also reasons not to.

I've seen some takes on the topic that seem to arise from either a deep discomfort of ever having to deal with anything at all misleading, or just from a straight-out desire to minimize social effort and use honesty as an excuse to blurt out whatever is on your mind. This post is very much not that, and rather goes into detail on how being deeply honest actually requires putting in more work to do it well.

No one (to my knowledge?) highlighted that the future might well go as follows:
“There’ll be gradual progress on increasingly helpful AI tools. Companies will roll these out for profit and connect them to the internet. There’ll be discussions about how these systems will eventually become dangerous, and safety-concerned groups might even set up testing protocols (“safety evals”). Still, it’ll be challenging to build regulatory or political mechanisms around these safety protocols so that, when they sound the alarm at a specific lab that the systems are becoming seriously dangerous, this will successfully trigger a slowdown and change the model release culture from ‘release by default’ to one where new models are air-gapped and where

Hmm, I feel like I always had something like this as one of my default scenarios. Though it would of course have been missing some key details such as the bit about model release culture, since that requires the concept of widely applicable pre-trained models that are released the way they are today. 

E.g. Sotala & Yampolskiy 2015 and Sotala 2018 both discussed there being financial incentives to deploy increasingly sophisticated narrow-AI systems until they finally crossed the point of becoming AGI.

S&Y 2015:

Ever since the Industrial Revolution, society has become increasingly automated. Brynjolfsson [60] argue that the current high unemployment rate in the United States is partially due to rapid advances in information technology, which has made it possible to replace human workers with computers faster than human workers can be trained in jobs that computers cannot yet perform. Vending machines are replacing shop attendants, automated discovery programs which locate relevant legal documents are replacing lawyers and legal aides, and automated virtual assistants are replacing customer service representatives.

Labor is becoming automated for reasons of cost, efficiency and quality. Once a machine becomes capable of performing a task as well as (or almost as well as) a human, the cost of purchasing and maintaining it may be less than the cost of having a salaried human perform the same task. In many cases, machines are also capable of doing the same job faster, for longer periods and with fewer errors. In addition to replacing workers entirely, machines may also take over aspects of jobs that were once the sole domain of highly trained professionals, making the job easier to perform by less-skilled employees [298].

If workers can be affordably replaced by developing more sophisticated AI, there is a strong economic incentive to do so. This is already happening with narrow AI, which often requires major modifications or even a complete redesign in order to be adapted for new tasks. ‘A roadmap for US robotics’ [154] calls for major investments into automation, citing the potential for considerable improvements in the fields of manufacturing, logistics, health care and services.

Similarly, the US Air Force Chief Scientistʼs [78] ‘Technology horizons’ report mentions ‘increased use of autonomy and autonomous systems’ as a key area of research to focus on in the next decade, and also notes that reducing the need for manpower provides the greatest potential for cutting costs. In 2000, the US Congress instructed the armed forces to have one third of their deep strike force aircraft be unmanned by 2010, and one third of their ground combat vehicles be unmanned by 2015 [4].

To the extent that an AGI could learn to do many kinds of tasks—or even any kind of task—without needing an extensive re-engineering effort, the AGI could make the replacement of humans by machines much cheaper and more profitable. As more tasks become automated, the bottlenecks for further automation will require adaptability and flexibility that narrow-AI systems are incapable of. These will then make up an increasing portion of the economy, further strengthening the incentive to develop AGI. Increasingly sophisticated AI may eventually lead to AGI, possibly within the next several decades [39, 200].

Eventually it will make economic sense to automate all or nearly all jobs [130, 136, 289].

And with regard to the difficulty of regulating them, S&Y 2015 mentioned that:

... there is no clear way to define what counts as dangerous AGI. Goertzel [115] point out that there is no clear division between narrow AI and AGI and attempts to establish such criteria have failed. They argue that since AGI has a nebulous definition, obvious wide-ranging economic benefits and potentially significant penetration into multiple industry sectors, it is unlikely to be regulated due to speculative long-term risks.

and in the context of discussing AI boxing and oracles, argued that both AI boxing and Oracle AI are likely to be of limited (though possibly still some) value, since there's an incentive to just keep deploying all AI in the real world as soon as it's developed:

Oracles are likely to be released. As with a boxed AGI, there are many factors that would tempt the owners of an Oracle AI to transform it to an autonomously acting agent. Such an AGI would be far more effective in furthering its goals, but also far more dangerous.

Current narrow-AI technology includes HFT algorithms, which make trading decisions within fractions of a second, far too fast to keep humans in the loop. HFT seeks to make a very short-term profit, but even traders looking for a longer-term investment benefit from being faster than their competitors. Market prices are also very effective at incorporating various sources of knowledge [135]. As a consequence, a trading algorithmʼs performance might be improved both by making it faster and by making it more capable of integrating various sources of knowledge. Most advances toward general AGI will likely be quickly taken advantage of in the financial markets, with little opportunity for a human to vet all the decisions. Oracle AIs are unlikely to remain as pure oracles for long.

Similarly, Wallach [283] discuss the topic of autonomous robotic weaponry and note that the US military is seeking to eventually transition to a state where the human operators of robot weapons are ‘on the loop’ rather than ‘in the loop’. In other words, whereas a human was previously required to explicitly give the order before a robot was allowed to initiate possibly lethal activity, in the future humans are meant to merely supervise the robotʼs actions and interfere if something goes wrong.

Human Rights Watch [90] reports on a number of military systems which are becoming increasingly autonomous, with the human oversight for automatic weapons defense systems—designed to detect and shoot down incoming missiles and rockets—already being limited to accepting or overriding the computerʼs plan of action in a matter of seconds. Although these systems are better described as automatic, carrying out pre-programmed sequences of actions in a structured environment, than autonomous, they are a good demonstration of a situation where rapid decisions are needed and the extent of human oversight is limited. A number of militaries are considering the future use of more autonomous weapons.

In general, any broad domain involving high stakes, adversarial decision making and a need to act rapidly is likely to become increasingly dominated by autonomous systems. The extent to which the systems will need general intelligence will depend on the domain, but domains such as corporate management, fraud detection and warfare could plausibly make use of all the intelligence they can get. If oneʼs opponents in the domain are also using increasingly autonomous AI/AGI, there will be an arms race where one might have little choice but to give increasing amounts of control to AI/AGI systems.

I also have a distinct memory of writing comments saying something "why does anyone bother with 'the AI could escape the box' type arguments, when the fact that financial incentives would make the release of those AIs inevitable anyway makes the whole argument irrelevant", but I don't remember whether it was on LW, FB or Twitter and none of those platforms has a good way of searching my old comments. But at least Sotala 2018 had an explicit graph showing the whole AI boxing thing as just one way by which the AI could escape, that was irrelevant if it was released otherwise:

Nice post! I like the ladder metaphor.

For events, one saving grace is that many people actively dislike events getting too large and having too many people, and start to long for the smaller cozier version at that point. So instead of the bigger event competing with the smaller one and drawing people away from it, it might actually work the other way around, with the smaller event being that one that "steals" people from the bigger one.

Previous LW discussion about the Inner Ring: [1, 2].

Good question! I would find it plausible that it would have changed, except maybe if the people you'd call would be in their fifties or older.