Software engineering, parenting, cognition, meditation, other
Linkedin, Facebook, Admonymous (anonymous feedback)

Wiki Contributions

Load More


The score seems to be 100 minus the number of answers you clicked. No answers mean completely pure.

If you got 65, then the test seems to measure something real. It's not the test's fault that you think you wasted your time. 

It would be interesting to have a different test that asks about your knowledge of the Sequences. Maybe you can create one?

This is a mix of nitpicks and more in-depth comments. I just finished reading the paper and these are my notes. I liked that it was quite technical, suggesting specific changes to existing systems and precise enough to make testeable predictions.  


Some discussion about the type of consciousness applied in the paper: Dehaene's Phenomenal Consciousness And The Brain reviewed on ACX

In the introduction the verb "to token" is used two times without it being clear what that means:

a being is conscious in the access sense to the extent that it tokens access-conscious states

The authors analyze multiple criteria for conscious Global Workspace systems and based on plausibility checks synthesize the following criteria:

A system is phenomenally conscious just in case: 

(1) It contains a set of parallel processing modules. 
(2) These modules generate representations that compete for entry through an information bottleneck into a workspace module, where the outcome of this competition is influenced both by the activity of the parallel processing modules (bottom-up attention) and by the state of the workspace module (top-down attention). 
(3) The workspace maintains and manipulates these representations, including in ways that improve synchronic and diachronic coherence. 
(4) The workspace broadcasts the resulting representations back to sufficiently many of the system’s modules.

In the introduction, the authors write:

We take it to be uncontroversial that any artificial system meeting these conditions [for phenomenal consciousness] would be access conscious

Access conscious means the ability to report on perceptions. We can apply this statement in reverse as the condition that a system that isn't access conscious can't meet all of the above conditions. 

To do so, we apply the thought experiment from the paper to test this: Young babies do not yet represent their desires and while they do show some awareness, many people would agree that they are not conscious (yet) and are likely not aware of their own existence, specifically, indepenent of their caretakers. Also, late stage dementia patients lose the ability to recognize themselves, in this model resulting from the loss of the ability to represent the self-concept in GWS. This indicates that something is missing.

Indeed, in section 7, the authors discuss the consequence that their four criteria could be fulfilled by very simple systems:

As we understand them, both objections [the small model objection and another one] are motivated by the idea that there may be some further necessary condition X on consciousness that is not described by GWT. The proponent of the small model objection takes X to be what is lacked by small models which prevents them from being conscious

The authors get quite close to an additional criteria in their discussion:

it has been suggested to us that X might be the capacity to represent, or the capacity to think, or the capacity for agency [...]

[...] Peter Godfrey-Smith’s [...] emphasizes the emergence of self-models in animals. In one picture, the essence of consciousness is having a point of view

But refrain from offering one:

while we have argued that no choice of X plausibly precludes consciousness in language agents, several of the choices do help with the small model objection.

As in the thought experiements above, there are readily available examples where too simple or impaired neuronal nets fail to appear conscious and to me this suggests the following criteria:

(5) To be phenomenally conscious, the system needs to have sufficient structure or learning capabilities to represent (specific or general) observations or perceptions as concepts and determine that the concept applies to the system itself

In terms of the Smallville system discussed, this criteria may already be fulfilled by the modeling strength of GTP-3.5 and would likely be fulfilled by later LLM versions. And as required, it is not fulfilled by simple neuronal networks that can't represent self-representation. This doesn't rule out system with far fewer neurons than humans, e.g., by avoiding all the complexities of sense processing and interacting purely textually with simulated worlds.

And there I thought with minimalist you meant Lisp and with maximalist Haskell or Lean.

You may want to make this a linkpost to that paper as that can then be tagged and may be noticed more widely.

I guess "I don't want to answer the question" is a decent summary. I spent so much time on answering why because I felt that there were too many assumption in the scenario.

Yes, that would be possible, but it would lead to different results. 

In your hypothetical scenario I had to come up with very specific worlds including a lot of suffering. In the "regular" existential risk (mostly from AI), that distribution is different, thus having babies is affected differently. 

You reject the usefulness of the thought experiment, but I do not really understand why. Your reasons are that "in practice, there is almost always a possibility to affect the outcome" and that "the outcome is also almost never absolute". With respect to the possibility to affect the outcome, I would say that I, as an individual, have to take most global situations as given. 

I agree that, as an individual, one cannot affect most outcomes significantly. But if everybody assumes everybody does that too, then nobody does anything and thus definitely nothing happens/is done. Everybody contributes small parts, but those aggregate to change, because somebody will be at the right place at the right time to do something or ask the right question or bring the right people together etc. By ruling out the possibility you take this effect away and I have to price that into my model. If you or society wants to achieve something, you have to convice large numbers that change is possible and that it is important that everybody contributes. In management, that is called "building momentum." 

With respect to whether the outcome is "absolute", you seem to mean that it is not a certain outcome or that not literally everybody would die. If it is just about the certainty, well, I included the subjective probability in the thought experiment. If it is about whether everybody dies, of course you can think of any probability distribution of outcomes, but what is gained by that? 

You only added a binary probability between two options keeping both individually rigid. It would have worked better to provide distributions for number of people suffering or the effectiveness of influence etc. - but because I didn't know you intention of the though experiment I couldn't just assume those.

Then you say: "And on top of that, my presumed inability to influence outcomes somehow also doesn't influence by interest in wanting to have children." I do not really understand that sentence. Do you imply that powerful people naturally have a different amount of interest in wanting to have children? If so, why does that matter for the decision in the thought experiment?

No, I don't want to make that specific implication. Maybe powerful people have a different interest in having children, but I don't know those forces and would't make a confident prediction either way.

But if I personally can't influence results, I have to make assumptions as to why I can't. Maybe I'm sick, maybe I'm legally limited in some way in your hypothetical. Such reasons would surely influence desire to have children. 

Following lesswrong or EA community discussions about decisions about having children, I get the impression that the factors that influence the decision seem to be:

  • potentially reduced productivity (less time and energy for saving the world?),
  • immediate happiness / stress effect on the parents.


I think that there are many more reasons than this including the ecological footprint of a child, personal reasons, general ethical reasons, and others. But I agree that there is no coherent picture. The community hasn't come to terms with this and this is more a market place of EA/LW flavioured ideas. What else do you expect of a young and preparadigmatic field. People try to think hard about it, but it is, well, hard. 

I am confused about this attitude, and I try to determine whether

  • I just do not understand whether people on lesswrong expect the future to be bad or good,

More bad than good, I guess. But it is a distribution as you can look up on Metaculus.

  • people think even in case of a disaster with relevant likelihood, the future will definitely not include suffering that could outweigh some years of happiness,

Some will think it and be worried. That's what the s-risk sub-community is about, but I get the impression that is a small part. And then there is the question what suffering is and whether it is "bad" or a problem to begin with (though most agree on that).

  • people (who have children) have not thought about this in detail,

Unsurprising as, having babies has always been and always will be (until/when/if uploading/bio-engineering) a normal thing of life. Normal is normal. People do think about how many children they would want to have, but rarely if.   

  • people do not think that any of this matters for some reason I overlook,
  • people tend to be taken in by motivated reasoning,
  • or something else.

Sure, some, but I don't think it is as bad as you seem to think.

So I tried to design a clear scenario to understand some parameters driving the decisions.

And here I think things went wrong. I think the scenario wasn't good. It was unrealistic - curring out too small a part of what you seem to be interested in.

Why did I ask you about it? You have four children, you take part in discussions about the topic, you also write about alignment / AI risk.

Thank you.

Load More