Wiki Contributions


Thanks for the thoughtful response, although I'm not sure quite of the approach. For starters, 'aligning humans' takes a long time and we may simply not have time to test any proposed alignment scheme on humans if we want to avoid AGI misalignment catastrophe. Not to mention ethical issues and so forth.

Society has been working on human alignment for a very long time, and we've settled on a dual approach: (1) training the model in particular ways (e.g. parenting, schooling etc.) and (2) a very complex, likely suboptimal system of 'checks and balances' to try and mitigate issues that occur for whatever reason post-training, e.g. the criminal justice system, national and foreign intelligence, etc. This is clearly imperfect, but it seems to work a lot better to maintain cohesion than prior systems of 'mob justice' where if you didn't like someone you'd just club them in the head. Unfortunately, we now also have much more destructive weapons than the clubs of the 'mob justice' era. Nonetheless, as of today you and I are still alive and significant chunks of society more or less function and continue to do so over extended periods of time, so the equilibrium we're in could be a lot worse, but per my original post re. the nuclear button it's clear we have ended up on a knife edge.

Fortunately, we do have a powerful advantage with AGI alignment over human alignment: for the former we can inspect the internal state (weights etc.). Interpretability of the internal state is of course a challenge, but we nonetheless have access. (The reason why human alignment requires a criminal justice system like we have, with all its complexity and failings, is largely because we cannot inspect the internal state.) So it seems that AGI alignment may well be achieved through a combination of the right approach during the training phase, and a subsequent 'check' methodology via a continuous analysis of the internal state. I believe that bringing the large body of understanding/research/data we already have on human alignment in the training phase (i.e. psychology and parenting) to the AI safety researcher's table may be very helpful. And right now, I don't see this. Look for example at the open recs posted on the OpenAI web site - they are all 'nerd' jobs and no experts of human behaviour. This surprises me and I don't really understand it. If AGI alignment is as important to us as we claim, we should be more proactive in bringing experts from other disciplines into the fold when there's a reasonable argument for it, not just more and more computer scientists.

This article I think exemplifies why many people do not follow this advice. It's 39 pages long. I read it all, and enjoyed it, but most people I claim have a shorter attention span and would have either read half a page then stopped, or would have balked when they saw it was a 26-minute read and not read any of it. And those people are probably most in need of this kind of advice.

Language has a wonderful flexibility to trade off precision/verbosity against accessibility/conciseness. If you really want to rule out everything you don't mean, you inevitably need a high level of verbosity because even for the simplest arguments there are so many things you need to rule out (including many that you think are 'obvious' but are not to the people who presumably you most wish to appreciate your argument).

Which is better? An argument that rules out 80% of the things you don't mean (and so is 'misunderstood' by 20%), or an argument that's 5X as long that rules out 95% of the things you don't mean ('misunderstood' by 5%)? Most people won't read (or stick around long enough to listen/pay attention to) the second version.

So maybe you intended to mean that there's an important counterpoint, which is that there's a cost to ruling out everything you don't mean, and this cost needs to be carefully weighed when deciding where to position yourself on the precision/accessibility line when writing/speaking. There was room for this in the 39 pages, but it wasn't stated or explored, which feels like an omission.

Alternatively, if you disagree and didn't intend to mean this, you failed to rule out something quite important you didn't mean.

(Apologies if in writing this response I have missed your point.)

I don't believe that in most polyamorous relationships there are clear (i.e. fixed) priorities. I think most people will appreciate that priorities will change depending on the situation. The point I was trying to make was that this kind of 'emotional availability uncertainty' is specific to polyamorous relationships. Yes work can be a higher priority than the person in some relationships or at some times, but this is similar regardless of relationship type. The specific failure mode in polyamorous relationships that I was describing was that - even assuming all parties act in good faith and act with the best intentions - X loses the guarantee of Y's emotional availability, because Y also feels a duty to respond to Z's needs (or Y's perception of Z's needs). (Repeat for all permutations of X, Y and Z.)

In a monogamous relationship there is no need for any emotional prioritization between multiple people. This failure mode is totally absent. Yes, the other factors like work are still in there to get in the way, same as in a poly relationship.

It may be that this 'emotional availability guarantee' is not that important to some people, in which case they can achieve something asymptotically equivalent by having lots of partners in a poly relationship, and then presume that at least one of them will probably be emotionally available at any given time.

A downside of polyamorous relationships not mentioned here is that it removes guarantees of availability, which for many is an important (the most important?) value component of a long-term relationship.

For example, consider a couple X and Y. Let's say X has a bad day at work. X knows that, when they get home, Y will be there to provide emotional support. This provides benefit for X in two ways - X knows that Y will be there for support later even while the bad day is playing out, and X additionally benefits from the actual support from Y once home. Y feels happy to be there for X. End result: everyone is OK.

Or, let's suppose X is sick. They know that, if they need care, Y can be there for them. Yes, Y may have other obligations that need to be pushed aside (e.g. work) but it's generally accepted for Y to take time off work for this kind of reason.

By contrast, in a polyamorous relationship between X, Y and Z, these guarantees no longer hold. X may have a bad day at work, but maybe Z has had an even worse day (or claims to). The result is that Y feels conflicted (but ends up supporting Z over X), and X feels unsupported. End result: 2 out of the 3 people are not OK.

There will always be (at least the risk of) competition for availability in a polyamorous relationship. This is a failure mode not present in the same way in monogamous relationships.

Yes, in polyamorous relationships one can unbundle sexual attraction, intellectual attraction, long-term companionship and childrearing to some degree and thus optimize those individually. But many in a long-term monogamous relationship already feel they are close (enough) to optimal on each of those dimensions already, so would not benefit from unbundling.

but is relaxing and having "fun" necessarily better than intellectual stimulation and learning from challenges? And won't experiences like that speed up self-discovery?

I think it speeds up self-discovery, at the expense of narrowing the domain within which that self-discovery takes place. So if you spend a lot of time as a teenager developing software, you certainly learn more about yourself in terms of your aptitude for developing software. But there's an opportunity cost. I favour unguided self-discovery (a.k.a. "having fun") for longer, because I view self-discovery during teenage years as a global optimization, for which algorithms like simulated annealing tend to find better optima with a higher temperature, albeit taking longer to do so. As a result, I do not favour cutting short 'childhood' so people can be 'useful' sooner.

Also, it may be well be that the LessWrong demographic favours intellectual stimulation as "better" than many other things, but for the general population, I don't see evidence this is the case. I know plenty of highly satisfied people, not driven by intellectual stimulation but nonetheless doing things most would regard as valuable to society. But yes, this comes down to subjective philosophy on what is "better" in terms of one's own utility function, and what we should be optimising for.

The proportion of "deference to authority" is too high, in my opinion.


In school, or in the real world? And if the latter, what context in particular? In a career context, for example, lower deference to authority (when carefully executed) tends to lead to more rapid promotion, where at the terminus (CEO) everyone in an organisation defers to you. It doesn't seem there's a huge supply/demand imbalance for senior roles, which suggests to me that the self-assertiveness vs. deference balance in working-age society is more or less optimal.

what society currently lacks is a path to do something useful with it from a younger age.

Agreed, but why should teenagers being 'useful' be a goal? A century ago, most teenagers did actually do useful things (work in factories etc.) but we've moved away from that these days. Being a teenager is fun, with low responsibility, a lot of free time for self-discovery, etc. We have a lifetime after that to be 'useful'. Why should we cut our young years short?

What were the principal factors that led to your decision that homeschooling and early graduation was 'better' for your kids then a 'conventional' schooling approach/timetable?

Clearly entering the workforce earlier leads to financial independence sooner, more years in employment hence greater lifetime wealth accumulation, etc. It's not clear that these things are that important either to individual well-being and happiness or in terms of one's place in broader society, so I'm interested in other kinds of reasons.

Full disclosure: I am 'a priori' against homeschooling in most cases. My belief is the principal value of schooling is to provide maximal 'contact time' with a highly heterogenous group of people, to enable acquisition of the kinds of skills that are important for societal cohesiveness in a heterogenous world. Substituting at least a significant part of this contact time for time at home (or time with a more homogeneous group i.e. other homeschooled children and their parents) reduces the size of the 'soft skills' training data set for teenagers at an important time in their lives. I'm not sure if this is a good thing, even if it leads to on-paper 'adult credentials' at age 18.

There's an inplicit assumption, both in the post and in many of the comments, that the 'value' of school for teenagers lies in knowledge acquisition. And therefore, if school is busywork a.k.a. does not lead to acquisition of useful knowledge, homeschooling or 'alternative' schooling must be better. I think this is wrong.

For society, the principal lasting value of schooling teenagers relates to the acquisition of skills like:

  • dynamic balancing of self-assertiveness vs. deference to authority (both on an interpersonal level, and also on a societal level)
  • productively handling highly heterogeneous interpersonal environments e.g. where people have a broad mix of skill levels, temperaments, abilities, interests, etc.
  • learning to handle common but undesirable traits in others e.g. aggressiveness/bullying, jealousy, etc.
  • balancing behavioural risk/reward factors (e.g. via 'playing up' and assessing the reactions of others)

These things are all important in the real world to ensure effective function of society. They emerge as a result of contact time, and whatever other things also happen during that time (e.g. learning synthesis paths for ammonia, busywork etc.) don't really matter that much. 'Alternative' schools where there's more homogeneity (everyone is super bright, 'finds school boring' etc.) deprive teenagers of time spent in 'normal' (heterogenous) society - at a time when such exposure has the most impact - and thus compromises acquisition of the skills that help thrive in the real world.

Knowledge acquisition, on the other hand, can be done via Wikipedia etc. and does not need to occupy school time. People who want to acquire knowledge can do this easily in their bedroom at night. Soft skills cannot be learned this way. 

There's a lot of comments here from people saying things like "I dropped out of school / did not attend regular school, became a software engineer and that was a great decision". Great for who? I've worked with many software engineers and I can say that "software engineer" as a profession is at least somewhat correlated with a lack of the soft skills that others do gain from teenage schooling. So it isn't surprising, then, that "software engineer" is also correlated with "didn't get much out of school". But the direction of the arrow of causality isn't clear. And having more teenagers spending more time on software engineering (and hence less time on acquiring soft society skills via extended time in heterogenous interpersonal environments i.e. 'normal schools') may be great for the teenagers if they're not interested in society, or for GDP, but I don't see any indication it's a net positive for the world as a whole.