LESSWRONG
LW

I'm brand new to LW, it's refreshing to be able to discuss things intelligently, I haven't come across many places on the internet that happens. I'm excited to hear all your references to some of my favorite, inspiring, interesting people. I stumbled on the site while trying to Google a scarcely answered question, and I don't even remember what it was because I went down the rabbit hole, reading post after post on so many different subjects. I was even recently discussing the AI winning Multiplayer poker with someone, talking about what it meant for the future of AI (exponential learning) and the problems it could potentially solve in x amount of years.

8habryka6y

Welcome! :)

[-]ryan_b6y170

I've been considering another run at Anki or similar, because I simultaneously found a new segment of a field to learn about and also because I am going to have to pivot my technical learning at work soon.

Reading Michael Nielson's essay on the subject, he makes frequent references to Feynman; I am wondering about the utility of using Anki to remember complex problems in better detail. The motivation is the famous story about Feynman where he always kept a bunch of his favorite open problems in mind, and then whenever he encountered a new technique he would test it against each problem. In this way, allegedly, he made several breakthroughs regarded as brilliant.

It feels to me like the point might be more broad and fundamental than mathematical techniques; I suspect if I could better articulate and memorize an important problem, I could make it more a part of my perspective rather than something I periodically take a crack at. If I can accomplish this, I expect I will be more likely to even notice relevant information in the first place.

4Matt Goldenberg6y

I haven't thought about doing this with open problems, but I really like the idea and feel like I've done it implicitly with a number of problems important to me (coordination issues, moloch, etc). I do however, do this explicitily when I learn new solutions, making sure I integrate them into my world model, can instinctively frame problems using those models, and tie them to other potential solutions. It feels like beginning to explicitly to do this with problems is a large next step that could really take effectiveness to the next level. Thanks!

3ryan_b6y

Short mashup from two sources: Nielson proposes an informal model of mastery: where the chunks in question fit into the 7+/-2 of working memory. Relatedly, there is Alan Kay's quip: Which is to say the new perspective provides a better way to chunk complex information. In retrospect this feels obvious, but my model for multiple perspectives beforehand mostly a matter of eliminating blind spots they might have. I'll have to integrate the contains-better-chunks possibility, which basically means that seeking out new perspectives is more valuable than I previously thought.

[-]brunoparga6y140

Hi, I'm Bruno from Brazil. I have been involved with stuff in the Lesswrongosphere since 2016. While I was in the US, I participated in the New Hampshire and Boston LW meetup groups, with occasional presence in SSC and EA meetups. I volunteered at EAG Boston 2017 and attended EAG London later that year. I did the CFAR workshop of February 2017 and hung out at the subsequent alumni reunion. After having to move back to Brazil I joined the São Paulo LW and EA groups and tried, unsuccessfully, to host a book club to read RAZ over the course of 2018. (We made it as far as mid-February, I think.)

I became convinced of the need to sort out the AI alignment problem after first reading RAZ. I knew I needed to level up on lots of basic subjects before I could venture into doing AI safety research. Because doing so could also have instrumental value to my goal of leaving Brazil for good, I studied at a Web development bootcamp and have been teaching there for a year now; I feel this has given me the confidence to acquire new tech skills.

I intend to start posting here in order to clarify my ideas, solve my confusion and eventually join the ranks of the AI safety researchers. My more immediate goal is to be able to live somewhere other than Brazil while doing some sort of relevant work (even if it is just self-study or something not directly related to AI safety that still allows me to study on the side, like my current gig here does).

4Alexei6y

Sounds great. Welcome!

[-]Zack_M_Davis6y120

In order to combat publication bias, I should probably tell the Open Thread about a post idea that I started drafting tonight but can't finish because it looks like my idea was wrong. Working title: "Information Theory Against Politeness." I had drafted this much—

Suppose the Quality of a Blog Post is an integer between 0 and 15 inclusive, and furthermore that the Quality of Posts is uniformly distributed. Commenters can roughly assess the Quality of a Post (with some error in either direction) and express their assessment in the form of a Comment, which is also an integer between 0 and 15 inclusive. If the True Quality of a post is $i$ , then the assessment $a$ expressed in a Comment on that Post follows the probability distribution
$P (a | i) = ⎧ ⎨ ⎩ \begin{matrix} 1 / 3 & a = i - 1 mod 16 1 / 3 & a = i 1 / 3 & a = i + 1 mod 16 \end{matrix}$
(Notice the "wraparound" between 15 and 0: it can be hard for a humble Commenter to tell the difference between brilliance-beyond-their-ken, and utter madness!)
The entropy of the Quality distribution is ${log}_{2} 16$ = 4 bits: in order to inform someone about the Quality of a Post, you need to transmit 4 bits of information. Comments can be thought of as a noisy "channel" conveying information about the post.
The mutual inf

... (read more)

8jessicata6y

It seems to me clear that, if Commenters have a policy of reporting the maximum of their perception and 10, then there is now less mutual information between the commenter's report and the actual post quality than there was previously. In particular, you now can't distinguish between a post of quality 2 and a post of quality 5 given any number of comments, whereas you could previously.

[-]roland6y120

Is the CFAR handbook publicly available? If yes, link please. If not, why not? It would be a great resource for those who can’t attend the workshops.

[-]Tetraspace6y210

There's no official, endorsed CFAR handbook that's publicly available for download. The CFAR handbook from summer 2016, which I found on libgen, warns

While you may be tempted to read ahead, be forewarned - we've often found that participants have a harder time grasping a given technique if they've already anchored themselves on an incomplete understanding. Many of the explanations here are intentionally approximate or incomplete, because we believe this content is best transmitted in person. It helps to think of this handbook as a companion to the workshop, rather than as a standalone resource.

which I think is still their view on the matter.

I have heard that they would be more comfortable with people learning rationality techniques in-person from a friend, so if you know any CFAR alumni you could ask them (they'd probably also have a better answer to your question).

[-]Said Achmiz6y370

This is, however, a position which has always made me extremely suspicious of CFAR’s output. I wish they would, at the very least, acknowledge what a huge red flag this sort of thing is.

[-]Raemon6y210

Noting that I do agree with this particular claim.

I see the situation as:

There are, in fact, good reasons that its hard to communicate and demonstrate some things, and that hyperfocus on "what can be made legible to a third party" results in a lot of looking under street lamps, rather than where the value necessarily is. I have very different priors than Said on how suspicious CFAR's actions are, as well as different firsthand experience that leads me to believe there's a lot of value in CFAR's work that Said presumably dismisses.

[this is not "zero suspicion", but I bet my suspicion takes a very different shape than Said's]

But, it's still important for group rationality to have sound game theory re: what sort of ideas gain what sort of momentum. An important meta-agreement/policy is for people and organizations to be clear about the epistemic status of their ideas and positions.

I think it takes effort to maintain the right epistemic state, as groups and as individuals. So I think it would have been better if CFAR explicitly stated in their handbook, or in a public blogpost*, that "yes, this limits how much people should trust us,

... (read more)

8Said Achmiz6y

“It is very difficult to find a black cat in a dark room—especially when the cat is not there.” I’ve quoted this saying before, more than once, but I think it’s very applicable to CFAR, and I think it is long past time we acknowledged this. The point is this: yes, it is possible that what CFAR is trying to excel at is in fact very difficult. It could, indeed, be that CFAR’s techniques are genuinely difficult to teach—and forget about trying to teach them via (non-personally-customized!) text. It could be, yes, that CFAR’s progress in toward the problem they are attacking is, while quite real, nonetheless fiendishly difficult to demonstrate, in any sort of legible way. All of these things are possible. But there is another possibility. It could be that what CFAR is trying to excel at is not very difficult, but impossible. And because of this, it could be that CFAR’s techniques are not hard to teach, but rather there is nothing there to be taught. It could be that CFAR’s progress toward the problem they are attacking, is not hard to demonstrate, but rather nonexistent. What you say is not, as far as it goes, wrong. But it seems to be predicated on the notion that there is a cat in the room—hidden quite well, perhaps, but nonetheless findable; and also, secondarily, on the notion that what CFAR is doing to look for said cat is even approximately the sort of thing which has any chance at all of finding it. But while it may have been sensible to start (fully 10 years ago, now!) with the assumption of an existing and definitely present cat, that was then; we’ve since had 10 years of searching, and no apparent progress. Now, it is a very large room, and the illumination is not at all bright, and we can expect no help from the cat, in our efforts to find it. So perhaps this is the sort of thing that takes 10 years to produce results, or 50, or 500—who knows! But the more time passes with no results, the more we should re-examine our initial assumption. It is even worth

[-]Raemon6y120

On the "is there something worth teaching there" front, I think you're just wrong, and obviously so from my perspective (since I have, in fact, learned things. Sunset at Noon is probably the best writeup of what CFAR-descended things I've learned and why they're valuable to me).

This doesn't mean you're obligated to believe me. I put moderate probability on "There is variation on what techniques are useful for what people, and Said's mind is shaped such that the CFAR paradigm isn't useful, and it will never be legible to Said that the CFAR paradigm is useful." But, enough words have been spent trying to demonstrate things to you that seem obvious to me that it doesn't seem worth further time on it.

The Multi-Agent Model of Mind is the best current writeup of (one of) the important elements of what I think of as the CFAR paradigm. I think it'd be more useful for you to critique that than to continue this conversation.

7Said Achmiz6y

I have read your post Sunset at Noon; I do not recall finding much to comment on at all, there (I didn’t really get the impression that it was meant to be a “writeup of … CFAR-descended things”!), but I will re-read it and get back to you. As for the multi-agent model of mind, I have already critiqued it, though, admittedly, in a haphazard way—a comment here, a comment there… I have not bothered to critique more in-depth, or in a more targeted way, because… well, to be frank, because of the many times when I’ve attempted to really critique anything, on the new Less Wrong, and been rebuffed for being insufficiently “prosocial”, insufficiently “nice”, etc., etc. Should I understand your suggestion to mean that if I post critiquing comments on some of the posts in the sequence you linked, they will not spawn lengthy threads about niceness, will not be met with moderator comments about whether I am inserting enough caveats about how of course I respect the OP very much, etc.? I’m sorry if this seems blunt or harsh; I really don’t mean to be antagonistic toward you in particular (or anyone, really)! But if you reply to my critical comments by saying “but we have put forth our best effort; the ball’s in your court now” (an entirely fair response, if true!), then before I run with said ball, I need to know it’s not a waste of my time. And, to be clear, if you respond with “no, all the politeness rules still apply, you have to stick to them if you want to critique these writeups [on Less Wrong]”, then—fair enough! But I’d like to know it in advance. (In such a case, I of course will not post any such critiques here; I may post them elsewhere, or not at all, as I find convenient.)

[-]Kaj_Sotala6y150

I think that your past criticisms have been useful, and I've explicitly tried to take them into account in the sequence. E.g. the way I defined subagents in the first post of the sequence, was IIRC in part copy-pasted from an earlier response to you, and it was your previous comment that helped/forced me to clarify what exactly I meant. I'd in fact been hoping to see more comments from you on the posts, and expect them to be useful regardless of the tone.

[-]Raemon6y120

I think I should actually punt this question to Kaj_Sotala, since they are his posts, and the meta rule is that authors get to set the norms on this posts. But:

a) if I had written the posts, I would see them as "yes, now these are actually at the stage where the sort of critique Said does is more relevant." I still think it'd be most useful if you came at it from the frame of "What product is Kaj trying to build, and if I think that product isn't useful, are there different products that would better solve the problem that Kaj's product is trying to solve?"

b) relatedly, if you have criticism of the Sunset at Noon content I'd be interested in that. (this is not a general rule about whether I want critiques of that sort. Most of my work is downstream of CFAR paradigm stuff, and I don't want most of my work to turn into a debate about CFAR. But it does seem interesting to revisit SaN through the "how content that Raemon attributes to CFAR holds up to Said" lens)

c) Even if Kaj prefers you not to engage with them (or to engage only in particular ways), it would be fine under the meta-rules for you to start a separate post and/or discussion thread for the purpose of critiquing. I actually think the most useful thing you might do is write a more extensive post that critiques the sequence as a whole.

[-]Said Achmiz6y180

I think I should actually punt this question to Kaj_Sotala, since they are his posts, and the meta rule is that authors get to set the norms on this posts.

Sure.

I still think it’d be most useful if you came at it from the frame of “What product is Kaj trying to build, and if I think that product isn’t useful, are there different products that would better solve the problem that Kaj’s product is trying to solve?”

Sure, but what if (as seems likely enough) I think there aren’t any different products that better solve the problem…?

I actually think the most useful thing you might do is write a more extensive post that critiques the sequence as a whole.

So, just as a general point (and this is related to the previous paragraph)…

The problem with the norm of writing critiques as separate posts, is that it biases (or, if you like, nudges) critiques toward the sort that constitute points or theses in their own right.

In other words, if you write a post, and I comment to say that your post is dumb and you are dumb for thinking and writing this and the whole thing is wrong and bad (except, you know, in a tactful way), well, that is, at least in some sense, appropriate (or we might say,

... (read more)

[-]Raemon6y*100

And the thing is, many of my critiques (of CFAR stuff, yes, and of many other things that are discussed in rationalist spaces) boil down to just “what you are saying is wrong”. If you ask me what I think the right answer is, in such cases, I will have nothing to offer you. I don’t know what the right answer is. I don’t think you know what the right answer is, either; I don’t think anyone has the right answer. Beyond saying that (hypothetical) you are wrong, I often really don’t have much to add.

If all you have to say is "this seems wrong", that... basically just seems fine. [edit to clarify: I mean making a comment, not a post].

I don't expect most LessWrong users would get annoyed at that. The specific complaint we've gotten about you has more to do with the way you Socratic-ly draw people into lengthy conversations that don't acknowledge the difference in frame, and leave people feeling like it was a waste of time. (This has more to do with implicitly demanding asymmetric effort between you and the author, than about criticism).

2Said Achmiz6y

I’m not quite sure what you’re saying. Yes, no doubt, no one’s complained about me doing the thing I described—because, obviously, I haven’t ever done it! You say that it “basically seems just fine”, but… I don’t expect that it would actually seem “just fine” if I (or anyone else) were to actually do it. Of course, I could be wrong. What are three examples of posts that others have written, that boil down simply to “other post X, written by person Y, is wrong”, and which have gotten a good reception? Perhaps if we did a case study or three, we’d gain some more insight into this thing. (As for the “specific complaint”—there I just don’t know what you mean. Feel free to elaborate, if you like.)

4Raemon6y

Slight clarification – I think I worded the previous comment confusingly. I meant to say, if the typical LessWrong user wrote a single comment in reply to a post saying "this seems wrong", I would expect that to basically be fine. I only recommend the "create a whole new post" thing when an author specifically asks you to stop commenting.

8Raemon6y

(In some cases I think creating a whole new post would actually be just fine, based on how I've seen, say, Eliezer, Robin Hanson, Zvi, Ben Hoffman and Sarah Constantin respond to each other in longform on occasion. In other cases creating a whole new post might go over less well, and/or might be a bit of an experiment rather than a tried-and-true-solution, but I think it's the correct experiment to try) Also want to be a clear - if authors are banning or asking lots of users to avoid criticism, I do think the author should take something of a social hit as "a person who can't accept any criticism". But I nonetheless think it's still a better metanorm for established authors to have control over their post's discussion area. [The LessWrong team is currently trying to develop a much clearer understanding of what good moderation policies are, which might result in some of my opinions changing over the next few weeks, this is just a quick summary of what I currently believe]

2Said Achmiz6y

Quite. A suggestion, then, if I may: display “how many people has this person banned from their posts” (with, upon a click or mouseover or some such, the full list of users available, who have been thus banned) prominently, when viewing a person’s post (somewhere near the post’s author line, perhaps). This way, if I open a post by one Carol, say, I can see at once that she’s banned 12 people from her posts; I take note of this (as that is unusually many); I then click/mouseover/etc., and see either that all the banned accounts are known trolls and curmudgeons (and conclude that Carol is a sensible person with a low tolerance for low-grade nonsense), or that all the banned accounts are people I judge to be reasonable and polite (and conclude that Carol is a prima donna with a low tolerance for having her ideas challenged).

6Raemon6y

Something in that space seems basically reasonable. Note that I haven't prioritized cleaning up (and then improving visibility for) the moderation log in part because the list of users who have ever banned users is actually just extremely short, and meanwhile there's a lot of other site features that seem higher priority. I have been revisiting it recently and think it'd be a good thing to include in the nearish future (esp. if I am prioritizing other features that'd make archipelago-norms more likely to actually get used), but for the immediate future I actually think just saying to the few people who've expressed concerns 'yo, when you look at the moderation log almost nobody has used it' is the right call given limited dev time.

4Said Achmiz6y

Ah, I see. Well, yes. But then, that’s also what I was saying: this sort of thing is generally fine as a comment, but as a post… I entirely understand your intention here, but consider: this would be even worse, “optics”-wise! “So,” thinks the reader, “this guy was so annoying, with his contrarian objections, that the victim of his nitpicking actually asked him to stop commenting; but he can’t let it go, so he wrote a whole post about it?!” And of course this is an uncharitable perspective, and one which isn’t consistent with “good truth-seeking norms”, etc. But… do you doubt that this is the sort of impression that will, if involuntarily, be formed in the minds of the commentariat?

[-]Raemon6y120

I'm fairly uncertain here. But I don't currently share the intuition.

Note that the order of events I'm suggesting is:

1. Author posts.

2. Commenter says "this seems wrong / bad". Disagreement ensues

3. Author says "this is annoying enough that I'd prefer you not to comment on my posts anymore." [Hopefully, although not necessarily, the author does this knowing that they are basically opting into you now being encouraged by LessWrong moderators to post your criticism elsewhere if you think it's important. This might not currently be communicated that well but I think it should be]

4. Then you go and write a post titled 'My thoughts on X' or 'Alternative Conversation about X' or whatever, that says 'the author seems wrong / bad.'

By that point, sure it might be annoying, but it's presumably an improvement from the author's take. (I know that if I wanted to write a post about some high level Weird Introspection Stuff that took a bunch of Weird Introspection Paradigm stuff for granted, I'd personally probably be annoyed if you made the discussion about whether the Weird Introspection Paradigm was eve... (read more)

[-]Raemon6y110

Addenda: my Strategies of Personal Growth post is also particularly downstream of CFAR. (I realize that much of it is something you can elsewhere. My perspective is that the main product CFAR provides is a culture that makes it easier to orient this sort of thing, and stick with it. CFAR iterates on "what combination of techniques can you present to a person in 4 days that best help jump-start them into that culture?", and they chose that feedback-loop-cycle after exploring others and finding them less effective)

One salient thing from the Strategies of Personal Growth perspective (which I attribute to exploration by CFAR researchers) is that many of the biggest improvements you can gain come from healing and removing psychological blockers.

1Said Achmiz6y

Separately from my other comment, I will say— This happens to be phrased such that it could be literally true, but the implication—that the CFAR paradigm is in fact useful (to some people), and that it could potentially be useful (to some people) but the fact of its usefulness could be illegible to me—cannot be true. (Or, to be more precisely, it cannot be true simultaneously with the claim being true that “the CFAR paradigm” constitutes CFAR succeeding at finding [at least part of] what they were looking for. Is this claim being made? It seems like it is—if not, that should be made clear!) The reason is simple: the kind of thing that CFAR (claimed to have) set out to look for, is the kind of thing that should be quite legible even to very skeptical third parties. “We found what we were looking for, but you just can’t tell that we did” is manifestly an evasion.

[-]habryka6y110

The reason is simple: the kind of thing that CFAR (claimed to have) set out to look for, is the kind of thing that should be quite legible even to very skeptical third parties.

What is your current model of what CFAR "claimed to have set out to look for"? I don't actually know much of an explicit statement of what CFAR was trying to look for, besides the basic concepts of "applied rationality".

9ESRogs6y

Correction: CFAR was started in 2012 (though I believe some of the founders ran rationality camps the previous summer, in 2011), so it's been 7 (or 8) years, not 10.

2Said Achmiz6y

Less Wrong, however, was launched in 2009, and that is what I was referring to (namely, Eliezer’s posts about the community and Bayesian dojos and so forth).

[-]Zack_M_Davis6y280

a harder time grasping a given technique if they've already anchored themselves on an incomplete understanding

This is certainly theoretically possible, but I'm very suspicious of it on reversal test grounds: if additional prior reading is bad, then why isn't less prior reading even better? Should aspiring rationalists not read the Sequences for fear of an incomplete understanding spoiling themselves for some future $3,900 CfAR workshop? (And is it bad that I know about the reversal test without having attended a CfAR workshop?)

I feel the same way about schoolteachers who discourage their students from studying textbooks on their own (because they "should" be learning that material by enrolling in the appropriate school course). Yes, when trying to learn from a book, there is some risk of making mistakes that you wouldn't make with the help of a sufficiently attentive personal tutor (which, realistically, you're not going to get from attending lecture classes in school anyway). But given the alternative of placing my intellectual trajectory at the mercy of an institution that has no particular reason to care about my welfare, I think I'll take my chances.

Note that I'm specificall

... (read more)

[-]PeterMcCluskey6y210

You use math as an example, but that's highly focused on System 2 learning. That suggests that you have false assumptions about what CFAR is trying to teach.

There are many subjects where written instructions are much less valuable than instruction that includes direct practice: circling, karate, meditation, dancing, etc. Most of those analogies are fairly imperfect, and some have partially useful written instructions (in the case of meditation, the written version might have lagged in-person instruction by many centuries). Circling is the example that I'd consider most apt, but it won't mean much to people who haven't taken a good circling workshop.

A different analogy, which more emphasizes the costs of false assumptions: people often imagine that economics teaches something like how to run a good business or how to predict the stock market, because there isn't any slot in their worldview for what a good economics course actually teaches. There are plenty of mediocre executive summaries of economics, which fail to convey to most people that economics requires a pervasive worldview shift (integrating utilitarianism, empiricism about preferences, and some counterintuitive empirical p

... (read more)

[-]Said Achmiz6y130

Sometimes, we don’t know how to teach a subject in writing because the subject matter is inherently about action (rather than concepts, analysis, explanation, prediction, numbers, words, etc.).

But sometimes, we don’t know how to teach a subject in writing because there is, in fact, nothing (or, at best, nothing much) to be taught. Sometimes, a subject is actually empty (or mostly empty) of content.

In the latter case, attempting to write it down reveals this (and opens the alleged “content” to criticism)—whereas in person, the charisma of the instructors, the social pressure of being in a group of others who are there to receive the instruction, possibly the various biases associated with having made some costly sacrifice (time, money, etc.) to be there, possibly the various biases associated with the status dynamics at play (e.g. if the instructors are respected, or at least if those around you act as if they are), all serve to mask the fundamental emptiness of what is being “taught”.

I leave it to the reader to discern which of the given examples fall into which category. I will only note that while the subjects found in the former category are often difficult to teach, nevertheless one’s mastery of them, and their effectiveness, is usually quite easy to verify—because action can be demonstrated.

8PeterMcCluskey6y

Meditation is action, in some important sense, and mostly can't be demonstrated. It is hard to reliably distinguish between the results of peer pressure and actual learning. I think CFAR's best reply to this has been it's refund policy: last I knew they offered full refunds to anyone who requested it within one year (although I can't find any online mention of their current policy).

8Said Achmiz6y

Everything is “action” in “some sense”. (Whether that sense is “important”, in any given case, is a matter of perspective.) As far as I am concerned—for the purposes of this topic—if it can’t be demonstrated, it ain’t action. I submit to you that if this is true of any given case, then that is an excellent signal that no actual learning has taken place. (And the more true it is—the harder it is to distinguish between actual learning and the results of various biases, social pressure included—the stronger the signal is.)

[-]Zack_M_Davis6y100

There are many subjects where written instructions are much less valuable than instruction that includes direct practice: circling, karate, meditation, dancing, etc.

Yes, I agree: for these subjects, the "there's a lot of stuff we don't know how to teach in writing" disclaimer I suggested in the grandparent would be a big understatement.

a syllabus is useless (possibly harmful) for teaching economics to people who have bad assumptions about what kind of questions economics answers

Useless, I can believe. (The extreme limiting case of "there's a lot of stuff we don't know how to teach in this format" is "there is literally nothing we know how to teach in this format.") But harmful? How? Won't the unexpected syllabus section titles at least disabuse them of their bad assumptions?

Reading the sequences [...] are unlikely to have much relevance to what CFAR teaches.

Really? The tagline on the website says, "Developing clear thinking for the sake of humanity’s future." I guess I'm having trouble imagining a developing-clear-thinking-for-the-sake-of-humanity's-future curriculum for which the things we write about on this website would be irrelevant. The "comfort zone expansion" exe

... (read more)

[-]PeterMcCluskey6y260

The idea that CFAR would be superfluous is fairly close to the kind of harm that CFAR worries about. (You might have been right to believe that it would have been superfluous in 2012, but CFAR has changed since then in ways that it hasn't managed to make very legible.)

I think meditation provides the best example for illustrating the harm. It's fairly easy to confuse simple meditation instructions (e.g. focus on your breath, sit still with a straight spine) with the most important features of meditation. It's fairly easy to underestimate the additional goals of meditation, because they're hard to observe and don't fit well with more widely accepted worldviews.

My experience suggests that getting value out of meditation is heavily dependent on a feeling (mostly at a system 1 level) that I'm trying something new, and there were times when I wasn't able to learn from meditation, because I mistakenly thought that focusing on my breath was a much more central part of meditation than it actually is.

The times when I got more value out of meditation were times when I tried new variations on the instructions, or new environments (e.g. on a meditation retreat). I can't see any signs that the n

... (read more)

4ChristianKl6y

A key problem is moving from knowing about a technique to action. When you know 20 techniques and use no one of them it's harder to get you to actually use the 21st technique that you are taught then if you start out with less techniques in your head and have no established pattern of not doing any of the exercises that you were taught. There's less that needs unlearning if you haven't been exposed to material beforehand. I would still err on the side of being more public with information, but I do understand that there is a tradeoff.

0Elo6y

I can offer an explanation that might fit. Rationalists tend toward expertise mode thinking (expert from the torbert action logic framework). Behaviour like reading the book is in line with the expert behaviour. Cfar techniques and related in-person methods are not always about being the expert, they are about doing the best thing. Being a better expert is not always the same as being the better munchkin, the better person or the person who can step out of their knowledge beliefs. In theory, the expert thing is the best thing. In theory there's no difference between theory and practice, in practice, there's a big difference between theory and practice. Having said that, I've never done cfar, and I teach workshops monthly in Sydney and I think they are wrong to discourage sharing of their resources. As the same time I accept the idea of intellectual property being protected even if that's not the case they are claiming. (I'm in the process of writing up my resources into a collection)

7roland6y

I suspect that this is the real reason. Although if the much vaster sequences by Yudkowsky are freely available I don't see it as a good justification for not making the CFAR handbook available.

-6Zack_M_Davis6y

4Said Achmiz6y

We’ve all heard this sort of thing many times, of course. The best response is probably Schopenhauer’s: You are, I assume, referring to the ideas of this person? He appears to be some variety of management consultant. Is there any reason to take this “action logic” of his seriously? It seems to be yet another among the many, many self-help / management consulting / etc. “frameworks” or “systems” etc. Do any of his ideas have any empirical verification, or… well, anything, really?

8Elo6y

That's up to you. I've got a lot of value from the structure he outlines. It's a lot more reasoned than some of the other mysterious odd things I read. If there is something wrong with the theory and the way it maps to the practice, is it better to read more theory or do more practice and make new theories? I would suggest it depends on the person and what they have found to work in the past. And also with an awareness to the loops of bad habits - "sharpen the saw" type problems. Sometimes it's more valuable to stop sharpening the saw, and start cutting down the tree. (rationality frame of thinking loves to sharpen more and cut less)

[-]Tetraspace6y110

I'm off from university (3rd year physics undergrad) for the summer and hence have a lot of free time, and I want to use this to make as much progress as possible towards the goal of getting a job in AI safety technical research. I have found that I don't really know how to do this.

Some things that I can do:

work through undergrad-level maths and CS textbooks
basic programming (since I do physics, this is at the level required to implement simple numerical methods in MATLAB)
the stuff in Andrew Ng's machine learning Coursera course

Thus far I've worked through the first half of Hutton's Programming in Haskell on the grounds that functional programming maybe teaches a style of thought that's useful and opens doors to more theoretical CS stuff.

I'm optimising for something slightly different that purely becoming good at AI safety, in that at the end I'd like to have some legible things to point to or list on a CV or something (or become better-placed to later acquire such legible things).

I'd be interested to hear from people who know more about what would be helpful for this.

[-]limerott6y90

Hi! I've known LW for quite a while, but only now decided to join. I remember reading a comment here and thinking "I like how this person thinks". Needless to say, this is not a common experience I have on the internet. What I hope to get from this site are fruitful intellectual discussions that trip me over and reveal the flaws in my reasoning :)

2habryka6y

Welcome limerott! I hope your time on the site will be well-spent and feel free to ask any questions here in the Open Thread, or via Intercom in the bottom right corner.

[-]Alexei6y80

Hypothesis: there are less comments per user on LW 2.0 than the old LW, because the user base is more educated as to where they have a valuable opinion vs where they don’t.

5habryka6y

Ruby can probably get us the answer to whether the premise is true next week. Seems likely true, but I would only give it 75%.

[-]Ruby6y*150

The question is fuzzier than it might seem at first. The issues is that population of commenters size changes too. You can have a world where the number of very frequent commenters has gone up but the average per commenter has gone done because the number of infrequent commenters has grown even faster than the number of frequent commenters.

There are also multiple possible causes for growth/decline, change in frequency, etc., that I don't think you could really link them to a mechanism as specific as being more educated about where your opinion is valuable. Though I'd definitely link the number of comments per person to the number of other active commenters and number of conversations going, a network effects kind of thing.

Anyhow, some graphs:

Indeed, the average (both mean and median) comments per active commenter each week has gone down.

But it's generally the case that the number of comments and commenters went way down, recovering only late 2017 at the time of Inadequate Equilibria and LessWrong 2.0

[-]Ruby6y130

We can also look at the composition commenter commenting frequency. Here I've been the commenters for each week into a bin/bucket and seen how they've changed. Top graph is overall volume, bottom graph is the percentage of commenting population in each frequency bucket:

I admit that we must conclude that high-frequency commenters (4+ comments/week) have diminished in absolute numbers and as a percentage over time, though a slight upward trend in the last six months.

3ryan_b6y

Great work! Are there any obvious tie-ins to the launch of the Alignment Forum? It seems plausible that the people who were here almost exclusively for the AI research posts might have migrated there. Alternatively, if Alignment Forum is in fact counted, it might be that the upward trend reflects growth in that segment.

3PeterMcCluskey6y

I'll suggest the main change is open threads get fewer comments because the system makes open threads less conspicuous, and provides alternatives, such as questions, that are decent substitutes for comments.

2Dr_Manhattan6y

I think the Hypothesis is not about Open Threads specifically

2Pattern6y

Upvoted for testability.

[-]Aleksi Liimatainen6y70

I noticed I was confused about how humans can learn novel concepts from verbal explanations without running into the symbol grounding problem. After some contemplation, I came up with this:

To the extent language relies on learned associations between linguistic structures and mental content, a verbal explanation can only work with what's already there. Instead of directly inserting new mental content, the explanation must leverage the receiving mind's established content in a way that lets the mind generate its own version of the new content.

The... (read more)

[-]Kaj_Sotala6y100

Constructivist learning theory is a relevant keyword; its premise is pretty much directly the same as in your quote (my emphasis added):

An important restriction of education is that teachers cannot simply transmit knowledge to students, but students need to actively construct knowledge in their own minds. That is, they discover and transform information, check new information against old, and revise rules when they do not longer apply. This constructivist view of learning considers the learner as an active agent in the process of knowledge acquisition. Constructivist conceptions of learning have their historical roots in the work of Dewey (192 9), Bruner (1961), Vygotsky (1962), and Piaget (1980). Bednar, Cunningham, Duffy, and Perry (1992) and von Glasersfeld (1995) have proposed several implications of constructivist theory for instructional developers stressing that learning outcomes should focus on the knowledge construction process and that learning goals should be determined from authentic tasks with specific objectives. Similarly, von Glasersfeld (1995) states that learning is not a stimulus-response phenomenon, but a process that requires self-regulation and the development

... (read more)

4Aleksi Liimatainen6y

Thanks, this is exactly what I was looking for. Not a new idea then, though there's something to be said for semi-independent reinvention. The obvious munchkin move would be to develop a reliable means of boostrapping a basic mental model of constructivist learning and grounding it in the learner's own direct experience of learning. Turning the learning process on itself should lead to some amount of recursive improvement, right? Has that been tried?

1mako yass6y

:D (I am delighted because constructivism is what is to be said for semi-independent reinvention, which aleksi just semi-independently constructed, thereby doing a constructivism on constructivism)

[-]ryan_b6y70

Has anyone been to DynamicLand in Berkeley? If so, what did you think of it?

[-]habryka6y110

I was there before it was fully done. As a person with a strong interest in UX I found it quite exciting.

It definitely tries to be a modern Xerox Park or something like that, and it does really feel like it's doing a lot of really interesting things in the UI space. I have a really hard time telling whether any of the UI ideas they are experimenting with will actually turn out to be useful and widely adopted, but it definitely helped me think about UX in a better way.

2ryan_b6y

It does seem to me like the kind of thing that would allow capitalizing strongly on something like a shared technical understanding. But that would be very difficult to pull off, because the overlap of people with shared technical understanding and advanced UI understanding is small. If I were to say something like "DynamicLand can add UX to any layer of abstraction," how would that sound?

8gwillen6y

I have been! I thought it was an interesting experiment, and I really hope they have another community day so I can visit again. I think there's probably a lot to learn from it, and I think there are things that you can only learn effectively by trying out weird experiments in real life to see "how this feels". But I don't really expect anything to directly come of it -- the project is pretty janky, and while it's a fantastic platform for tiny cute demos, I don't think any concrete part of it (other than the general sense of "this is an inspiration to try to go recreate this neat type of interaction in a more robust way") is really useful.

4Richard_Kennaway6y

I haven't been there, but I was reading about Dynamic Land just yesterday (via Dominic Cummings' blog), and I've read some of Bret Victor's writings. I approve of the ideas tremendously, but it's not clear to me that in practice the work has provided any more of an advance in "visual programming" than other efforts in this area. Beyond the decades-old WIMP (ETA: and spreadsheets) interface, none of these, it seems to me, ever make more than toy demos. I have never seen them scale up to real power tools that someone would use to accomplish something. Ideas like these have been around long enough that toys and dreams will no longer do. There are lathes that can make all of their own parts. Could Dynamic Land create Dynamic Land? What would such a system look like if it could?

2gwillen6y

I agree that it only makes toy demos, but it definitely goes beyond WIMP. It's a simulation of the sort of interface one might expect in a future where every surface is a screen -- it's a janky, extremely-low-fidelity simulation, which holds together barely well enough to serve the purpose, but it does serve, and it's an interesting way to try out this interaction style. A surprisingly large part of Dynamic Land is actually (in some sense) self-hosting. There is a small core/kernel that is extrinsic, including e.g. the parser for the language. But a lot of low-level functionality is indeed implemented inside the interpreter (as I recall, they use a sort of pseudo-Lua, crossed with Smalltalk-like agent stuff -- my vague recollection is that there's something like a Lua interpreter with a custom preprocessor underlying it.)

[-]Richard_Kennaway6y60

Recently out: "The Transhumanism Handbook" ed. Newton Lee (Springer, 2019). Costs money, of course, but you can see the table of contents, the abstracts, and the references for each paper for free. It contains:

5 chapters on yay, transhumanism!

10 on AI

12 on longevity

5 on biohacking

3 on cryptocurrency

5 on art

16 on society and ethics

10 on philosophy and religion

[-]Sherrinford6y50

Is there exactly one RSS feed of lesswrong.com, i.e. https://www.lesswrong.com/feed.xml ? As I know too little about the technical side, is it easily possible for you to add different RSS feeds?

6habryka6y

There are quite a few RSS feeds. You can construct them by clicking on the "Subscribe via RSS" button on the frontpage:

1Sherrinford6y

Thanks!

[-]ryan_b6y40

Reading about libraries in Julia for geometric algebra, I found Grassmann.jl. This is going to require more knowledge of advanced algebra than I have to use effectively, but while reading about it I noticed the author describe how they can achieve very high dimension numbers. They claim ~4.6e18 dimensions.

That's a lotta dimensions!

4Max foobar6y

Only barely related, but Grassmann numbers are hilariously weird. Among other properties, their square is always zero (though they’re generally non-zero).

[-]ryan_b6y*40

The American National Institute of Standards and Technology has a draft plan for AI standards up. There is an announcement on their website; an announcement post on LessWrong; the plan itself on NIST's website; an outline of said plan on LessWrong.

Edit: changed style of links in response to the Please Give Your Links Speaking Names post.

[-]Jason Geringer6y30

Hey what's up everybody, i'm jason. i found lesswrong as i was researching questions for a quiz app that i'm workin on called wisdom of the crowd app. and i got started doin that cause i started a facebook group called wisdom of the crowd. i noticed right away that you are vibrating at my same frequency so to speak. the rationality philosophy is what motivated me to start that fb group. it's only days old but i got the impression that the idea i had is pretty similar to what you guys are doing here. i love your library lol i'm... (read more)

3ryan_b6y

The front page features are very useful for getting up to speed. Recently Curated is newer posts the mods thought were important, From the Archives are old posts that were well received, and Continue Reading helps keep track of the sequences (the core content of the site) so you can consume them over time. Welcome!

[-]Mary Chernyshenko6y30

Kind of stupid question, actually. I Googled up clothes for one-armed children (tried knitting, didn't go as planned, thought I'd donate it), and there were much fewer search results than I'd expected. Is it because one-armed people just have their clothes re-sewn from ordinary stuff, or what? Or are there different key words for it?

5eigen6y

I would think they would just buy regular clothes. The same way that you cannot buy only one shoe of a pair of shoes.

1Mary Chernyshenko6y

Still seems kind of inefficient, though :(

2ryan_b6y

I wonder if there would be enough interest to support a kind of matching app that would let people put their Amazon wishlist up and then match them with someone who had the same item but opposite needs (ie left vs right shoe), and then split the cost.

1Mary Chernyshenko6y

Could be. But it is still only shoes... and sending them to two different customers might drown any difference in cost.

[-]Liam Donovan6y30

It seems like there are some intrinisic connections between the clusters of concepts known as "EA", "LW-style rationality", and "HRAD research"; is this a worrying sign?

Specifically, it seems like the core premise of EA relies largely on a good understanding of the world, in a systemic and explicit manner (beause existing heuristics aren't selected for "maximizing altruism"[1]), linking closely to LW, which tries to answer the same question. At the same time, my understanding of HRAD research is that it aims to... (read more)

4ryan_b6y

The intrinsic connection is primarily that they arose out of the same broad community, and there is heavy overlap between personnel as a consequence. I say this is not a worrying sign, because the comparison isn't between their shared methods and some better memeplex, it is between their shared methods and the status quo. That is to say, there's no reason to believe something else would be happening in place of this; more likely everyone would have scattered throughout what was already happening. It's very important to distinguish those etceteras you listed, because those are three different worlds. In the world where AI risk is in fact low, HRAD can still be very successful in mitigating it further and also fruitful in thinking about similar risks. In the world where the underlying philosophical assumptions are wrong, demonstrating that wrongness is valuable in-and-of itself to the greater safety project. In the world where alternate safety approaches have higher tractability, how would we even tell without comparison to the challenges encountered in HRAD? HRAD is also the product of specific investigations into tractability and philosophical soundness. I expect they will iterate on these very questions again in the future. If it winds up a dead end I expect the associated communities to notice and then to shift focus elsewhere. To sum up, we have noticed the skulls. Hail, humanity! We who are about to die salute you.

1Liam Donovan6y

I disagree with this though! I think anyone that wants to think along EA lines is inevitably going to want to investigate how to improve epistemic rationality, which naturally leads to thinking about decision making for idealized agents. Having community overlap is one thing, but the ideas seem so closely related that EA can't develop in any possible world without being biased towards HRAD research. I mean surely there would be some worlds in which HRAD research was not the most valuable use of (some portion of*) EA money; it doesn't really matter whether the specific examples I gave work, just that EA would be unable to distinguish worlds where HRAD is an optimal use of resources from the world where it is not. But why? Is it not at all concerning that aliens with no knowledge of Earth or humanity could plausibly guess that a movement dedicated to a maximizing, impartial, welfarist conception of the good would also be intrinsically attracted to learning about idealized reasoning procedures? The link between them is completely unconnected to the object-level question "is HRAD research the best use of [some] EA money?", or even to the specifics of how the LW/EA communities formed around specific personalities in this world.

2ryan_b6y

I don't understand the source of your concern. This is not at all concerning. If we are concerned about this then we should also be concerned that aliens could plausibly guess a movement dedicated to space exploration would be intrinsically attracted to learning about idealized dynamical procedures. It seems to me this is just a prior that groups with a goal investigate instrumentally useful things. My model of your model so far is this: because the EA community is interested in LessWrong, and because LessWrong facilitated the group that work on HRAD research, the EA community will move their practices closer to implications of this research even in the case where it is wrong. Is that accurate? My expectation is that EAs will give low weight to the details of HRAD research, even in the case where it is a successful program. The biggest factor is the timelines: HRAD research is in service of the long term goal of reasoning correctly about AGI; EA is about doing as much good as possible, as soon as possible. The iconic feature of the EA movement is the giving pledge, which is largely predicated on the idea that money given now is more impactful than money given later. There is a lot of discussion about alternatives and different practices, for example the donor's dilemma and mission hedging, but these are operational concerns rather than theoretical/idealized ones. Even if I assume HRAD is a productive line of research, I strongly expect that the path to changing EA practice leads from some surprising result, evaluated all the way up to the level of employment and investment decisions. This means the result would need to be surprising, then it would need to withstand scrutiny, then it would need to lead to conclusions big enough to shift activity like donations, employment, and investments, cost of change included and all. I would be deeply shocked if this happened, and then further shocked if it had a broad enough impact to change the course of EA as a group.

Moderation Log