In this post, I proclaim/endorse forum participation (aka commenting) as a productive research strategy that I've managed to stumble upon, and recommend it to others (at least to try). Note that this is different from saying that forum/blog posts are a good way for a research community to communicate. It's about individually doing better as researchers.

I like the fact that despite not being (relatively) young when they died, the LW banner states that Kahneman & Vinge have died "FAR TOO YOUNG", pointing to the fact that death is always bad and/or it is bad when people die when they were still making positive contributions to the world (Kahneman published "Noise" in 2021!).
Novel Science is Inherently Illegible Legibility, transparency, and open science are generally considered positive attributes, while opacity, elitism, and obscurantism are viewed as negative. However, increased legibility in science is not always beneficial and can often be detrimental. Scientific management, with some exceptions, likely underperforms compared to simpler heuristics such as giving money to smart people or implementing grant lotteries. Scientific legibility suffers from the classic "Seeing like a State" problems. It constrains endeavors to the least informed stakeholder, hinders exploration, inevitably biases research to be simple and myopic, and exposes researchers to constant political tug-of-war between different interest groups poisoning objectivity.  I think the above would be considered relatively uncontroversial in EA circles.  But I posit there is something deeper going on:  Novel research is inherently illegible. If it were legible, someone else would have already pursued it. As science advances her concepts become increasingly counterintuitive and further from common sense. Most of the legible low-hanging fruit has already been picked, and novel research requires venturing higher into the tree, pursuing illegible paths with indirect and hard-to-foresee impacts.
I thought I didn’t get angry much in response to people making specific claims. I did some introspection about times in the recent past when I got angry, defensive, or withdrew from a conversation in response to claims that the other person made.  After some introspection, I think these are the mechanisms that made me feel that way: * They were very confident about their claim. Partly I felt annoyance because I didn’t feel like there was anything that would change their mind, partly I felt annoyance because it felt like they didn’t have enough status to make very confident claims like that. This is more linked to confidence in body language and tone rather than their confidence in their own claims though both matter.  * Credentialism: them being unwilling to explain things and taking it as a given that they were correct because I didn’t have the specific experiences or credentials that they had without mentioning what specifically from gaining that experience would help me understand their argument. * Not letting me speak and interrupting quickly to take down the fuzzy strawman version of what I meant rather than letting me take my time to explain my argument. * Morality: I felt like one of my cherished values was being threatened.  * The other person was relatively smart and powerful, at least within the specific situation. If they were dumb or not powerful, I would have just found the conversation amusing instead.  * The other person assumed I was dumb or naive, perhaps because they had met other people with the same position as me and those people came across as not knowledgeable.  * The other person getting worked up, for example, raising their voice or showing other signs of being irritated, offended, or angry while acting as if I was the emotional/offended one. This one particularly stings because of gender stereotypes. I think I’m more calm and reasonable and less easily offended than most people. I’ve had a few conversations with men where it felt like they were just really bad at noticing when they were getting angry or emotional themselves and kept pointing out that I was being emotional despite me remaining pretty calm (and perhaps even a little indifferent to the actual content of the conversation before the conversation moved to them being annoyed at me for being emotional).  * The other person’s thinking is very black-and-white, thinking in terms of a very clear good and evil and not being open to nuance. Sort of a similar mechanism to the first thing.  Some examples of claims that recently triggered me. They’re not so important themselves so I’ll just point at the rough thing rather than list out actual claims.  * AI killing all humans would be good because thermodynamics god/laws of physics good * Animals feel pain but this doesn’t mean we should care about them * We are quite far from getting AGI * Women as a whole are less rational than men are * Palestine/Israel stuff   Doing the above exercise was helpful because it helped me generate ideas for things to try if I’m in situations like that in the future. But it feels like the most important thing is to just get better at noticing what I’m feeling in the conversation and if I’m feeling bad and uncomfortable, to think about if the conversation is useful to me at all and if so, for what reason. And if not, make a conscious decision to leave the conversation. Reasons the conversation could be useful to me: * I change their mind * I figure out what is true * I get a greater understanding of why they believe what they believe * Enjoyment of the social interaction itself * I want to impress the other person with my intelligence or knowledge Things to try will differ depending on why I feel like having the conversation. 
habryka4d5120
10
A thing that I've been thinking about for a while has been to somehow make LessWrong into something that could give rise to more personal-wikis and wiki-like content. Gwern's writing has a very different structure and quality to it than the posts on LW, with the key components being that they get updated regularly and serve as more stable references for some concept, as opposed to a post which is usually anchored in a specific point in time.  We have a pretty good wiki system for our tags, but never really allowed people to just make their personal wiki pages, mostly because there isn't really any place to find them. We could list the wiki pages you created on your profile, but that doesn't really seem like it would allocate attention to them successfully. I was thinking about this more recently as Arbital is going through another round of slowly rotting away (its search currently being broken and this being very hard to fix due to annoying Google Apps Engine restrictions) and thinking about importing all the Arbital content into LessWrong. That might be a natural time to do a final push to enable people to write more wiki-like content on the site.
Recently someone either suggested to me (or maybe told me they or someone where going to do this?) that we should train AI on legal texts, to teach it human values. Ignoring the technical problem of how to do this, I'm pretty sure legal text are not the right training data. But at the time, I could not clearly put into words why. Todays SMBC explains this for me: Saturday Morning Breakfast Cereal - Law (smbc-comics.com) Law is not a good representation or explanation of most of what we care about, because it's not trying to be. Law is mainly focused on the contentious edge cases.  Training an AI on trolly problems and other ethical dilemmas is even worse, for the same reason. 

Popular Comments

Recent Discussion

Kaj_Sotala

I just started thinking about what I would write to someone who disagreed with me on the claim "Rationalists would be better off if they were more spiritual/religious", and for this I'd need to define what I mean by "spiritual". 

Here are some things that I would classify under "spirituality":

  • Rationalist Solstices (based on what I've read about them, not actually having been in one)
  • Meditation, especially the kind that shows you new things about the way your mind works
  • Some forms of therapy, especially ones that help you notice blindspots or significantly reframe your experience or relationship to yourself or the world (e.g. parts work where you first shift to perceiving yourself as being made of parts, and then to seeing those parts with love)
  • Devoting yourself to the practice of
...
1sliqz14h
Thanks, for the answer(s). Watched the video as well, always cool to hear about other peoples journeys. If you want there is a discordserver (MD) with some pretty advanced practitioners (3rd/4th path) you and/or Kaj could join (for some data points or practice or fun, feels more useful than Dharmaoverground these days). Not sure whether different enlightenment levels would be more recommendable for random people. E.g. stream-entry might be relatively easy and helpful, but then there is a "risk" of spending the next years trying to get 2nd/3rd/4th. It's such a transformative experience that it's hard to predict on an individual level what the person will do afterwards.  
2Kaj_Sotala6h
That sounds fun, feel free to message me with an invite. :) Worth noting that stream entry isn't necessarily a net positive either:

Carl Jung is a perfect exemplar of all of that, because when he had his extended episode of such after his break with Freud, he indeed had a period where his ego was completely toast and nonfunctional, as he tells it.

BTW when I was 16, and my family and I had landed in Germany, I was suffering from a very bad case of jet lag, and in said state of utter exhaustion dreamt of Jung's Man Eater:

https://jungcurrents.com/carl-jungs-first-dream-the-man-eater

Every basic detail was the same: the underground cavern, the sense that the thing was alive and very dangero... (read more)

3greylag16h
THANK YOU! In personal development circles, I hear a lot about the benefits of spirituality, with vague assurances that you don't have to be a theist to be spiritual, but with no pointers in non-woo directions, except possibly meditation. You have unblurred a large area of my mental map. (Upvoted!)

This is my personal opinion, and in particular, does not represent anything like a MIRI consensus; I've gotten push-back from almost everyone I've spoken with about this, although in most cases I believe I eventually convinced them of the narrow terminological point I'm making.

In the AI x-risk community, I think there is a tendency to ask people to estimate "time to AGI" when what is meant is really something more like "time to doom" (or, better, point-of-no-return). For about a year, I've been answering this question "zero" when asked.

This strikes some people as absurd or at best misleading. I disagree.

The term "Artificial General Intelligence" (AGI) was coined in the early 00s, to contrast with the prevalent paradigm of Narrow AI. I was getting my undergraduate computer science...

I agree that filling a context window with worked sudoku examples wouldn't help for solving hidouku. But, there is a common element here to the games. Both look like math, but aren't about numbers except that there's an ordered sequence. The sequence of items could just as easily be an alphabetically ordered set of words. Both are much more about geometry, or topology, or graph theory, for how a set of points is connected. I would not be surprised to learn that there is a set of tokens, containing no examples of either game, combined with a checker (like y... (read more)

2AnthonyC1h
Oh, by "as qualitatively smart as humans" I meant "as qualitatively smart as the best human experts". I think that is more comparable to saying "as smart as humanity." No individual human is as smart as humanity in general.
2AnthonyC1h
This is an excellent short mental handle for this concept. I'll definitely be using it.
2AnthonyC1h
I was going to say the same. I can't count the number of times a human customer service agent has tried to do something for me, or told me they already did do something for me, only for me to later find out they were wrong (because of a mistake they made), lying (because their scripts required it or their metrics essentially forced them into it), or foiled (because of badly designed backend systems opaque to both of us). 

On the 3rd of October 2351 a machine flared to life. Huge energies coursed into it via cables, only to leave moments later as heat dumped unwanted into its radiators. With an enormous puff the machine unleashed sixty years of human metabolic entropy into superheated steam.

In the heart of the machine was Jane, a person of the early 21st century.

From her perspective there was no transition. One moment she had been in the year 2021, sat beneath a tree in a park. Reading a detective novel.

Then the book was gone, and the tree. Also the park. Even the year.

She found herself laid in a bathtub, immersed in sickly fatty fluids. She was naked and cold.

The first question Jane had for the operators and technicians who greeted her...

Also, thank you for mentioning Worth the Candle. I had not heard of it before but am now enjoying it quite a lot.

Suppose rationality is a set of principles that people agreed on to process information then arrive at conclusions. Then, on the basis of cost-free information exchange, should rational disagreements still exist? In that case, both parties would have the same information which will then be processed the same way. Just by these factors, there shouldn't be.

However, disagreements do still exist, and we'd like to believe we're rational, so the problem must be in the exchange of information. Previous posts have mentioned how sometimes there are too much background information to be exchanged fully. Here I'd like to point to a more general culprit: language.

Not all knowledge can be expressed through language, and not all languages express knowledge. Yet language, including obscure symbols that take in mathematics, n...

This is a linkpost for https://arxiv.org/abs/2403.09863

Hi, I’d like to share my paper that proposes a novel approach for building white box neural networks. It introduces a concept of „semantic feature” and builds a simple white box PoC.

As an independent researcher I’d be grateful for your feedback!

4mishka8h
This looks interesting, thanks! This post could benefit from an extended summary. In lieu of such a summary, in addition to the abstract I'll quote a paragraph from Section 1.2, "The core idea"

Thank you! I wanted to keep the original post concise but I might have overshot in terms of brevity, this is my first post on this forum. Do you think I should edit the original post to include the quote you mentioned? I think it is on point.

On 16 March 2024, I sat down to chat with New York Times technology reporter Cade Metz! In part of our conversation, transcribed below, we discussed his February 2021 article "Silicon Valley's Safe Space", covering Scott Alexander's Slate Star Codex blog and the surrounding community.

The transcript has been significantly edited for clarity. (It turns out that real-time conversation transcribed completely verbatim is full of filler words, false starts, crosstalk, "uh huh"s, "yeah"s, pauses while one party picks up their coffee order, &c. that do not seem particularly substantive.)


ZMD: I actually have some questions for you.

CM: Great, let's start with that.

ZMD: They're critical questions, but one of the secret-lore-of-rationality things is that a lot of people think criticism is bad, because if someone criticizes you, it hurts your...

I was actually looking for specific examples, precisely so that we could test our intuitions, rather than just stating our intuitions. Do you happen to have any particular ones in mind?

2frankybegs3h
So despite it being "hard to substantiate", or to "find Scott saying" it, you think it's so certainly true that a journalist is justified in essentially lying in order to convey it to his audience?
3wilkox4h
I'd have more trust in the writing of a journalist who presents what they believe to be the actual facts in support of a claim, than one who publishes vague insinuations because writing articles is hard. He really didn’t. Firstly, in the literal sense that Metz carefully avoided making this claim (he stated that Scott aligned himself with Murray, and that Murray holds views on race and IQ, but not that Scott aligns himself with Murray on these views). Secondly, and more importantly, even if I accept the implied claim I still don’t know what Scott supposedly believes about race and IQ. I don’t know what ‘is aligned with Murray on race and IQ’ actually means beyond connotatively ‘is racist’. If this paragraph of Metz’s article was intended to be informative (it was not), I am not informed.
2tailcalled4h
It's totally possible to say taboo things, I do it quite often. But my point is more, this doesn't seem to disprove the existence of the tension/Motte-Bailey/whatever dynamic that I'm pointing at.
To get the best posts emailed to you, create an account! (2-3 posts per week, selected by the LessWrong moderation team.)
Log In Reset Password
...or continue with

This is the ninth post in my series on Anthropics. The previous one is The Solution to Sleeping Beauty.

Introduction

There are some quite pervasive misconceptions about betting in regards to the Sleeping Beauty problem.

One is that you need to switch between halfer and thirder stances based on the betting scheme proposed. As if learning about a betting scheme is supposed to affect your credence in an event.

Another is that halfers should bet at thirders odds and, therefore, thirdism is vindicated on the grounds of betting. What do halfers even mean by probability of Heads being 1/2 if they bet as if it's 1/3?

In this post we are going to correct them. We will understand how to arrive to correct betting odds from both thirdist and halfist positions, and...

1Ape in the coat3h
Yes! There is 50% chance that the coin is Tails and so the room is to be Red in this experiment.
1Signer3h
No, I mean the Beauty awakes, sees Blue, gets a proposal to bet on Red with 1:1 odds, and you recommend accepting this bet?
1Ape in the coat2h
Yes, if the bet is about whether the room takes the color Red in this experiment. Which is what event "Red" means in Technicolor Sleeping Beauty according to the correct model. The fact that you do not observe event Red in this awakening doesn't mean that you don't observe it in the experiment as a whole. The situation is somewhat resembling learning that today is Monday and still being ready to bet at 1:1 that Tuesday awakening will happen in this experiment. Though, with colors there is actually an update from 3/4 to 1/2. What you, probably, tried to ask, is whether you should agree to bet at 1:1 odds that the room is Red in this particular awakening after you wake up and saw that the room is Blue. And the answer is no, you shouldn't. But probability space for Technicolor Sleeping beauty is not talking about probabilities of events happening in this awakening, because most of them are illdefined for reasons explained in the previous post.

And the answer is no, you shouldn’t. But probability space for Technicolor Sleeping beauty is not talking about probabilities of events happening in this awakening, because most of them are illdefined for reasons explained in the previous post.

So probability theory can't possibly answer whether I should take free money, got it.

And even if "Blue" is "Blue happens during experiment", you wouldn't accept worse odds than 1:1 for Blue, even when you see Blue?

There's a particular kind of widespread human behavior that is kind on the surface, but upon closer inspection reveals quite the opposite. This post is about four such patterns.

 

Computational Kindness

One of the most useful ideas I got out of Algorithms to Live By is that of computational kindness. I was quite surprised to only find a single mention of the term on lesswrong. So now there's two.

Computational kindness is the antidote to a common situation: imagine a friend from a different country is visiting and will stay with you for a while. You're exchanging some text messages beforehand in order to figure out how to spend your time together. You want to show your friend the city, and you want to be very accommodating and make sure...

Forget where I read it, but this Idea seems similar. When responding to a request, being upfront about your boundaries or constraints feels intense but can be helpful for both parties. If Bob asks Alice to help him move, and Alice responds "sure thing" that leaves the interaction open to miscommunication. But if instead Alice says, " yeah! I am available 1pm to 5pm and my neck has been bothering me so no heavy lifting for me!" Although that's seems like less of a kind response Bob now doesn't have to guess at Alice's constraints and can comfortably move forward without feeling the need to tiptoe around how long and to what degree Alice can help.

1CstineSublime12h
This is an extremely relatable post, in both ways. I often find myself on the other side of the these interactions too and not knowing how to label and describe my awareness of what's happening without coming across as Larry David from Curb Your Enthusiasm.
2Lukas_Gloor14h
I really liked this post! I will probably link to it in the future. Edit: Just came to my mind that these are things I tend to think of under the heading "considerateness" rather than kindness, but it's something I really appreciate in people either way (and the concepts are definitely linked). 

LessOnline

A Festival of Writers Who are Wrong on the Internet

May 31 - Jun 2, Berkeley, CA