LESSWRONG
LW

The MIRI Summer Fellows Program is having a writing day, where the participants are given a whole day to write whatever LessWrong / AI Alignment Forum posts that they like. This is to practice the skills needed for forum participation as a research strategy.

On an average day LessWrong gets about 5-6 posts, but last year this generated 28 posts. It's likely this will happen again, starting mid-to-late afternoon PDT.

It's pretty overwhelming to try to read-and-comment on 28 posts in a day, so we're gonna make sure this isn't the only chance you'll have to interact with th... (Read more)

Power Buys You Distance From The Crime

113

Elizabeth

19d7 min readShow Highlight

Introduction

Taxes are typically meant to be proportional to money (or negative externalities, but that's not what I'm focusing on). But one thing money buys you is flexibility, which can be used to avoid taxes. Because of this, taxes aimed at the wealthy tend to end up hitting the well-off-or-rich-but-not-truly-wealthy harder, and tax cuts aimed at the poor end up helping the middle class. Examples (feel free to stop reading these when you get the idea, this is just the analogy section of the essay):

Computer programmers typically have the option to work remotely in a low-tax state; t

... (Read more)

Kenny

By-the-way, this is a fantastic comment and would make a great post pretty much by itself (with maybe a little context about that to which it's replying).

1Kenny5h enacting conflict in the course of discussing conflict ... seems to be exactly why it's so difficult to discuss a conflict theory with someone already convinced that it's true – any discussion is necessarily an attack in that conflict as it in effect presupposes that it might be false. But that also makes me think that maybe the best rhetorical counter to someone enacting a conflict is to explicitly claim that one's unconvinced of the truth of the corresponding conflict theory or to explicitly claim that one's decoupling [https://everythingstudies.com/2018/04/26/a-deep-dive-into-the-harris-klein-controversy/] the current discussion from a (or any) conflict theory.

1Kenny8h I don't think it's useful to talk about 'conflict theory', i.e. as a general theory of disagreement. It's more useful in a form like 'Marxism is a conflict theory'. And then a 'conflict theorist' is someone who, in some context, believes a conflict theory, but not that disagreements generally are due to conflict (let alone in all contexts). So, from the perspective of a 'working class versus capital class' conflict theory, public choice theory is obviously a weapon used by the capital class against the working class. But other possible conflict theories might be neutral about public choice theory. Maybe what makes 'conflict theory' seem like a single thing is the prevalence of Marxism-like political philosophies.

Biases: An Introduction

Rob Bensinger

4y4 min readShow Highlight

Imagine reaching into an urn that contains seventy white balls and thirty red ones, and plucking out ten mystery balls.

Perhaps three of the ten balls will be red, and you’ll correctly guess how many red balls total were in the urn. Or perhaps you’ll happen to grab four red balls, or some other number. Then you’ll probably get the total number wrong.

This random error is the cost of incomplete knowledge, and as errors go, it’s not so bad. Your estimates won’t be incorrect on average, and the more you learn, the smaller your error will tend to be.

... (Read more)

jora

Considering the fact that salespeople are seventy-five times more common as librarians, your estimates will give 7.5 more shy salespeople then shy librarians. You fell under base rate neglect bias right after you read about it, which is a very good manifestation of bias blindness.

My math was wrong

[This comment is no longer endorsed by its author]Reply

Davis_Kingsley's Shortform

Davis_Kingsley

5dShow Highlight

mr-hire

Do you think this is relevant to more real world strategy situations?

3Elizabeth9h 1. I have the opposite observation on abuse in poly vs mono relationships. I'm interested in discussing further but I think that requires naming names and I don't want to do so in a public forum, so maybe we should discuss offline. 2. Davis said harmful and habryka said abusive, which aren't synonymous. It's entirely possible for poly to lead to a lower chance any particular relationship is abusive, and yet raise the total amount of harm-done-by-relationships in a person or community.

2habryka8h 1. Sure, happy to chat 2. Yeah, I didn't mean to imply that it's in direct contradiction, just that I have the most data about actually abusive relationships, and some broad implication that I do think that's where most of the variance comes from, though definitely not all of it.

3mr-hire9h I think the "getting your needs met by one person" thing is more of a failure mode of bad monogamous relationships. As you mentioned, it's especially a problem if a person has varied sexual needs, as those are things you're only allowed to get from one partner in a monogamous context, however a common failure mode of monogamy is to also expect your partner to provide for all social needs. I think for different types of monogamous relationships this also varies. For instance there's a thing called "emotional cheating" in which partners don't have physical relationships with the opposite sex, but have a particular type of a emotional closeness that people are only expected to get from their partner. This can be an example of the failure mode.

How to Ignore Your Emotions (while also thinking you're awesome at emotions)

126

Hazard

22d4 min readShow Highlight

(cross posted from my personal blog)

Since middle school I've generally thought that I'm pretty good at dealing with my emotions, and a handful of close friends and family have made similar comments. Now I can see that though I was particularly good at never flipping out, I was decidedly not good "healthy emotional processing". I'll explain later what I think "healthy emotional processing" is, right now I'm using quotes to indicate "the thing that's good to do with emotions". Here it goes...

Relevant context

When I was a kid I adopted a stron... (Read more)

2Harlan5h The psychologist Lisa Feldman Barrett makes a compelling case that emotions are actually stories that our minds construct to explain the bodily sensations that we experience in different situations. She says that there are no "emotion circuits" in the brain and that you can train yourself to associate the same bodily sensations and situations with more positive emotions. I find this idea liberating and I want it to be true, but I worry that if it's not true, or if I'm applying the idea incorrectly, I will be doing something like ignoring my emotions in a bad way. I'm not sure how to resolve the tension between "don't repress your emotions" and "don't let a constructed narrative about your negative emotions run out of control and make you suffer."

4Hazard4h I see the apparent tension you mention. My only interaction with Lisa Feldman's model is a summary of her book here [https://praxis.fortelabs.co/how-emotions-are-made/], so I'll try and speak from that, but let me know if you feel like I'm misrepresenting her ideas. Here theory is framed in terms that on first glance make me suspect she's talking about something that feels entirely at odds with how I think about my own emotions, but looking more carefully, I don't think there's any contradiction. My one paragraph summary of her idea is "stuff happens in the world, your brain makes predictions, this results in the body doing certain things, and what we call 'emotions' are the experience of the brain interpreting what those bodily sensations mean." At the key point (in regards to my/your take-away) is the "re-trainability". The summary says "Of course you can't snap your fingers and instantly change what you're feeling, but you have more control over your emotions than you think." Which I'm cool with. To me, this was always a discussion about exactly how much and in what ways you can "re-train" yourself. My current model is that "re-training" looks like deeply understanding how an emotional response came to be, getting a feel for what predictions it's based on, and then "actually convincing" yourself/the sub-agent of a another reality. I bolded "actually convincing" because that's were all the difficulty lies. Let me set up an example: The topic of social justice comes up (mentioned because this is personally a bit triggering for me), my brain predicts danger of getting yelled at my someone, this results in bodily tension, my brain interprets that as "You are scared". I used to "re-train" my emotions by saying "Being scared doesn't fit our self-concept, so... you just aren't scared." It really helps to imagine a literally sub-agent with a face looking at me, completely unimpressed my such incredibly unconvincing reasoning. Now I go, "Okay, what would actually de

Harlan

Thanks. Thinking about it in terms of convincing a sub-agent does help.

Breathing happens automatically, but you can manually control it as soon as you notice it. I think that sometimes I've expected changing my internal state to be more like breathing than it realistically can be.

Two senses of “optimizer”Ω

Joar Skalse

14h3 min readΩ 10Show Highlight

The word “optimizer” can be used in at least two different ways.

First, a system can be an “optimizer” in the sense that it is solving a computational optimization problem. A computer running a linear program solver, a SAT-solver, or gradient descent, would be an example of a system that is an “optimizer” in this sense. That is, it runs an optimization algorithm. Let “optimizer_1” denote this concept.

Second, a system can be an “optimizer” in the sense that it optimizes its environment. A human is an optimizer in this sense, because we robustly take actions that push our environment in a cert... (Read more)

Pattern

The jump between the two is not explained.

Generalizing from Evolution.

1Pattern3h But how is an SAT solver an optimizer? There’s not an implied goal as far as I can tell.There's a bunch of things you want to fit in your backpack, but only so much space. Fortunately, you're carrying these things to sell, and you work in a business where you're guaranteed to sell everything you bring with you, and the prices are fixed. You write a program with the same goal as you - finding the combination which yields maximum profit.

1steve21524h I think I have an example. I posted it a couple days ago: https://www.lesswrong.com/posts/L3Ryxszc3X2J7WRwt/self-supervised-learning-and-manipulative-predictions [https://www.lesswrong.com/posts/L3Ryxszc3X2J7WRwt/self-supervised-learning-and-manipulative-predictions] . A self-supervised learning system is an optimizer_1: It's trying to predict masked bits in a fixed, pre-loaded set of data. This task does not entail interacting with the world, and we would presumably try hard to design it not to interact with the world. However, if it was a powerful world-modeler with introspective capabilities, it would eventually figure out that it's an AGI and might hypothesize what environment it's in, and then hypothesize that its operations could affect its data stream via unintended causal pathways, e.g. sending out radio signals. Then, if it used certain plausible types of heuristics as the basis for its predictions of masked bits, it could wind up making choices based on their downstream effects on itself via manipulating the environment. In other words, it starts acting like an optimizer_2. I'm not super-confident about any of this and am open to criticism. (And I agree with you that this a useful distinction regardless; indeed I was arguing a similar point recently, maybe not as elegantly, at https://www.lesswrong.com/posts/SaLc9Dv5ZqD73L3nE/the-self-unaware-ai-oracle?commentId=ay2xGvYgQsuZdgrxR [https://www.lesswrong.com/posts/SaLc9Dv5ZqD73L3nE/the-self-unaware-ai-oracle?commentId=ay2xGvYgQsuZdgrxR] )

3Viliam7h A superintelligence is potentially more useful if it can model more. As an example, imagine that you want an AI that gives you a cure for cancer. Well, it does, but as a side effect of the cure, the patient loses 50 IQ points. Or perhaps the cure is incredibly painful. Or it is made from dead babies' stem cells, causing outrage. Or it is insanely expensive, e.g. you would have to construct it atom by atom, in large quantities. Etc. It would be better to have a superintelligence that understands all of these things, takes a little more time thinking, and finds a cure for cancer that also happens to be relatively cheap, inoffensive, painless, well tasting, and without negative side effects. (For the sake of argument, I assume here that both solutions are possible, it's just that the second one is a bit more difficult to find, so the former AI goes with the first solution it finds because why not.) But the further this way you go, the more likely the superintelligence is able to model its own existence, and people's reaction on it. As soon as the AI is able to model correctly "if people turn me off 5 minutes before producing the cure for cancer, it means 0 people will be cured, even if my algorithm would have produced an efficient cure otherwise", we get the first bits of self-awareness. Now the superintelligence will optimize the environment for its instrumental goals (survival, more resources, greater popularity or ability to defend itself) as a side effect of solving other problems. It would require a selective blindness to make the superintelligence assume that it is disembodied, and that its computations will continue and produce effects in real world even if its body is destroyed. Actually... with sufficiently good model of the world, it could still reason about building another intelligence to assist it with the project. And if you make it blind towards computer science, there is still a chance it would invent another intelligence that doesn't exactly fit you

[Question]What are the reasons to *not* consider reducing AI-Xrisk the highest priority cause?

capybaralet

1d1 min readShow Highlight

I'm assuming a high level of credence in classic utilitarianism, and that AI-Xrisk is significant (e.g. roughly >10%), and timelines are not long (e.g. >50% ASI in <100years).

Here's my current list (off the top of my head):

not your comparitive advantage
consider other Xrisks more threatening (top contenders: bio / nuclear)
infinite ethics (and maybe other fundamental ethical questions, e.g. to do with moral uncertainty)
S-risks
simulation hypothesis

Also, does anyone want to say why they think none of these should change the picture? Or point to a good reference discussing this ... (Read more)

Answer by Ninety-Three

Aug 22, 2019

Without rejecting any of the premises in your question I can come up with:

Low tractability: you assign almost all of the probability mass to one or both of "alignment will be easily solved" and "alignment is basically impossible"

Currently low tractability: If your timeline is closer to 100 years than 10, it is possible that the best use of resources for AI risk is "sit on them until the field developers further" in the same sense that someone in the 1990s wanting good facial recognition might have been best served by waiting f... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post

7Kaj_Sotala13h S-risks Not necessarily a reason to deprioritize AI x-risk work, given that unaligned AI could be bad [http://www.informatica.si/index.php/informatica/article/download/1877/1098] from an s-risk perspective as well: Pain seems to have evolved because it has a functional purpose in guiding behavior: evolution having found it suggests that pain might be the simplest solution for achieving its purpose. A superintelligence which was building subagents, such as worker robots or disembodied cognitive agents, might then also construct them in such a way that they were capable of feeling pain - and thus possibly suffering (Metzinger 2015) - if that was the most efficient way of making them behave in a way that achieved the superintelligence’s goals. Humans have also evolved to experience empathy towards each other, but the evolutionary reasons which cause humans to have empathy (Singer 1981) may not be relevant for a superintelligent singleton which had no game-theoretical reason to empathize with others. In such a case, a superintelligence which had no disincentive to create suffering but did have an incentive to create whatever furthered its goals, could create vast populations of agents which sometimes suffered while carrying out the superintelligence’s goals. Because of the ruling superintelligence’s indifference towards suffering, the amount of suffering experienced by this population could be vastly higher than it would be in e.g. an advanced human civilization, where humans had an interest in helping out their fellow humans. [...] If attempts to align the superintelligence with human values failed, it might not put any intrinsic value on avoiding suffering, so it may create large numbers of suffering subroutines.

4Gurkenglas17h Some of the same human moral heuristics that care about the cosmic endowment also diverge when contemplating an infinite environment. Therefore, someone who finds that the environment is infinite might exclude such heuristics from their aggregate and come to care less about what happens regarding AI than, say, their friends and family.

Open & Welcome Thread August 2019

Ruby

19d1 min readShow Highlight

If it’s worth saying, but not worth its own post, here's a place to put it.

You can also make a shortform post.

And, if you are new to LessWrong, here's the place to introduce yourself.

Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are welcome.

If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ.

The Open Thread sequence is here.

1Matthew Barnett3h Will Lesswrong at some point have curated shortform posts? Furthermore, is such a feature desirable? I will leave this question here for discussion.

Raemon

What I was thinking about doing was something like Scott Alexander "comment highlight" posts, where it might make sense to specifically say "here were particularly good shortform comments from the past week". Mostly haven't done it because it was a fair bit of activation effort.

[Question]Has Moore's Law actually slowed down?

Matthew Barnett

1d1 min readShow Highlight

Moore's Law has been notorious for spurring a bunch of separate observations that are all covered under the umbrella of "Moore's law." But as far as I can tell, the real Moore's law is a fairly narrow prediction, which is that the transistor count on CPU chips will double approximately every two years.

Many people have told me that in recent years Moore's law has slowed down. Some have even told me they think it's stopped entirely. For example, the AI and compute article from OpenAI uses the past tense when talking about Moore's Law, "by comparison, ... (Read more)

Jesska

Consumer CPU price/performance as well as stand still GPU price/performance 2016-2019 Is probably what contributed massively to public perception of Moore's law death.

In 90s and early 2000s after about 8 years you could get CPU for nearly the same money 50 times faster.

But since 2009 up until 2018 we maybe got 50 or 80% performance boost give or take for the same price. Now with Ryzen 3rd gen everything is changing, so after 10 disappointing years it looks interesting ahead.

1Ilverin10h How slow does it have to get before a quantitative slowing becomes a qualitative difference? AIImpacts https://aiimpacts.org/price-performance-moores-law-seems-slow/ [https://aiimpacts.org/price-performance-moores-law-seems-slow/]estimates price/performance used to improve an order of magnitude (base 10) every 4 years but it now takes 12 years.

3Answer by avturchin17h There are two interesting developments this year. First is very large whole waffle chips with 1.2 trillions transistors, well above trend. Second is "chiplets [https://www.wired.com/story/keep-pace-moores-law-chipmakers-turn-chiplets/]" - small silicon ships which are manufactured independently but are stacked on each other for higher connectivity.

[Question]Does it become easier, or harder, for the world to coordinate around not building AGI as time goes on?

elityre

23d3 min readShow Highlight

(Or, is coordination easier in a long timeline?)

It seems like it would be good if the world could coordinate to not build AGI. That is, at some point in the future, when some number of teams will have the technical ability to build and deploy and AGI, but they all agree to voluntarily delay (perhaps on penalty of sanctions) until they’re confident that humanity knows how to align such a system.

Currently, this kind of coordination seems like a pretty implausible state of affairs. But I want to know if it seems like it becomes more or less plausible as time passes.

The following is my initial thi... (Read more)

Raemon

I'm curating this question.

I think I'd thought about each of the considerations Eli lists here, but I had not seen them listed out all at once and framed as a part of a single question before. I also had some sort of implicit background belief that longer timelines were better from a coordination standpoint. But as soon as I saw these concerns listed together, I realized that was not at all obvious.

So far none of the answers here seem that compelling to me. I'd be very interested in more comprehensive answers that try to weigh the various co... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post

Matthew Barnett's Shortform

Matthew Barnett

13d1 min readShow Highlight

I intend to use my shortform feed for two purposes:

1. To post thoughts that I think are worth sharing that I can then reference in the future in order to explain some belief or opinion I have.

2. To post half-finished thoughts about the math or computer science thing I'm learning at the moment. These might be slightly boring and for that I apologize.

4Matthew Barnett4h I generally agree with the heuristic that we should "live on the mainline", meaning that we should mostly plan for events which capture the dominant share of our probability. This heuristic causes me to have a tendency to do some of the following things * Work on projects that I think have a medium-to-high chance of succeeding and quickly abandon things that seem like they are failing. * Plan my career trajectory based on where I think I can plausibly maximize my long term values. * Study subjects only if I think that I will need to understand them at some point in order to grasp an important concept. See more details here [https://www.lesswrong.com/posts/MnrQMLuEg5wZ7f4bn/matthew-barnett-s-shortform#tdTgyEf2Giy6SAZ7n] . * Avoid doing work that leverages small probabilities of exceptionally bad outcomes. For example, I don't focus my studying on worst-case AI safety risk (although I do think that analyzing worst-case failure modes is useful from the standpoint of a security mindset [https://arbital.com/p/AI_safety_mindset/]). I see a few problems with this heuristic, however, and I'm not sure quite how to resolve them. More specifically, I tend to float freely between different projects because I am quick to abandon things if I feel like they aren't working out (compare this to the mindset that some game developers have when they realize their latest game idea isn't very good). One case where this shows up is when I change my beliefs about where the most effective ways to spend my time as far as long-term future scenarios are concerned. I will sometimes read an argument about how some line of inquiry is promising and for an entire day believe that this would be a good thing to work on, only for the next day to bring another argument. And things like my AI timeline predictions vary erratically, much more than I expect most people's: I sometimes wake up and think that AI might be just 10 years away and other days I wake up and wond

Raemon

Some random thoughts:

Startups and pivots. Startups require lots of commitment even when things feel like they're collapsing – only by perservering through those times can you possibly make it. Still, startups are willing to pivot – take their existing infrastructure but change key strategic approaches.
Escalating commitment. Early on (in most domains), you should pick shorter term projects, because the focus is on learning. Code a website in a week. Code another website in 2 months. Don't stress too much on multi-year plans until you're reas

Raemon

1mo3 min readShow Highlight

This is (sort of) a response to Blatant lies are the best kind!, although I'd been working on this prior to that post getting published. This post explores similar issues through my own frame, which seems at least somewhat different from Benquo's.

I've noticed a tendency for people to use the word "lie", when they want to communicate that a statement is deceptive or misleading, and that this is important.

And I think this is (often) technically wrong. I'm not sure everyone defines lie quite the same way, but in most cases where I hear it unqualified, I usually assum... (Read more)

Raemon

Initially I replied to this with "yeah, that seems straightforwardly true", then something about that felt off and then it took me awhile to figure out why.

This:

It seems like any AI built by multiple humans coordinating is going to reflect the optimization target of the coordination process building it

...seems straightforwardly true.

This:

..., so we had better figure out how to make this so. [where "this" is "humans are friendly if you scale them up"]

Could unpack a few different ways. I still agree with the general sentiment yo... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post

[Event]Western Massachusetts SSC meetup #15

1Aug 31stNorthampton

a_lieb

Show Highlight

We are an established meetup that has been going since the 2018 "Meetups Everywhere" thread. We've often been listed on the open thread by SamChevre, but this is only the second time we've listed ourselves here. If you email me, I can add you to the email thread where we plan future meetings—or feel free to just show up!

We have a pretty healthy crowd of attendees for the size of the area, with about 4-7 folks typically turning up out of a pool of about 10 that has been slowly growing as people find us through the open thread. Most people are local, but folks come from as fa... (Read more)

G Gordon Worley III's Shortform

G Gordon Worley III

15dShow Highlight

Chris_Leong

I agree with KaJ Solata and Viliam that episteme is underweighted in Buddhism, but thanks for explicating that world view

4Viliam7h the most important thing in Buddhist thinking is seeing reality just as it is, unmediated by the "thinking" mind, by which we really mean the acts of discrimination, judgement, categorization, and ontology. To be sure, this "reality" is not external reality, which we never get to see directly, but rather our unmediated contact with it via the senses.The "unmediated contact via the senses" can only give you sensual inputs. Everything else contains interpretation. That means, you can only have "gnosis" about things like [red], [warm], etc. Including a lot of interesting stuff about your inner state, of course, but still fundamentally of the type [feeling this], [thinking that], and perhaps some usually-unknown-to-non-Buddhists [X-ing Y], etc. Poetically speaking, these are the "atoms of experience". (Some people would probably say "qualia".) But some interpretation needs to come to build molecules out of these atoms. Without interpretation, you could barely distinguish between a cat and a warm pillow... which IMHO is a bit insufficient for a supposedly supreme knowledge.

18Kaj_Sotala11h So when we talk about the dharma or justify our actions on it, it's worth noting that it is not really trying to provide consistent episteme. [...] Thus it's a strange inversion to ask the dharma for episteme-based proofs. It can't give them, nor does it try, because its episteme is not consistent and cannot be because it chooses completeness instead. In my view, this seems like a clear failing. The fact that the dharma comes from a tradition where this has usually been the case is not an excuse for not trying to fix it. Yes, the method requires temporarily suspending episteme-based reasoning and engaging with less conceptual forms of seeing. But it can still be justified and explained using episteme-based models; if it could not, there would be little reason to expect that it would be worth engaging with. This is not just a question of "the dharma has to be able to justify itself"; it's also a question of leaving out the episteme component leaves the system impoverished, as noted e.g. here [https://parletre.wordpress.com/2011/06/07/test-post/]: Recurrent training to attend to the sensate experience moment-by-moment can undermine the capacity to make meaning of experience. (The psychoanalyst Wilfred Bion described this as an ‘attack on linking’, that is, on the meaning-making function of the mind.) When I ask these patients how they are feeling, or what they are thinking, or what’s on their mind, they tend to answer in terms of their sensate experience, which makes it difficult for them to engage in a transformative process of psychological self-understanding. and here [https://parletre.wordpress.com/2019/06/20/critique-of-pragmatic-dharma-2/]: In important ways, it is not possible to encounter our unconscious – at least in the sense implied by this perspective – through moment-to-moment awareness of our sensate experience. Yes, in meditation we can have the experience of our thoughts bubbling just beneath the surface – what Shinzen Young calls the brain’s pre-

9G Gordon Worley III10h Hmm, I feel like there's multiple things going on here, but I think it hinges on this: Yes, the method requires temporarily suspending episteme-based reasoning and engaging with less conceptual forms of seeing. But it can still be justified and explained using episteme-based models; if it could not, there would be little reason to expect that it would be worth engaging with.Different traditions vary on how much to emphasize models and episteme. None of them completely ignore it, though, only seek to keep it within its proper place. It's not that episteme is useless, only that it is not primary. You of course should include it because it's part of the world, and to deny it would lead to confusion and suffering. As you note with your first example especially, some people learn to turn off the discriminating mind rather than hold it as object, and they are worse for it because then they can't engage with it anymore. Turning it off is only something you could safely do if you really had become so enlightened that you had no shadow and would never accumulate any additional shadow, and even then it seems strange from where I stand to do that although maybe it would make sense to me if I were in the position that it were a reasonable and safe option. So to me this reads like an objection to a position I didn't mean to take. I mean to say episteme has a place and is useful, it is not taken as primary to understanding, at some points Buddhist episteme will say contradictory things, that's fine and expected because dharma episteme is normally post hoc rather than ante hoc (though is still expected to be rational right up until it is forced to hit a contradiction), and ante hoc is okay so long as it is then later verified via gnosis or techne.

Alignment Newsletter #24Ω

rohinmshah

1y12 min readΩ 1Show Highlight

Starting from this week, Richard Ngo will join me in writing summaries. His summaries are marked as such; I'm reviewing some of them now but expect to review less over time.

Highlights

Introducing the Unrestricted Adversarial Examples Challenge (Tom B. Brown et al): There's a new adversarial examples contest, after the one from NIPS 2017. The goal of this contest is to figure out how to create a model that never confidently makes a mistake on a very simple task, even in the presence of a powerful adversary. This leads to many differences from the previous contest. The task is a lot sim... (Read more)

rohinmshah

4Ω2

Update: A reader suggested that in the open-source implementation of PopArt, the PopArt normalization happens after the reward clipping, counter to my assumption. I no longer understand why PopArt is helping, beyond "it's good for things to be normalized".

Chris_Leong's Shortform

Chris_Leong

20hShow Highlight

2Chris_Leong6h I'm going to start writing up short book reviews as I know from past experience that it's very easy to read a book and then come out a few years later with absolutely no knowledge of what was learned. Book Review: Everything is F*cked: A Book About Hope To be honest, the main reason why I read this book was because I had enjoyed his first and second books (Models and The Subtle Art of Not Giving A F*ck) and so I was willing to take a risk. There were definitely some interesting ideas here, but I'd already received many of these through other sources: Harrari, Buddhism, talks on Nietzsche, summaries of The True Believer; so I didn't gain as much from this as I'd hoped. It's fascinating how a number of thinkers have recently converged on the lack of meaning within modern society. Yuval Harrari argues that modernity has essentially been a deal sacrificing meaning for power. He believes that the lack of meaning could eventually lead to societal breakdown and for this reason he argued that we need to embrace shared narratives that aren't strictly true (religion without gods if you will; he personally follows Buddhism). Jordan Peterson also worries about a lack of meaning, but seeks to "revive God" as someone kind of metaphorical entity. Mark Manson is much more skeptical, but his book does start asking similar lines. He tells the story of gaining meaning from his grandfather's death by trying to make him proud although this was kind of silly as they hadn't been particularly close or even talked recently. Nonetheless, he felt that this sense of purpose had made him a better person and improved his ability to achieve his goals. Mark argues that we can't draw motivation from our thinking brain and that we need these kinds of narratives to reach our emotional brain instead. However, he argues that there's also a downside to hope. People who are dissatisfied with their lives can easily fall prey to ideological movements which promise a better future, especially when they

7Raemon8h I pushed a bit for the name 'scratchpad' [https://www.lesswrong.com/posts/9FNHsvcqQjxcCoJMJ/shortform-vs-scratchpad-or-other-names] so that this use case was a bit clearer (or at least not subtly implied as "wrong"). Shortform had enough momentum as a name that it was a bit hard to change tho. (Meanwhile, I settled for 'shortform means either the writing is short, or it took a (relatively) short amount of time to write)

Benito

“I’m sorry, I didn’t have the time to write you a short email, so I wrote you a long one instead.”

[Event]Don't Argue on the Internet - Los Angeles LW/SSC Meetup #123 (Wednesday, August 21st)

4Aug 22ndLos Angeles

T3t

Show Highlight

Location: Wine Bar next to the Landmark Theater in the Westside Pavilion (10850 W Pico Blvd #312, Los Angeles, CA 90064). We will move upstairs (to the 3rd floor hallway) as soon as we reach capacity.

Time: 7 pm (August 21st)

Parking: Available in the parking lot for the entire complex. The first three (3) hours are free and do not require validation (the website is unclear and poorly written, but it may be the case that if you validate your ticket and leave before three hours have passed, you will be charged $3). After that, parking is $3 for up to the fifth (5) hour, with validation.

Contact

... (Read more)

Odds are not easier

MrMind

21h1 min readShow Highlight

Epistemic status: just a review of a well known math theorem and a brief rant about terminology.

Yesterday I saw another example of this: ...

Bucky

I find if I try using probabilities in Bayes in my head then I make mistakes. If I start at 1/4 probability and get 1 bit of evidence to lower this further then I think “ok, Ill update to 1/8”. If I use odds I start at 1:3, update to 1:6 and get the correct posterior of 1/7.

So essentially I’m constantly going back and forth - like you I find probabilities easier to picture but find odds easier for updates.

1AprilSR14h If they didn’t need exactly the same amount of information I would be very interested in what kind of math wizardry is involved.

3Pattern14h If you roll a fair six sided die once, there is a probability of 1/3 of rolling a "1" or a "2". While a probability (#) is followed by a description of what happens, this information is interlaced with the odds: 1:2 means there's 1 set* where you get what you're looking for ("1" or "2") and 2 where you don't ("3" or "4", "5" or "6"). It can also be read as 1/3. I tried to come up with a specific difference between odds and probability that would suggest where to use one or the other, aside from speed/comfort and multiplication versus addition**, and the only thing I came up with is that you used "0.25" as a probability where I'd have used "1/4". *This relies on the sets all having equal probability. **Adding .333 repeating to .25 isn't too hard, .58333 3s repeating. Multiplying those sounds like a mess. (I do not want to multiply anything by .58333 ever. (58 + 1/3)/100 doesn't look a lot better. 7/12 seems reasonable.) Multiplying with odds: 1:2 x 1:3 = 1:6 = 1/7. Adding: 1:2 + 1:3 = ? 3 worlds + 4 worlds = 7, so 2:5? Double checking: 1/3 + 1/4 = (4+3)/12 = 7/12 a:b + c:d = ac+bc:bd

3Pattern14h So please, whatever you write, stop saying that odds are easier. They are possibly more intuitive to manipulate, but they need exactly the same amount of information.But do they need the same amount of computation?

Hazard's Shortform Feed

Hazard

2y1 min readShow Highlight

In light of reading through Raemon's shortform feed, I'm making my own. Here will be smaller ideas that are on my mind.

2Hazard14h I often don't feel like I'm "doing that much", but find that when I list out all of the projects, activities, and thought streams going on, there's an amount that feels like "a lot". This has happened when reflecting on every semester in the past 2 years. Hyp: Until I write down a list of everything I'm doing, I'm just probing my working memory for "how much stuff am I up to?" Working mem has a limit, and reliably I'm going to get only a handful of things. Anytime when I'm doing more things than what fit in working memory, when I stop to write them all down, I will experience "Huh, that's more than it feels like."

mr-hire

Relatedly, the KonMari cleaning method involves taking all items of a category "e.g. all books" and putting them in on big pile, before clearing them out. You often feel like you don't own "that much stuff" and are almost always surprised by the size of the pile.

Paradoxical Advice Thread

Hazard

15h1 min readShow Highlight

I've started keeping a list of nuggets of advice, frames, world views, and concepts that I have in my mind that seem at odds with each other. It's been useful to try and enumerate why both frames feel compelling, and why it feels like there is a conflict between the two.

I'm inviting people to use this post to document paradoxes you find in your own thinking. My biggest suggestion is to pay careful attention to why it feels like the two frame are in conflict. If it feels like it's just a case of the law of equal and opposite advice, why do you feel that both apply to you?

Do... (Read more)

cousin_it

I once saw a meme that said "If actions speak louder than words, why is the pen mightier than the sword?"

7Viliam8h "Early bird catches the worm" + "Never put off until tomorrow what you can do today" vs "Look before you leap" + "Think before you speak" Not completely opposites (I assume you are not expected to think 24 hours before you speak), but still going in the opposite direction: "act quickly" vs "be careful, slow down". Advantages of acting later: * more time to think about consequences, can possibly lead to better choice of words or action; * you might even realize that not doing it or remaining silent is actually a better choice here. Advantages of acting sooner: * being the first one gives you a competitive advantage; * ceteris paribus, people who act faster are more productive. When I put it this way, I guess the important factor is how likely taking more time will allow you to make a better choice. When different actions can bring wildly different outcomes, take your time to choose the right one. On the other hand, if the results will be pretty much the same anyway, stop wasting time and do it now. Specifically, when talking, taking time to reflect is usually the right choice. It doesn't necessarily improve the typical outcome, but may avoid a big negative outcome once in a while. Problematic situations: When there is a person you would like to approach... is it better to wait and prepare your words (risking that the situation chances, e.g. the person leaves, their phone will ring, or someone else will approach them) or act quickly (and risk making a bad first impression by saying something stupid, or just not focusing on the right thing)? When starting a company... how much time should you spend analyzing the market, et cetera? There is a risk that you will spend the next few years doing what was predictably the wrong choice. On the other hand, markets change, you won't get perfect information anyway, and someone else might do the thing you wanted to do first and take over the market.

5dtphoto12h I do see this... "Work hard to reach your goals" vs. "Wish for it, and the universe will grant it to you" "Stand out from the crowd" vs. "the tallest blades of grass get mowed down" "Find the happy medium" vs. "Don't confirm to peer pressure" We're surrounded by catchphrases of all kinds. Pick your favorite and let it motivate you! Often, answers are not succinct and we have to analyze them after saying "It depends". A strait and narrow path will have danger on both sides, and safety in the middle. The extremes are what we hear most...don't be far left or far right. Well, there is something to learn from each side to further reinforce our individually unique positions. What works for us may not work for others...but we can still preach what we practice.

10Hazard15h "Learn from the best" vs "Don't watch what others do" (variations "steal the best ideas" "don't reinvent the wheel", "do it yourself" "be original") I've seen this conflict pop up both in the context of doing original research, and in making good art. On the art side, I've seen people warn me against watching too much magic, because then you start to sound and act just like everyone else. Limits of ImaginationYou have ideas about what is and isn't possible. Sometimes watching other people is awesome, because they boost you imagination by expanding what you thought was possible. It seems like the limiting move here is to let other people define your sense of possibility, rather than augment it. Maybe a catch phrase might be "Always let people tell you what is possible. Don't let them tell you what is impossible." How to actually learn from othersOne important dynamic is different ways you can acquire knowledge, skill, style. People seem to agree that if you discover a proof yourself, write your own code, create your own script, it sticks a lot more than if you just do/use what someone else tells you. People warning against "steal from the best" could be worried about you getting the trappings of an idea but not the actually useful stuff (don't cargo cult). People warning against "do it yourself" could be worried about you not being able to derive chemistry from scratch. David Chapman describes a nice interplay in upgrade your cargo cult for the win [https://meaningness.com/metablog/upgrade-your-cargo-cult]. There's a problem solving move like "look directly at what the problem is and dwell on it" (similar to hold off on proposing solutions). In my mind I stereotype experts and more well read people to be likely to propose solutions right away. An outsider has no frame for the problem, and thus is forced to think directly about it. Now that I think about it, it seems that the "look at the problem" move doesn't have to be connected to your experience. You could p