Recent Discussion

The strategy-stealing assumptionΩ
541mo11 min readΩ 20Show Highlight

Suppose that 1% of the world’s resources are controlled by unaligned AI, and 99% of the world’s resources are controlled by humans. We might hope that at least 99% of the universe’s resources end up being used for stuff-humans-like (in expectation).

Jessica Taylor argued for this conclusion in Strategies for Coalitions in Unit-Sum Games: if the humans divide into 99 groups each of which acquires influence as effectively as the unaligned AI, then by symmetry each group should end, up with as much influence as the AI, i.e. they should end up with 99% of the influence.

This argument rests on what I... (Read more)

This ar­gu­ment rests on what I’ll call the strat­egy-steal­ing as­sump­tion: for any strat­egy an un­al­igned AI could use to in­fluence the long-run fu­ture, there is an analo­gous strat­egy that a similarly-sized group of hu­mans can use in or­der to cap­ture a similar amount of flex­ible in­fluence over the fu­ture.

The word "assumption" in "strat­egy-steal­ing as­sump­tion" keeps making me think that you're assuming this as a proposition and deriving consequences from it, but the actual assumption you're making is more like "it's a good idea to pick

... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post
The Parable of Predict-O-MaticΩ
1103d13 min readΩ 30Show Highlight

I've been thinking more about partial agency. I want to expand on some issues brought up in the comments to my previous post, and on other complications which I've been thinking about. But for now, a more informal parable. (Mainly because this is easier to write than my more technical thoughts.)

This relates to oracle AI and to inner optimizers, but my focus is a little different.

1

Suppose you are designing a new invention, a predict-o-matic. It is a wonderous machine which will predict everything for us: weather, politics, the newest advances in quantum physics, you name it. The machi... (Read more)

[1] Whoever gets control of the share gets control of the company for one year, and gets dividends based on how well the company did that year. [2] Each person bids based on what they expect they could make. [3] So the highest bidder is the person who can run the company the best, and they can't be out-bid. [4] So, you get the best possible person to run your company, and they're incentivized to do their best, so that they get the most money at the end of the year.

[3] doesn't seem to follow. The person who wins an auction is usually the per... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post

Below I will present a (small but qualitative ) list of those that I think are some of the best sites/blog that a human being can find on the world wide web.

The main criterion I used to draw up the list was to consider how the websites promote the dissemination of knowledge among people and how, over the course of time, they have helped me both with regards to work and in terms of intellectual self-formation. The order in which they are listed is not to be considered restrictive ( except perhaps for the first two ).

Please feel free to criticize the catalog (as long as the criticisms are r... (Read more)

This is a great list.

The main criticism I have is that this list overlaps way too much with my own internal list of high-quality sites, making it not very useful.

Gradient hackingΩ
412d3 min readΩ 19Show Highlight

"Gradient hacking" is a term I've been using recently to describe the phenomenon wherein a deceptively aligned mesa-optimizer might be able to purposefully act in ways which cause gradient descent to update it in a particular way. In Risks from Learned Optimization, we included the following footnote to reflect this possibility:

Furthermore, a deceptively aligned mesa-optimizer would be incentivized to cause there to be a systematic bias in the direction of preventing the base optimizer from modifying its mesa-objective. Thus, in the context of a local optimization process, a deceptive mesa-o

... (Read more)

It occurs to me that despite my lack of expertise in alignment, I am really enjoying the mesa optimization conversation because it does a good job of contextualizing machine learning performance. This is good even for narrow applications.

1Gurkenglas8h That obvious trick relies on that we will only verify its prediction for strategies that it recommends. Here's a protocol that doesn't fail to it: A known number of gems are distributed among boxes, we can only open one box and want many gems. Ask for the distribution, select one gem at random from the answer and open its box. For every gem it hides elsewhere, selecting it reveals deception.

Introduction

Epistemic spot checks are a series in which I select claims from the first few chapters of a book and investigate them for accuracy, to determine if a book is worth my time. This month’s subject is The Fall of Rome, by Bryan Ward-Perkins, which advocates for the view that Rome fell, and it was probably a military problem.

Like August’s The Fate of Rome, this spot check was done as part of a collaboration with Parallel Forecasting and Foretold, which means that instead of resolving a claim as true or false, I give a confidence distribution of what I think I would answer if I spent 1... (Read more)

I feel like this is an excellent middle ground between a full review and relying solely on the reputation of the author, and I am excited to see the eventual list of books which pass the epistemic spot checks.

Introduction

Internal Family Systems (IFS) is a psychotherapy school/technique/model which lends itself particularly well for being used alone or with a peer. For years, I had noticed that many of the kinds of people who put in a lot of work into developing their emotional and communication skills, some within the rationalist community and some outside it, kept mentioning IFS.

So I looked at the Wikipedia page about the IFS model, and bounced off, since it sounded like nonsense to me. Then someone brought it up again, and I thought that maybe I should reconsider. So I looked at the WP page again... (Read more)

This is confusing Dissociation and Integration. I made a 2x2 that helps disambiguate.

http://mattgoldenberg.net/wp-content/uploads/2019/10/2x2s-Integration-vs.-Association-2x2.jpg

3Kaj_Sotala6h Voila! The same three things (Exile, Firefighter, Manager), described in less text and without the need for a concept of "parts". If it was just that brief description, then sure, the parts metaphor would be unnecessary. But the IFS model contains all kinds of additional predictions and applications which make further use of those concepts. For example, firefighters are called that because "they are willing to let the house burn down to contain the fire"; that is, when they are triggered, they typically act to make the pain stop, without any regard for consequences (such as loss of social standing). At the same time, managers tend to be terrified of exactly the kind of lack of control that's involved with a typical firefighter response. This makes firefighters and managers typically polarized - mutually opposed - with each other. Now, it's true that you don't need to use the "part" expression for explaining this. But if we only talked about various behaviors getting reinforced, we wouldn't predict that the system simultaneously considers a loss of a social standing to be a bad thing, and that it also keeps reinforcing behaviors which cause exactly that thing. Now, obviously it can still be explained in a more sophisticated reinforcement model, in which you talk about e.g. differing prioritizations in different situations, and some behavioral routines kicking in at different situations... ...but if at the end, this comes down to there being two distinct kinds of responses depending on whether you are trying to avoid a situation or are already in it, then you need names for those two categories anyway. So why not go with "manager" and "firefighter" while you're at it? And sure, you could call it, say, "a response pattern" instead of "part" - but the response pattern is still physically instantiated in some collection of neurons, so it's not like "part" would be any less correct, or worse at reductionism. Either way, you still get a useful model of how those pat
3Kaj_Sotala8h Because this description creates a new entity for each thing that happens, such that the total number of entities under discussion is "count(subject matter) times count(strategies)" instead of "count(subject matter) plus count(strategies)". By simple math, a formulation which uses brain modules for strategies plus rules they operate on, is fewer entities than one entity for every rule+strategy combo. It seems to me that the emotional schemas that Unlocking the Emotional Brain talks about, are basically the same as what IFS calls parts. You didn't seem to object to the description of schemas; does your objection also apply to them? IFS in general is very vague about how exactly the parts are implemented on a neural level. It's not entirely clear to me what kind of a model you are arguing against and what kind of a model you are arguing for instead, but I would think that IFS would be compatible with both. Similarly, you can model most types of self-distraction behaviors as simple negative reinforcement learning: i.e., they make pain go away, so they're reinforced. So you get "firefighting" for free as a side-effect of the brain being able to learn from reinforcementI agree that reinforcement learning definitely plays a role in which parts/behaviors get activated, and discussed that in some of my later posts [1 [https://www.lesswrong.com/posts/oJwJzeZ6ar2Hr7KAX/subagents-akrasia-and-coherence-in-humans] 2 [https://www.lesswrong.com/posts/7zQPYQB5EeaqLrhBh/subagents-neural-turing-machines-thought-selection-and] ]; but there need to be some innate hardwired behaviors which trigger when the organism is in sufficient pain. An infant which needs help cries; it doesn't just try out different behaviors until it hits upon one which gets it help and which then gets reinforced. And e.g. my own compulsive behaviors tend to have very specific signatures which do not fit together with your description; e.g. a desire to keep playing a game can get "stuck on" way past the ti

Let me clarify what I mean when I say that math consists of nouns and verbs. Think about elementary school mathematics like addition and subtraction. What you learn to do is take a bunch of nouns—1, 2, 3, etc.—and a bunch of verbs—addition, subtraction—and make sentences. “1 + 2 = 3.”

When you make a sentence like that, what you're doing is taking an object, 1, and observing how it changes when it interacts—specifically, adds—with another object, 2. You observe that becomes a 3. Just like how you can observe a person (object) bump the... (Read more)

Thanks for this sequence.

(a)
384d1 min readShow Highlight

Cross-posted to my personal blog.


For a while now, I've been using "(a)" notation to denote archived versions of linked pages. This is a small effort towards creating Long Content (a) – content that has a lifespan of decades or centuries, rather than months or years.

I think basically anyone whose writing includes links to other work should include archived links alongside the original hyperlinks, if the writing is intended to be long-lived. (And if you're not trying to write long-lived content, what are you doing, even?)

I was happy to see Zuck (a) & Guzey (a) usin... (Read more)

5ioannes_shade18h Thanks, this is great. (And I didn't know about your Archiving URLs [https://www.gwern.net/Archiving-URLs] page!) And the functionality is one that will be rarely exercised by users, who will click on only a few links and will click on the archived version for only a small subset of said links, unless link rot is a huge issue - in which case, why are you linking to the broken link at all instead of the working archived version? I feel like I'm often publishing content with two audiences in mind – my present-tense audience and a future audience who may come across the post. The original link feels important to include because it's more helpful to the present-tense audience. e.g. Often folks update the content of a linked page in response to reactions elsewhere, and it's good to be able quickly point to the latest version of the link. The archived link is more aimed at the future audience. By the time they stumble across the post, the original link will likely be broken, and there's a better chance that the archived version will still be intact. (e.g. many of the links on Aaron Swartz's blog [http://www.aaronsw.com/weblog/fullarchive] are now broken; whenever I read it I find myself wishing there were convenient archived versions of the links).
6gwern16h Certainly there are links which are regularly updated, like Wikipedia pages. They should be whitelisted. There are others which wouldn't make any sense to archive, stuff like services or tools - something like Waifu Labs [https://waifulabs.com/] which I link in several places wouldn't make much sense to 'archive' because the entire point is to interact with the service and generate images. But examples like blogs or LW pages make sense to archive after a particular timepoint. For example, many blogs or websites like Reddit lock comments after a set number of days. Once that's passed, typically nothing in the page will change substantially except to be deleted. I think most of my links to blogs are of that type. Even on LW, where threads can be necroed at any time, how often does anyone comment on an old post, and if your archived copy happens to omit some stray recent comments, how big a deal is that? Acceptable collateral damage compared to a website where 5 or 10% of links are broken and the percentage keeps increasing with time, I'd say... For this issue, you could implement something like a 'first seen' timestamp in your link database and only create the final archive & substituting after a certain time period - I think a period like 3 months would capture 99% of the changes which are ever going to be made, while not risking exposing readers to too much linkrot.
1ioannes_shade15h For this issue, you could implement something like a 'first seen' timestamp in your link database and only create the final archive & substituting after a certain time period - I think a period like 3 months would capture 99% of the changes which are ever going to be made, while not risking exposing readers to too much linkrot. This makes sense, but it takes a lot of activation energy. I don't think a practice like this will spread (like even I probably won't chunk out the time to learn how to implement it, and I care a bunch about this stuff). Plausibly "(a)" could spread in some circles – activation energy is low and it only adds 10-20 seconds of friction per archived link. But even "(a)" probably won't spread far (10-20 seconds of friction per link is too much for almost everyone). Maybe there's room for a company doing this as a service...

But even "(a)" probably won't spread far (10-20 seconds of friction per link is too much for almost everyone). Maybe there's room for a company doing this as a service...

If adoption is your only concern, doing it by website is hopeless in the first place. Your only choice is creating some sort of web browser plugin to do it automatically.

Open & Welcome Thread - October 2019
816d1 min readShow Highlight
  • If it’s worth saying, but not worth its own post, here's a place to put it.
  • And, if you are new to LessWrong, here's the place to introduce yourself.
    • Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are welcome.

If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ.

The Open Thread sequence is here.

36jasoncrawford18h Hi everyone. I've discovered the rationality community gradually over the last several years, starting with Slate Star Codex, at some point discovering Julia Galef on Twitter/Facebook, and then reading Inadequate Equilibria. I still have tons of material on this site to go through! I'm also the author of a blog, The Roots of Progress ( https://rootsofprogress.org [https://rootsofprogress.org]), about the history of technology and industry, and more generally the story of human progress.

I, like eigen, am also a fan of your blog! Welcome!

This is a response to Abram's The Parable of Predict-O-Matic, but you probably don't need to read Abram's post to understand mine. While writing this, I thought of a way in which I think things could wrong with dualist Predict-O-Matic, which I plan to post in about a week. I'm offering a $100 prize to the first commenter who's able to explain how things might go wrong in a sufficiently crisp way before I make my follow-up post.

Dualism

Currently, machine learning algorithms are essentially "Cartesian dualists" when it comes to themselves and their environment. (Not a philosophy major -- let

... (Read more)

What's going on when you try to model yourself thinking about the answer to this question?

If a system is analyzing (itself analyzing (itself analyzing (...))) , not realizing it's doing so, I suspect that it will come up with some best guess answer, but that answer will be ill-determined and dependent on implementation details. Thus a better approach would be to avoid asking self-unaware systems any question that requires that type of analysis!

For example, you can ask "Please output the least improbable scenario, according to your predictive world-model

... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post
1Bunthut7h One possibility is that it's able to find a useful outside view model such as "the Predict-O-Matic has a history of making negative self-fulfilling prophecies". This could lead to the Predict-O-Matic making a negative prophecy ("the Predict-O-Matic will continue to make negative prophecies which result in terrible outcomes"), but this prophecy wouldn't be selected for being self-fulfilling. And we might usefully ask the Predict-O-Matic whether the terrible self-fulfilling prophecies will continue conditional on us taking Action A.Maybe I misunderstood what you mean by dualism, but I dont think that's true. Say the Predict-O-Matic has an outside view model (of itself) like "The metal box on your desk (the Predict-O-Matic) will make a self-fullfilling prophecy that maximizes the number of paperclips". Then you ask it how likely it is that your digital records will survive for 100 years. It notices that that depends significantly on how much effort you make to secure them. It notices that that significantly depends on what the metal box on your desk tells you. It uses it's low-model resolution of what the box says. To work that out, it checks which outputs would be self-fulfilling, and then which of these leads to the most paperclips. The more unsecure your digital records are, the more you will invest in paper, and the more paperclips you will need. Therefore the metal box will tell you the lowest self-fulfilling propability for your question. Since that number is *self-fulfilling*, it is in fact the correct answer, and the Predict-O-Matic will answer with it. I think this avoids your argument that I contend that Predict-O-Matic doesn't know it will predict P = A at the relevant time. It would require time travel -- to know whether it will predict P = A, it will have to have made a prediction already, and but it's still formulating its prediction as it thinks about what it will predict. because it doesn't have to simulate itself in detail to know what the metal box
1steve21528h I disagree that self-unawareness / dualism should be the default assumption, for reasons I explained in this comment [https://www.lesswrong.com/posts/rvxcSc6wdcCfaX6GZ/two-senses-of-optimizer#HdLKkm8QDThH3nyo3] . In fact I think that making a system that knowably remains self-unaware through arbitrary increases in knowledge and capability would be a giant leap towards solving AI alignment. I have vague speculative ideas for how that might be done with a type-checking proof, again see that comment I linked.
4Lanrian9h If dualism holds for Abram’s prediction AI, the “Predict-O-Matic”, its world model may happen to include this thing called the Predict-O-Matic which seems to make accurate predictions—but it’s not special in any way and isn’t being modeled any differently than anything else in the world. Again, I think this is a pretty reasonable guess for the Predict-O-Matic’s default behavior. I suspect other behavior would require special code which attempts to pinpoint the Predict-O-Matic in its own world model and give it special treatment (an “ego”). I don't see why we should expect this. We're told that the Predict-O-Matic is being trained with something like sgd, and sgd doesn't really care about whether the model it's implementing is dualist or non-dualist; it just tries to find a model that generates a lot of reward. In particular, this seems wrong to me: The Predict-O-Matic doesn't care about looking bad, and there's nothing contradictory about it predicting that it won't make the very prediction it makes, or something like that.If the Predict-O-Matic has a model that makes bad prediction (i.e. looks bad), that model will be selected against. And if it accidentally stumbled upon a model that could correctly think about it's own behaviour in a non-dualist fashion, and find fixed points, that model would be selected for (since its predictions come true). So at least in the limit of search and exploration, we should expect sgd to end up with a model that finds fixed points, if we train it in a situation where its predictions affect the future. If we only train it on data where it can't affect the data that it's evaluated against, and then freeze the model, I agree that it probably won't exhibit this kind of behaviour; is that the scenario that you're thinking about?

I’m happy to announce a semi-public beta of Foretold.io for the EA/LessWrong community. I’ve spent much of the last year working on coding & development, with lots of help by Jacob Lagerros on product and scoring design. Special thanks to the Long-Term Future Fund and it’s donors, who’s contribution to the project helped us to hire contractors to do much of the engineering & design.

You can use Foretold.io right away by following this link. Currently public activity is only shown to logged in users, but I expect that to be opened up over the next few weeks. There are currently only a fe

... (Read more)

Thanks so much! :)

Festival Stats 2019
78h1 min readShow Highlight

Each year in the fall, since 2014, I've been sharing counts of how many weekend and festival gigs different bands and callers have been doing. Over the course of the year I collect bookings in a big spreadsheet, trying to check each dance weekend's website about a month before the event when they're likely to have a their performers listed.

I got into this as kind of a "market research" thing for the Free Raisins: how many weekends are there? What are the bands that are getting booked a lot, so I can go see what they sound like? Since then I've played a lot more of these ... (Read more)

Social Class
176d3 min readShow Highlight

Notes from the Salon

ETA: This is a write-up of some interesting points raised at an in-person rationalist meetup on October 6th, 2019. It is not intended to be a comprehensive overview of the topic. It is conventional for attendees to do all the suggested readings before salon starts, so some parts of the write-up might not make sense without that context.

Assigned readings: Siderea on Class, Thoughts on the "STEM" class

Economic Class vs. Social Class

Economic class and social class are not the same thing. The two are decoupled, but only partly. You can be wealthy but lower class (e.g.... (Read more)

I think Church's 3-ladder system (linked to in the original article) offers a good foundation to think about this question because Church's system has 12 classes instead of the 3-class system pixx uses.

How to go up a class depends not only on where you come from but also what you're aiming for. Getting from lower to middle class is a different process than getting from middle class to upper class. Even getting from lower middle class to upper middle class is a different process than getting from lower lower class to upper lower class.

So the first question

... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post
1lsusr9h Thank you especially for the link to Church's ladders. I've never seen that before. It was helpful and interesting.

What do balance and alignment mean with respect to the human body?

This is an introduction piece for my hypothesis of human health and movement.

The anatomical information presented here should be easily verifiable.


Alignment and Balance.

A couple of definitions for balanced:

1. Different parts of something exist in equal or correct amounts.

2. A state of equilibrium, being in harmonious arrangement.


Alignment has many definitions, the two I feel most relevant to "body alignment" are:

1. Arrangement in a straight line.

2. Arranged in the correct relative positions.


The Median Pla

... (Read more)

Thank you for writing this series (which somehow I missed seeing when it first appeared). There has been a fair amount posted on nutrition in the past, on the grounds that the better you make your body work, the better the mind within it will work. Mens sana in corpore sano. So material like this on how to use the body is every bit as relevant here, and the subject has previously not much been discussed. To have a body which simply does what you ask of it is a wonderful thing, and supports a mind that does likewise.

[Epistemic status: Sharing current impressions in a quick, simplified way in case others have details to add or have a more illuminating account. Medium-confidence that this is one of the most important parts of the story.]


Here's my current sense of how we ended up in this weird world where:

  • I still intermittently run into people who claim that there's no such thing as reality or truth;
  • a lot of 20th-century psychologists made a habit of saying things like 'minds don't exist, only behaviors';
  • a lot of 20th-century physicists made a habit of saying things like 'quarks
... (Read more)

Like I said before, it means that instrumentalism is the point of view that is the most useful for designing AI or answering questions about AI. According to the “Yudkowskian computationalist” metaphilosophical view, this also makes it the most useful for rationality in general.

Except that if you are the kind of rationalist who cares about what is really real, you should reject instrumentalism immediately.

1TAG11h Ant realists still need to do the things that words like "good" and "true" do - - praise and condemn, and do on. That leads to a euphemism treadmill, where "false" is substituted with "problematic", "evil" with "toxic".

This follows an introduction to the median plane and our midline anatomy as the references for alignment and balance of the human body.

Now consider the moving body and the muscles that you use.

Do you have a full range of natural movement?

A Full Range of Natural Movement.

A full range of natural movement is what the body should be able to do. The body's full potential. Not what you have got used to!

Movement should be smooth and controlled, the body stable as it glides through an almost infinite number of potential positions.

. Image text

To have a full range of natural movement, the bo... (Read more)

my impression of what you were proposing was that it was a fake framework.

I'm going to say no. This is not a fake framework. The '5 main muscles of movement' ARE the central muscular framework of the body. The muscles that, when free to fully function, allow a full range of natural movement and dynamic alignment of the body.

The muscles are paired - left and right sides - so technically that's 10 muscles - but 5 is the correct nomenclature, and sounds less daunting to anyone who doesn't know much anatomy.

The only fake framewo... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post

Make more land
781d2 min readShow Highlight

We used to make land. We built long wharves for docking ships, and then over time filled in the areas between them. Later we built up mudflats wholesale to make even larger areas. Here's a map of Boston showing how much of the land wasn't previously dry:


(Map reproduction courtesy of the Norman B. Leventhal Map & Education Center at the Boston Public Library)

In expensive areas, converting wetlands and shallow water into usable land is a very good thing on balance, and we should start doing it again. To take a specific example, we should make land out of the San Francisco B... (Read more)

https://images.app.goo.gl/c5E11LJfLETAFFBy9

Here is a nice map of the parts of the Netherlands that are under sea level. I live in Amsterdam. Not on the ground floor :)

People in the Netherlands are very concerned about environmentalism and climate change. But not so much about sea level rise. I imagine it has to do with the opportunity to help cities like Miami build infrastructure that can help them livel below sea level, too

1Derek Lomas12h Our great experiment has a reboot mechanism. It's called an election.
3romeostevensit13h Happy to chat about this elsewhere (too many politics tentacles)
12G A13h Hi. I work in the area, and occasionally my job takes me out into this part of the bay. The author is correct...although a portion of this area is a designated wildlife refuge, the majority of it fairly useless. Most of the salt production has moved to cheaper regions. Also consider this...when the tide is out, that portion of the bay is literally only feet deep. Seriously, on average 1-3 feet deep...it’s a giant mud flat. You can’t boat in it or use it for recreation (without sinking into the mud). I’ve tried to walk in that mud...and sank past my waist! There’s very little wildlife out there, the water is pretty stagnant and on some days very smelly. I’ve traveled out into that water (obviously during high tide) and looked at the crowded land mass in every direction and thought the same thing...this spot would be so useful if it wasn’t mud and water!

Reply to Extreme Rationality: It's Not That Great, Extreme Rationality: It could Be Great, the Craft and the Community and Why Don't Rationalists Win?

I’m going to say something which might be extremely obvious in hindsight:

If LessWrong had originally been targeted at and introduced to an audience of competent business people and self-improvement health buffs instead of an audience of STEM specialists and Harry Potter fans, things would have been drastically different. Rationalists would be winning.

Right now, rationalists aren’t winning. Rationality helps us choose which charities to ... (Read more)

7Said Achmiz16h You’re equivocating between the following: 1. To become more X, find a crowd of people who are more X. 2. To become more X, find a crowd of people who are trying to be more X. Perhaps #1 works. But what is actually happening is #2. … or at least, that’s what we might charitably hope is happening. But actually instead what often happens is: 1. To become more X, find a crowd of people who are pretending to try to be more X. And that definitely doesn’t work.
4linkhyrule515h Actually, no, I explicitly want both 1 and 2. Merely being more X than me doesn't help me nearly as much as being both more X and also always on the lookout for ways to be even more X, because they can give me pointers and keep up with me when I catch up. And sure, 3 is indeed what often happens. ... First of all, part of the whole point of all of this is to be able to do things that often fail, and succeed at them anyway; being able to do the difficult is something of prerequisite to doing the impossible. Secondly, all shounen quips aside, it's actually not that hard to tell when someone is merely pretending to be more X. It's easy enough that random faux-philosophical teenagers can do it, after all :V. The hard part isn't staying away from the affective death spiral, it's trying to find the people who are actually trying among them -- the ones who, almost definitionally, are not talking nearly as much about it, because "slay the Buddha" is actually surprisingly general advice.
7Said Achmiz15h Ac­tu­ally, no, I ex­plic­itly want both 1 and 2. Merely be­ing more X than me doesn’t help me nearly as much as be­ing both more X and also always on the look­out for ways to be even more X, be­cause they can give me poin­t­ers and keep up with me when I catch up. What I meant by #2 is “a crowd of people who are trying to be more X, but who, currently, aren’t any more X than you (or indeed very X at all, in the grand scheme of things)”, not that they’re already very X but are trying to be even more X. EDIT: Se­condly, all shounen quips aside, it’s ac­tu­ally not that hard to tell when some­one is merely pre­tend­ing to be more X. Empirically, it seems rather hard, in fact. Well, either that, or a whole lot of people seem to have some reason for pretending not to be able to tell…

a whole lot of people seem to have some reason for pretending not to be able to tell ...

Right—they call it the "principle of charity."

Strong stances
293d9 min readShow Highlight

I. The question of confidence

Should one hold strong opinions? Some say yes. Some say that while it’s hard to tell, it tentatively seems pretty bad (probably).

A quick review of purported or plausible pros:

  1. Strong opinions lend themselves to revision:
    1. Nothing will surprise you into updating your opinion if you thought that anything could happen. A perfect Bayesian might be able to deal with myriad subtle updates to vast uncertainties, but a human is more likely to notice a red cupcake if they have claimed that cupcakes are never red. (Arguably—some would say having opinions makes you less able
... (Read more)
2pjeby17h Since RSS doesn't provide unique IDs for posts, I notice that I am confused, as RSS has a guid field [https://validator.w3.org/feed/docs/rss2.html#ltguidgtSubelementOfLtitemgt] for precisely this purpose. Is it that LW's RSS generation does not include it, or is it some other site producing the RSS?

Oh yeah, I remember experimenting with that, though ended up running into similar problems as comparing links for the wordpress case. I remember the ID changing depending on some kind of context, though I don't remember the exact thing (this code was some of the first code I wrote for the new LessWrong, so it's been a while).

I do think this is a pretty straightforwardly solvable problem, we just haven't put much effort into it, since it hasn't been much of a problem in the past.

Is it that LW's RSS generation does not include it, o
... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post

[A draft section from a longer piece I am writing on prediction and forecasting. Epistemic Status: I don't know what I am missing, and I am filled with doubt and uncertainty.]

If the notion of professional forecasters disturbs you in your sleep, and you toss and turn worrying about the blight of experts brooding upon the world, perhaps the golden light of distributed information systems have peaked out from beyond these darkest visions, and you have hope for the wisdom of crowds.

Prediction markets aggregate information by incentivizing predictors to place bets on the outcomes of well-defin... (Read more)

My other comment was about the ambitious side of prediction markets. This will be about the unambitious side, how they don't have to do much to be better than the status quo.

above technical problems. We are optimistic that these can be overcome.

What problems do you mean, the paragraphs one and three before? That could be clearer. Are you really optimistic, or is this apophasis in which you deniably assert problems? Well, I'm going to talk about them anyway.

ad­di­tional limi­ta­tion to pre­dic­tion mar­kets is th
... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post
2Douglas_Knight17h There's a lot I don't follow here. In particular, you say a bunch of things and it's not clear if you think that they are the same thing or related or unrelated. Some of that may be the excerpt nature. What does the "territory" of the title mean? Snappy titles are good, but you should also explain the metaphor. Perhaps you mean the laws of science, rather than the concrete observations or even interventional claims about specific experiments? Robin Hanson's response [https://mason.gmu.edu/~rhanson/gamble.html] is: Just Do It: make decade long bets on vaguely worded claims. He proposes lots of infrastructure to fix these problems and it doesn't seem very convincing to me, but the proposal seems built incrementally, so it is easy to start small. What does it matter if prediction markets don't do X? If people are proposing prediction markets as an additional institution, then it matters what they do, rather than what they don't do. If they are proposed as a substitute for existing institutions, then it matters if they are as good as the existing ones. But there is a serious instance of status quo bias that people pretend that existing institutions work, whereas they often don't. Seemingly unambitious that work may well be an improvement over ambitious institutions that don't. Robin Hanson does propose substituting prizes for research grants, so there he would have to make that argument. But research funding is highly divisible, so it is easy to start small and see what happens.
Load More