All of Optimization Process's Comments + Replies

A Semitechnical Introductory Dialogue on Solomonoff Induction

Hmm. If we're trying to argmax some function over the real numbers, then the simplest algorithm would be something like "iterate over all mathematical expressions ; for each one, check whether the program 'iterate over all provable theorems, halting when you find one that says ' halts; if it does, return ."

...but I guess that's not guaranteed to ever halt, since there could conceivably be an infinite procession of ever-more-complex expressions, eking out ever-smaller gains on . It seems possible that no matter what (reasonably powerful) mathe... (read more)

A Semitechnical Introductory Dialogue on Solomonoff Induction

The understanding I came away with: there are (at least) three stages of understanding a problem:

  1. You can't write a program to solve it.
  2. You can write a cartoonishly wasteful program to solve it.
  3. You can write a computationally feasible program to solve it.

"Shuffle-sort" achieves the second level of knowledge re: sorting lists. Yeah, it's cartoonishly wasteful, and it doesn't even resemble any computationally feasible sorting algorithm (that I'm aware of) -- but, y'know, viewed through this lens, it's still a huge step up from not even understanding "so... (read more)

"New EA cause area: voting"; or, "what's wrong with this calculation?"

Ah! You're saying: if my "500k coin flips" model were accurate, then most elections would be very tight (with the winner winning by a margin of around 1/800, i.e. 0.125%), which empirically isn't what happens. So, in reality, if you don't know how an election is going to turn out, it's not that there are 500k fair coins, it's that there are either 500k 51% coins or 500k 49% coins, and the uncertainty in the election outcome comes from not knowing which of those worlds you're in. But, in either case, your chance of swinging the election is vanishingly small... (read more)

3CarlShulman2moThat is the opposite error, where one cuts off the close election cases. The joint probability density function over vote totals is smooth because of uncertainty (which you can see from polling errors), so your chance of being decisive scales proportionally with the size of the electorate and the margin of error in polling estimation.
"New EA cause area: voting"; or, "what's wrong with this calculation?"

Also, note that your probability of swinging the election is only 1/√n if the population is split exactly 50/50; it drops off superexponentially as the distribution shifts to one side or the other by √n voters or more.

Yesss, this seems related to shadonra's answer. If my "500k coin flips" model were accurate, then most elections would be very tight (with the winner winning by a margin of 1/800, i.e. 0.125%), which empirically isn't what happens. So, in reality, if you don't know how an election is going to turn out, it's not that there are 500k fair co... (read more)

"New EA cause area: voting"; or, "what's wrong with this calculation?"

Wait... your county has a GDP of over half a million dollars per capita? That is insanely high!

I agree! (Well, actually more like $1-200k/capita, because there are more people than voters, but still.) Sources: population, GDP, turnout.

3Dagon2moUmm, that's a very misleading statistic - King County, WA has a very uneven distribution of contribution to GDP, as it includes a number of international powerhouses that local politics don't affect very much (nonzero, but nowhere near the few percent you're estimating). per-capita averages are just about useless for any planning or valuation of action.
"New EA cause area: voting"; or, "what's wrong with this calculation?"

Sure! I'm modeling the election as being coin flips: if there are more Heads than Tails, then candidate H wins, else candidate T wins.

If you flip coins, each coin coming up Heads with probability , then the number of Heads is binomially distributed with standard deviation , which I lazily rounded to .

The probability of being at a particular value near the peak of that distribution is approximately 1 / [that standard deviation]. ("Proof": numerical simulation of flipping 500k coins 1M times, getting 250k Heads about 1/80... (read more)

"New EA cause area: voting"; or, "what's wrong with this calculation?"
  • Possible answer: "Sure, it's individually rational for you to devote your energy to Getting Out The Vote instead of donating to charity, but the group-level rational thing for people to do is to donate to charity, rather than playing tug-o'-war against each other."

    Ugh, yeah, maybe. I see the point of this sort of... double-think... but I've never been fully comfortable with it. It sounds like this argument is saying "Hey, you put yourself at a 60% probability of being right, but actually, Outside View, it should be much smaller, like 51%." But, buddy, th

... (read more)
"New EA cause area: voting"; or, "what's wrong with this calculation?"
  • Possible answer: "You're doing a causal-decision-theory calculation here (assuming that your vote might swing the election while everything else stays constant); but in reality, we need to break out [functional decision theory or whatever the new hotness is], on account of politicians predicting and "pricing in" your vote as they design their platforms."

    Hmm, yeah, maybe. In which case, the model shouldn't be "my vote might swing the election," but instead "my vote will acausally incrementally change candidates' platforms," which I don't have very good models for.

"New EA cause area: voting"; or, "what's wrong with this calculation?"
  • Possible answer: "No election is decided by a single vote; if it's that close, it'll be decided by lawyers."

    Rebuttal: yeah, it's a little fuzzy, but, without having cranked through the math, I don't think it matters: my null hypothesis is that my vote shifts the probability distribution for who wins the legal battle in my desired direction, with an effect size around the same as in the naive lawyer-free model.

Promoting Prediction Markets With Meaningless Internet-Point Badges

I would love to live in this world.

This seems like a really hard problem: if a market like this "wins," so that having a lot of points makes you high-status, people will try to game it, and if gaming it is easy, this will kill respect for the market.

Specific gaming strategies I can think of:

  • Sybil attacks: I create one "real" account and 100 sock puppets; my sock puppets make dumb bets against my real account; my real account gains points, and I discard my sock puppets. Defenses I've heard of against Sybil attacks: make it costly to participate (e.g. proo
... (read more)
2ChristianKl3moThis is not really a problem at Metaculus. Metaculus has metrics for player prediction, community prediction and Metaculus prediction on the questions someone answers. The community prediction could be changed with sockpuppets but the Metaculus prediction can't. You can judge people on how near they are to the Metaculus prediction in predictive accuracy or whether they even outperform it. Metaculus however decides to keep that metric mostly private and makes it nonpublic. Metaculus has the problem of not wanting to embarrass users who make a lot of predictions when those predictions are on average bad I don't think Amazon makes a serious effort at battling review fraud just as Youtube doesn't make a serious effort at comment quality when it easily could do something about it. Amazon also has a harder problem because as it's ground truth is less clear.
2Davidmanheim3moFor scoring systems, rather than betting markets, none of these particular attacks work. This is trivially true for the first and third attack, since you don't be against individuals. And for any proper scoring rule, calibration-fluffing is worse than predicting your true odds for the dumb predictions. (Aligning incentives is still very tricky, but the set of attacks are very different.)
4TurnTrout3moNote that this could be mitigated by other people being able to profit off of obvious epistemic inefficiencies in the prediction markets: if your bots drive the community credence down super far, and if other people notice this, then other people might come in and correct part of the issue. This would reduce your advantage relative to other Metaculites.

Is this some kind of attempt at code injection? :P

Only the benign kind! I've got some ideas burbling in my brain re: embedding dynamic content in my writing, so I'm just exploring the limits of what Less Wrong permits in its HTML. (Conclusion: images hosted on arbitrary other domains are okay, but svgs are not. Seems sane.)

Does there exist a detailed Bayesian COVID tracker?

If no such thing exists, I might take a stab at creating one -- so I'd even love to hear if you know of some causal-graph-inference-toolkit-thing that isn't specifically for COVID but seems like a promising foundation to build atop!

But, if no such thing exists, that also seems like evidence that it... wouldn't be useful? Maybe because very few social graphs have the communication and methodicalness to compose a detailed list of all the interactions they take part in? Conceivably because it's a computationally intractable problem? (I dunno, I hear that large Bayes nets are extremely hard to compute with.)

2Vasco Figueira6moMaybe BayesDB [http://probcomp.csail.mit.edu/software/bayesdb/] can help?
4habryka6moI would probably copy the MicroCOVID spreadsheet, and then write some custom logic into the cells that track people's microcovid levels. Seems like it wouldn't be too hard, and at the level of customization you want, I expect you would have to do something equally complicated with almost any other tool.
How can I reconcile these COVID test false-negative numbers?

Further point of confusion: the Emergency Use Authorization summary mentions n=31 positive samples and n=11 negative samples in the "Analytical Specificity" section -- how do you get "98%" or "99%" out of those sample sizes? Shouldn't you need at least n=50 to get 98%? Heck, why do they have any positive (edit: negative) samples in a "Specificity" section?

1Teerth Aloke6moPlease follow up on what you find.
Optimization Process's Shortform

Clearly not all - the extreme version of this is betting on human extinction. It's hard to imagine the payout that has any value after that comes to pass.

Agreed that post-extinction payouts are essentially worthless -- but doesn't the contract "For $90, I will sell you an IOU that pays out $100 in one year if humans aren't extinct" avoid that problem?

1Donald Hobson1yThis is exactly conditional to a bond that pays out in one year "unconditionally". Ie this is a loan with interest. (There are a few contrived scenarios where humans are extinct and money isn't worthless, depending on the definitions of those words. Would this bond pay out in a society of uploaded minds?)
2Dagon1ySmall amounts and near-even-money ($90 for $100) are bad intuition pumps - this is in the range where other considerations dominate the outcome estimates. In fact, you probably can't find many people to accept only 11% for a one-year unsecured loan.
Optimization Process's Shortform

Some wagers have the problem that their outcome correlates with the value of what's promised. For example, "I bet $90 against your $10 that the dollar will not undergo >1000% inflation in the next ten years": the apparent odds of 9:1 don't equal the probability of hyperinflation at which you'd be indifferent to this bet.

For some (all?) of these problematic bets, you can mitigate the problem by making the money change hands in only one arm of the bet, reframing it as e.g. "For $90, I will sell you an IOU that pays out $100 in ten years if the dollar hasn

... (read more)
2Dagon1yClearly not all - the extreme version of this is betting on human extinction. It's hard to imagine the payout that has any value after that comes to pass. In some, you can find the conditional wagers that work, in some you can find a better resource or measurement to wager (one gram of gold, or one day's average wage as reported by X government). In many, though, there just is no wager possible, as the utility of the parties diverges too much from the resources available to account for in the wager.
Heads I Win, Tails?—Never Heard of Her; Or, Selective Reporting and the Tragedy of the Green Rationalists

(Strong approval for this post. Figuring out how to deal with filtered evidence is close to my heart.)

Suppose that the facts relevant to making optimal decisions about an Issue are represented by nine rolls of the Reality die, and that the quality (utility) of Society's decision is proportional to the (base-two logarithm) entropy of the distribution of what facts get heard and discussed.

Sorry-- what distribution are we measuring the entropy of? When I hear "entropy of a distribution," I think -- but it's not clear to me how to get from

... (read more)
7Zack_M_Davis2yYou have three green slots, three gray slots, and three blue slots. You put three counters each on each of the green and gray slots, and one counter each on each of the blue slots. The frequencies of counters per slot is [3, 3, 3, 3, 3, 3, 1, 1, 1]. The total number of counters you put down is 3*6 + 3 = 18 + 3 = 21. To turn the frequencies into a probability distribution, you divide everything by 21, to get [1/7, 1/7, 1/7, 1/7, 1/7, 1/7, 1/21, 1/21, 1/21]. Then the entropy is 6⋅−17log217+3⋅−121log2121, which is 67log27+321log221. Right? (Thanks for checking—it would be really embarrassing if I got this wrong. I might edit the post later to include more steps.)
Goodhart Taxonomy

Very interesting! I like this formalization/categorization.

Hm... I'd have filed "Why the tails come apart" under "Extremal Goodhart": this image from that post is almost exactly what I was picturing while reading your abstract example for Extremal Goodhart. Is Extremal "just" a special case of Regressional, where that ellipse is a circle? Or am I missing something?

9Unnamed3yHeight is correlated with basketball ability. Regressional: But the best basketball player in the world (according to the NBA MVP award) is just 6'3" (1.91m), and a randomly selected 7 foot (2.13m) tall person in his 20s would probably be pretty good at basketball but not NBA caliber. That's regression to the mean; the tails come apart. Extremal: The tallest person on record, Robert Wadlow [https://en.wikipedia.org/wiki/Robert_Wadlow], was 8'11" (2.72m). He grew to that height because of a pituitary disorder, he would have struggled to play basketball because he "required leg braces to walk and had little feeling in his legs and feet", and he died at age 22. His basketball ability was well below what one might naively predict based on his height and the regression line, and that is unsurprising because the human body wasn't designed for such an extreme height.
4Scott Garrabrant3yExtremal is not a special case of regressional, but you cannot seperate them completely because regressional is always there. I think the tails come apart is in the right place. (but I didn't reread the post when I made this) If you sample a bunch of points from a multivariate normal without the large circular boundary in my example, the points will roughly form an ellipse, and the tails come apart thing will still happen. This would be Regerssional Goodhart. When you add the circular boundary, something weird happens where now you optimization is not just failing to find the best point, but actively working against you. If you optimize weakly for the proxy, you will get a large true value, but when you optimize very strongly you will end up with a low true value.