All of Watercressed's Comments + Replies

If a post older than thirty days is downvoted, it doesn't appear in the past 30 days karma.

How do we assign zero probability to 0=1 when we can't prove our logic consistent?

For this sequence we'll just stick to statements in first-order logic, thus avoiding the issue. If you want to look deeper into how consistency interacts with logical probability, you should check out MIRI's stuff on assigning probabilities to classically undecidable statements.

Concern trolling in the false flag political operation sense is a thing that happened

An example of this occurred in 2006 when Tad Furtado, a staffer for then-Congressman Charles Bass (R-NH), was caught posing as a "concerned" supporter of Bass' opponent, Democrat Paul Hodes, on several liberal New Hampshire blogs, using the pseudonyms "IndieNH" or "IndyNH". "IndyNH" expressed concern that Democrats might just be wasting their time or money on Hodes, because Bass was unbeatable.[37][38] Hodes eventually won the election.

Does fundamentalist Christianity indicate that the believer would be irrational about issues other than religion?

If yes, what's the difference?

You could spend the tax-evaded income on the black market, since you're hiding contraband from the police anyway.

Since when did epistemic rationality demand making the truth common knowledge? It just means you should know what's true yourself.

It seems fair to consider epistemic hygiene [] part of epistemic rationality.

Different information about part of nature is not sufficient to change an inference--the probabilities could be independent of the researcher's intentions.

The posterior probability of the observed data given the hidden variable of interest is in general not independent from the intentions of the researcher who is in charge of the data generation process.

One of his "desiderata", his principles of construction, is that the robot gives equal plausibility assignments to logically equivalent statements

I don't see this desiderata. The consistency requirement is that if there are multiple ways of calculating something, then all of them yield the same result. A few minutes of thought didn't lead to any way of leveraging a non 1 or zero probability for Prime(53) into two different results.

If I try to do anything with P(Prime(53)|PA), I get stuff like P(PA|Prime(53)), and I don't have any idea how to i... (read more)

I now think you're right that logical uncertainty doesn't violate any of Jaynes's desiderata. Which means I should probably try to follow them more closely, if they don't create problems like I thought they would. An Aspiring Rationalist's Ramble has a post [] asserting the same thing, that nothing in the desiderata implies logical omniscience.

You can skip this pararaph and the next if you're familiar with the problem. But if you're not, here's an illustration. Suppose your friend has some pennies that she would like to arrange into a rectangle, which of course is impossible if the number of pennies is prime. Let's call the number of pennies N. Your friend would like to use probability theory to guess whether it's worth trying; if there's a 50% chance that Prime(N), she won't bother trying to make the rectangle. You might imagine that if she counts them and finds that there's an odd number, thi

... (read more)
the problem is that, in probability theory as usually formalized and discussed, we assign the same probabilities, though we shouldn't... do you see? EDIT: and it's a problem because she can't calculate her probability without proving whether or not it's prime.

*I keep seeing probability referred to as an estimation of how certain you are in a belief. And while I guess it could be argued that you should be certain of a belief relative to the number of possible worlds left or whatever, that doesn't necessarily follow. Does the above explanation differ from how other people use probability?

One can ground probability in Cox's Theorem, which uniquely derives probability from a few things we would like our reasoning system to do.

Why should anyone expect a specific kind of word input to be capable of persuading everyone? They're just words, not magic spells.

The specific word sequence is evidence for something or other. It's still unreasonable to expect people to respond to evidence in every domain, but many people do respond to words, and calling them just sounds in air doesn't capture the reasons they do so.

I wouldn't call it orthogonal either. Rationality is about having correct beliefs, and I would label a belief-based litmus test rational to the extent it's correct.

Writing a post about how $political_belief is a litmus test is probably a bad idea because of the reasons you mentioned.

Rationality is about have correct beliefs. But a single belief that has only two possible answers is never going to stand in for the entirety of a person's belief structure. That's why you have to look at the process by which a person forms beliefs to have any idea if they are rational.

I generally agree with this post, but since people's beliefs are evidence for how they change their beliefs in response to evidence, I would call it bias-inducing and usually tribal cheering instead of totally backwards.

If I would want to estimate people rationality from beliefs I would look at whether the belief is nuanced. There are a lot of people who say irrational stuff such that they evidence we have for global warming is comparable to the evidence we have for evolution. In reality the p value doesn't even approach the 5 sigma level that you need to validate a result about a new result in particle physics. It's just as irrational as being a global warming denier who thinks that p(global warming)<0.5. Yet we do see smart people making both mistakes. You have smart people who claim that the evidence for global warming is comparable to evolution and you have smart people who are global warming deniers. People don't get mind killed by political issues because they are dumb. It might be completely rational for them because signaling is more important for them. If you want a useful metric do judge someone rationality don't take something where group identities matter a good deal. The metric is just too noisy because the person might get something from signaling group identity. I think the only reason to choose such a metric is because you get yourself mindkilled and want to label people who don't belong to your tribe as irrational and seek some rationalisation for it. As far as empirics [] go, collegue educated Republicans just have a higher rate of climate change denial than Republicans who didn't go to collegue. While we can discuss whether collegue causes people to be more rational it certainly correlates with it. If you want to use beliefs to judge people rationality, calibrate the test. Give people ratioanlity quizes and quiz them for their beliefs. If you get strong correlations you have something that you can use. Don't intellectually analyse the content of the beliefs and think about what rational people should believe if you want an effective metric.
If not "totally backwards" surely "orthogonal". Why not a test that supplies it's own evidence and asks the one being tested to come to a conclusion? Like the Amanda Knox case was for people here who hadn't heard of it before reading about it here.

Hash functions map multiple inputs to the same hash, so you would need to limit the input in some other way, and that makes it harder to verify.

The usual formulation of Omega does not lie.

If Omega maintains a 99.9% accuracy rate against a strategy that changes its decision based on the lottery numbers, it means that Omega can predict the lottery numbers. Therefore, if the lottery number is composite, Omega has multiple choices against an agent that one-boxes when the numbers are different and two-boxes when the numbers are the same: it can pick the same composite number as the lottery, in which case the agent will two-box and earn 2,001,000, or it can pick a different prime number, and have the agent one-box and earn 3,001,000. It seems like the agent that one-boxes all the time does better by eliminating the cases where Omega selects the same number as the lottery, so I would one box.

Above the top-level comment box, there's an option to sort comments by date. Perhaps that should be the default.

But if we're 99.9% confident that a child is going to die (say, they have a very terminal disease), is being cruel to the child 99.99% less bad?

No. (If this is making some clever rhetorical point then perhaps consider a quotation? Right now it is just a rather easy question.)

First-years can't cast AK for reasons of raw magical power, so an organization of first-years can't use the Killing Curse as a membership criteria.

But if say third years can, than ones who got their wands early may be able to. And it isn't all first years either. Plus, it's more about the mindset than the actual ability.

Perhaps the only difference between fake gold and real gold is magical--if there's a ritual that permanently transfigures a rock into gold, people can switch that with the gold in vaults. Of course, no one in the magical world would accept transfigured gold as payment.

Dumbledore may be able to overrule the contract, but that would do little to stop the political effects of Harry's statement that Lucius did not kill Hermione. Since it would also reinstate the debt, it doesn't seem like a net benefit to Dumbledore.

Here's a situation where an anecdote should reduce our confidence in a belief:

  • A person's beliefs are usually well-supported.
  • When he offers supporting evidence, he usually offers the strongest evidence he knows about.

If this person were to offer an anecdote, it should reduce our confidence in his proposition, because it makes it unlikely he knows of stronger supporting evidence.

I don't know how applicable this is to actual people.

I don't think this is necessarily valid, because people also know that anecdotes can be highly persuasive. So for many people, if you have an anecdote it will make sense to say so, since most people argue not to reach the truth but to persuade.
I agree that it is at least hypothetically possible that the offering of an anecdote should reduce our credence in what the anecdote claims.

I would raise a hypothesis to consideration because someone was arguing for it, but I don't think anecdotes are good evidence in that I would have similar confidence in a hypothesis supported by an anecdote, and a hypothesis that is flatly stated with no justification. The evidence to raise it to consideration comes from the fact that someone took the time to advocate it.

This is more of a heuristic than a rule, because there are anecdotes that are strong evidence ("I ran experiments on this last year and they didn't fit"), but when dealing with murkier issues, they don't count for much.

Yes, it may be that the mere fact that a hypothesis is advocated screens off whether that hypothesis is also supported by an anecdote. But I suspect that the existence of anecdotes still moves a little probability mass around, even among just those hypotheses that are being advocated. I mean, if someone advocated for a hypothesis, and they couldn't even offer an anecdote in support of it, that would be pretty deadly to their credibility. So, unless I am certain that every advocated hypothesis has supporting anecdotes (which I am not), I must concede that anecdotes are evidence, howsoever weak, over and above mere advocacy.

A related mistake I made was to be impressed by the cleverness of the aphorism "The plural of 'anecdote' is not 'data'." There may be a helpful distinction between scientific evidence and Bayesian evidence. But anecdotal evidence is evidence, and it ought to sway my beliefs.

Anecdotal evidence is filtered evidence. People often cite the anecdote that supports their belief, while not remembering or not mentioning events that contradict them. You can find people saying anecdotes on any side of a debate, and I see no reason the people who are righ... (read more)

The value of anecdotal evidence on a subject depends on how good the other sources are. For example, in something like medicine where something like 1 in 5 studies wind up retracted, anecdotal evidence is reasonable useful. To say nothing of the social "sciences".

I think the value of anecdotes often doesn't lie so much in changing probabilities of belief but in illustrating what a belief actually is about.

Still evidence.

Of course, if you witness an anecdote with your own eyes, that is not filtered

Unless you too selectively (mis)remember things.

Anecdotal evidence is filtered evidence.

Right, the existence of the anecdote is the evidence, not the occurrence of the events that it alleges.

You can find people saying anecdotes on any side of a debate, and I see no reason the people who are right would cite anecdotes more.

It is true that, if a hypothesis has reached the point of being seriously debated, then there are probably anecdotes being offered in support of it. (... assuming that we're taking about the kinds of hypotheses that would ever have an anecdote offered in support of it.) Theref... (read more)

I made a hash of that comment; I'm sorry.

If we were to put a bunch of chickens into a room, and on one side of the room was a wolf, and the other side had factory farming cages that protected the chickens from the wolf, I would expect the chickens to run into the cages.

It's true that chickens can comprehend a wolf much better than they can comprehend factory farming, but I'm not quite sure how that affects this thought experiment.

[This comment is no longer endorsed by its author]Reply
And I expect that a human would do the same thing.

It takes a bit of work to set up, but Tagtime does both the notifications and the logging

Thanks! Downloaded it, will report back after trying for a bit

There's a bit of a problem with the claim that nobody knows what's what: the usual procedure when someone lacks knowledge is to assign an ignorance prior. The standard methods for generating ignorance priors, usually some formulation of Occam's razor, assign very low probability to claims as complex as common religions.

What do you mean by not rational? People reporting higher satisfaction when they're rich even though they feel less happiness?

Isn't there an irony in belonging to an organisation dedicated to the plight of sentient but cognitively humble beings in the imminent face of vastly superior intelligence and claiming that the plight of sentient but cognitively humble beings in the face of vastly superior intelligence is of no ethical consequence whatsoever. Insofar as we want a benign outcome for humans, I'd have thought that the computational equivalent of Godlike capacity for perspective-taking is precisely what we should be aiming for.

No. Someone who cares about human-level beings ... (read more)

Also, 100% certainty can't be impossible, because impossibility implies that it is 0% likely, which would be a self-defeating argument. You may find it highly improbable that I can truly be 100% certain. What probability do you assign to me being able to assign 100% probability?

When I say 100% certainty is impossible, I mean that there are no cases where assigning 100% to something is correct, but I have less than 100% confidence in this claim. It's similar to the claim that it's impossible to travel faster than the speed of light.

I'm already taking insects or nematodes into consideration probabilistically; I think it is highly unlikely that they are sentient, and I think that even if they are sentient, their suffering might not be as intense as that of mammals, but since their numbers are so huge, the well-being of all those small creatures makes up a non-negligible term in my utility function.

A priori, it seems that the moral weight of insects would either be dominated by their massive numbers or by their tiny capacities. It's a narrow space where the two balance and you get a non-negligible but still-not-overwhelming weight for insects in a utility function. How did you decide that this was right?

Good point, there is reason to expect that I'm just assigning numbers in a way that makes the result come out convenient. Last time I did a very rough estimate, the expected suffering of insects and nematodes (given my subjective probabilities) came out around half the expected suffering of all decapodes/amphibians-and-larger wild animals. And then wild animals outnumber farm animals by around 2-3 orders of magnitude in terms of expected suffering, and farm animals outnumber humans by a large margin too. So if I just cared about current suffering, or suffering on earth only, then "non-negligible" would indeed be an understatement for insect suffering. However, what worries me most is not the suffering that is happening on earth. If space colonization goes wrong or even non-optimal, the current amount of suffering could be multiplied by orders of magnitude. And this might happen even if our values will improve. Consider the case with farmed animals, humans probably never cared as much for the welfare of animals as they do now, but at the same time, we have never caused as much direct suffering to animals as we do now. If you're primarily care about reducing the absolute amount of suffering, then whatever lets the amount of sentience skyrocket is a priori very dangerous.
I think there are good arguments for for suffering not being weighted by number of neurons [] and if you assign even a 10% to that being the case you end up with insects (and maybe nematodes and zooplankton) dominating the utility function because of their overwhelming numbers []. Having said that, ways on increasing the well being of these may be quite a bit different from increasing it for larger animals. In particular, because they so many of them die so within the first few days of life, their averaged life quality seems like it would be terrible. So reducing the populations looks like the current best option. There may be good instrumental reasons for focusing on less controversial animals and hoping that they promote the kind of antispeciesism that spills over to concern about insects and does work for improving similar situations in the future.

It's not really fair to call a range of .02 to 65.92 four digit precision just because the upper bound was written with four digits.

In order to bound the states at a number n, it would need to assign probability zero to ever getting an upgrade allowing it to access log n bytes of memory. I don't know how this zero-probability assignment would be justified for any n--there's a non-zero probability that one's model of physics is completely wrong, and once that's gone, there's not much left to make something impossible.

Er, now I see that Eliezer's post is discussing finite sets of physical laws, which rules out the cosmological horizon diagonalization. But, I think this causal models as function mapping fails in another way: we can't predict the n in n-valued future experiential states. Before the camera was switched, B9 would assign low probability to these high n-valued experiences. If B9 can get a camera that allows it to perceive color, it could also get an attachment that allows it to calculate the permittivity constant to arbitrary precision. Since it can't put a bound on the number of values in the L states, the set is uncountable and so is the set of functions.

What? Of course we can - it's much simpler with a computer program, of course. Suppose you have M bits of state data. There are 2^M possible states of experience. What I mean by n-valued is that there are a certain discrete set of possible experiences. Arbitrary, yes. Unbounded, no. It's still bounded by the amount of physical memory it can use to represent state.

But the set of causal models is not the set of experience mappings. The model where things disappear after they cross the cosmological horizon is a different model than standard physics, even though they predict the same experiences. We can differentiate between them because Occam's Razor favors one over the other, and our experiences give us ample cause to trust Occam's Razor.

At first glance, it seems this gives us enough to diagonalize models--1 meter outside the horizon is different from model one, two meters is different from model two...

There might be... (read more)

[This comment is no longer endorsed by its author]Reply

Causal models are countable? Are irrational constants not part of causal models?

There are only so many distinct states of experience, so yes, causal models are countable. The set of all causal models is a set of functions that map K n-valued past experiential states into L n-valued future experiential states. This is a monstrously huge number of functions in the set, but still countable, so long as K and L are at most countably infinite. Note that this assumes that states of experience with zero discernible difference between them are the same thing - eg, if you come up with the same predictions using the first million digits of sqrt(2) and the irrational number sqrt(2), then they're the same model.

Let's say someone is eating pizza for 20% of their meals. Do you think that replacing pizza with Soylent would result in a worse diet?

Soylent as a supplement and Soylent as a total food replacement are very different things.

That depends on the knowledge that the AI has. If B9 had deduced the existence of different light wavelengths, and knew how blue corresponded to a particular range, and how human eyes see stuff, the probability would be something close to the range of colors that would be considered blue divided by the range of all possible colors. If B9 has no idea what blue is, then it would depend on priors for how often statements end up being true when B9 doesn't know their meaning.

Without knowing what B9's knowledge is, the problem is under-defined.

Anecdotally, I know nobody who has suffered a nutritional deficiency as lethal as zero iron, and the diets in my circle of college students are not very good. I think Soylent will be healthier than my current diet, but I also think the chance of serious nutritional deficiencies is higher.

Edit: To be clear, I'm talking about nutritional deficiencies where one's metabolism starts to fail for want of a crucial element, not deficiencies where someone is consuming marginally less of a nutrient than the optimal amount. I think Soylent will be better than my current diet in the latter category.

Fair point.

Because if something goes wrong, the things you are breaking will be people's well-being. There were two instances where Rob noticed that he was feeling ill and had to correct a nutritional deficiency on the fly. It's less likely that this will happen during large-scale production, but if it does, the people consuming exclusively Soylent will not have all the knowledge Rob did wrt the formula or the symptoms of nutritional deficiency.

I think Soylent is a good idea, and ordered a week's supply, but I'm going to try it slowly; I think the chance that they screw up the production is large enough to merit caution.

I agree that nutritional deficiencies are a problem to watch out for but disagree that consuming Soylent would vastly increase my risk of nutritional deficiencies relative to my current diet. I don't think you're taking into account how bad many people's current diets are / could be. The question is not whether eating Soylent is dangerous but whether it's substantially more dangerous than whatever people are already eating.

"Move fast and break things" is not a good mantra when dealing with nutrition.

Why not?

Objectivisim includes an ethic that many here dislike.

Or rather, I think that genetics possibly plays a large role now, but that if we raised people better than we could essentially eliminate these issues without focusing on genes.

What model of hereditary intelligence predicts significant hereditary differences in the current environment and negligible differences in an environment where people are raised better?

I was not saying that everyone would have the same level of intelligence, but merely that the baseline might be high enough that violence becomes less of an issue. That was the original subject. Similar to drethelin's comment.
(this is talking out of my ass but:) Variation in genetic robustness/fragility. A known example of this is Iron deficiency in women. If iron in the food is plentiful, no one will notice iron deficiencies, and if it's non existent then everyone will suffer. But if there's an almost sufficient amount of Iron, women will be far more likely to be deficient than men. Women are less robust to lack of environmental iron than men. You can imagine brain development such that everyone has the ability to develop a great brain in a great environment but certain genetics will deal better or worse with certain deficiencies.

x + 0 = 0

I think you mean x + 0 = x

yes. yes. i remember thinking "x + 0 =". after that it gets a bit fuzzy.

You can edit the URL directly, and it will point to a new image; there's no need to submit it back into the website.

if you copy the url of the image, it contains the LaTeX in a url-encoded form (so spaces are replaced by %20 and ^ is replaced by %5E)

OK, and how do I drop it back into the editor? By manually fixing the %20 and %5E? Or is there something better?

Probability Theory: The logic of science is available in postscript form at

Load More