On the contrary, I think there is no norm against board members criticizing corporate direction.
I think it is accepted that a member of the board of a for-profit corporation might publicly say that they think the corporation's X division should be shut down, in order to concentrate investment in the Y division, since they think the future market for Y will be greater than for X, even though the rest of the board disagrees. This might be done to get shareholders on-side for this change of direction.
For a non-profit, criticism regarding whether the cor... (read more)
The mathematics of a latent variable model expresses the probabilities, p(x), for observations x as marginal probabilities integrating over unobserved z. That is, p(x) = integral over z of p(x,z), where p(x,z) is typically written as p(z)p(x|z).
It's certainly correct that nothing in this formulation says anything about whether z captures the "causes" of x.
However, I think it sometimes is usefully seen that way. Your presentation would be clearer if you started with one or more examples of what you see as typical models, in which you argue that ... (read more)
OK. My views now are not far from those of some time ago, expressed at https://glizen.com/radfordneal/res-bayes-ex.html
With regard to machine learning, for many problems of small to moderate size, some Bayesian methods, such as those based on neural networks or mixture models that I've worked on, are not just theoretically attractive, but also practically superior to the alternatives.
This is not the case for large-scale image or language models, for which any close approximation to true Bayesian inference is very difficult computationally.
However, I ... (read more)
OK. I think we may agree on the technical points. The issue may be with the use of the word "Bayesian".
Me: But they aren't guaranteed to eventually get a Bayesian to think the null hypothesis is likely to be false, when it is actually true.
You: Importantly, this is false! This statement is wrong if you have only one hypothesis rather than two.
I'm correct, by the usual definition of "Bayesian", as someone who does inference by combining likelihood and prior. Bayesians always have more than one hypothesis (outside trivial situations ... (read more)
If I do an experiment, you generally don't know the precise alternate hypothesis in advance -- you want to test if the coin is fair, but you don't know precisely what bias it will have if it's unfair.
Yes. But as far as I can see this isn't of any particular importance to this discussion. Why do you think it is?
If we fix the two alternate hypotheses in advance, and if I have to report all data, then I'm reduced to only hacking by choosing the experiment that maximizes the chance of luckily passing your threshold via fluke. This is unlikely, as you say... (read more)
I am saying that Yudkowsky is just plain wrong here, because omitting info is not the same as outright lying.
This is silly. Obviously, Yudkowsky isn't going to go off on a tangent about all the ways people can lie indirectly, and how a Bayesian ought to account for such possibilities - that's not the topic. In a scientific paper, it is implicit that all relevant information must be disclosed - not doing so is lying. Similarly, a scientific journal must ethically publish papers based on quality, not conclusion. They're lying if they don't. As for auth... (read more)
If you say that you are reporting all your observations, but actually report only a favourable subset of them, and the Bayesian for some reason assigns low probability to you deceiving them in this way, when actually you are deceiving them, then the Bayesian will come to the wrong conclusion. I don't think this is surprising or controversial.
But I don't see how the Bayesian comes to a wrong conclusion if you truthfully report all your observations, even if they are taken according to some scheme that produces a distribution of likelihood ratios that is sup... (read more)
I think that various "pro-fertility" people have a variety of motivations.
But "more people are better" ought to be a belief of everyone, whether pro-fertility or not. It's an "other things being equal" statement, of course - more people at no cost or other tradeoff is good. One can believe that and still think that less people would be a good idea in the current situation. But if you don't think more people are good when there's no tradeoff, I don't see what moral view you can have other than nihilism or some form of extreme egoism.
BTW: I... (read more)
Integrals of the likelihood function aren't really meaningful, even if normalized so the integral is one over the whole range. This is because the result depends on the arbitrary choice of parameterization - eg, whether you parameterize a probability by p in [0,1], or by log(p) in [-oo,0]. In Bayesian inference, one always integrates the likelihood only after multiplying by the prior, which can be seen as a specification of how the integration is to be done.
I think you've got his pretty much figured out. But you may be missing an additional subtlety.
You say "Bayesian likelihood ratios really do only depend on the probability each hypothesis assigned only to the information that you received". Which could be interpreted as saying that the "likelihood function" is the probability assigned to the information received, seen as a function of f. But the likelihood function is actually not a function at all, but rather an equivalence class of functions of f that differ only by an overall pos... (read more)
Your factual claims here seem at least somewhat reasonable. Naively extrapolating sub-replacement fertility centuries into the future is silly. Our wealthy civilization ought to be capable of finding some way of coping with increased elder care. The current number of humans may perhaps be more than is optimal.
But your moral view is atrocious. Human extinction would be bad - very bad. Because humans are the most interesting species to arise so far. Because human striving is properly focused on the future, not just hedonistic pl... (read more)
I tested this with ChatGPT4 just now. It mostly got my specific questions right, but its commentary reveals some substantial misconceptions, such as that a flight from Ecuador to Sri Lanka would first pass over the Atlantic Ocean and then over Brazil.
Here it is:
If I flew by the most direct route from Ecuador to Sri Lanka, would I pass over Egypt?
If you were to fly the most direct route (a great circle route) from Ecuador to Sri Lanka, you would not pass over Egypt.
Here's a general idea of what the path would look like:
Well, given that the text of the US constitution seems to clearly state that all powers not explicitly granted to the federal government belong to the states (or the people), I don't see how "power is devolved to the states from the federal government". It seems that the states don't need to wait for the federal government to "devolve" power to them in order to do something. As indeed we saw recently with respect to covid policy.
You could argue that the federal government "lets" the states do this, in the sense that the federal government has more guns tha... (read more)
the constitution is quite clear that power is devolved to the states from the federal government, and not that the federal government is granted power at the behest of the states
The 10th amendment to the US constitution says:
The powers not delegated to the United States by the Constitution, nor prohibited by it to the States, are reserved to the States respectively, or to the people.
which sounds like the opposite of what you say. Of course, practice may be different.
Can you tell me why?
It think if we encountered aliens who were apparently not hostile, but presumably strange, and likely disgusting or disturbing in some ways, there would be three groups (likely overlapping) of people opposed to wiping them out:
There would also be three groups in favour of wi... (read more)
We are a species that has evolved in competition with other species. Yet, I think there is at least a 5% chance that if we encountered an intelligent alien species that we wouldn't try to wipe them out (unless they were trying to wipe us out).
Biological evolution of us and aliens would in itself be a commonality, that might produce some common values, whereas there need be no common values with an AI created by a much different process and not successfully aligned.
Perhaps of relevance:
How to Tell the Birds from the Flowers
One problem I have with Diamond's theory is that I doubt that there is anything for it to explain. The Americas and Eurasia/Africa were essentially isolated from each other for about 15,000 years. In 1500 AD, the Americas were roughly 3500 years less advanced than Eurasia/Africa. That seems well within the random variation one would expect between two isolated instances of human cultural development over a 15,000 year time span. If you think there is still some remaining indication that the Americas were disadvantaged, the fact that the Americas are about half the size of Eurasia/Africa seems like a sufficient explanation.
Perhaps you could give the definition you would use for the word "probability".
I define it as one's personal degree of belief in a proposition, at the time the judgement of probability is being made. It has meaning only in so far it is (or may be) used to make a decision, or is part of a general world model that is itself meaningful. (For example, we might assign a probability to Jupiter having a solid core, even though that makes no difference to anything we plan to do, because that proposition is part of an overall theory of physics that is meaning... (read more)
I think we actually have two quantities:
"Quobability" - The frequency of correct guesses made divided by the total number of guesses made.
"Srobability" - The frequency of trials in which the correct guess was made, divided by the number of trials.
Quabability is 1/3, Scrobability is 1/2. "Probability" is (I think) an under-precise term that could mean either of the two.
I suspect that the real problem isn't with the word "probability", but rather the word "guess". In everyday usage, we use "guess" when the aim is to guess correctly. But the aim here is to no... (read more)
By "GWYL" do you actually mean "GRYL" (ie, Guess Right You Live)?
One could argue that if the coin is flicked and comes up tails then we have both "Tails&Monday" and "Tails&Tuesday" as both being correct, sequentially.
Yes, it is a commonplace occurrence that "Today is Monday" and "Today is Tuesday" can both be true, on different days. This doesn't ordinarily prevent people from assigning probabilities to statements like "Today is Monday", when they happen to not remember for sure whether it is Monday or not now. And the situation is the same for Beauty - it is either Monday or Tuesday, she doesn't know which, but... (read more)
If the Tuesday bet is considered to be "you can take the bet, but it will replace the one you may or may not have given on a previous day if their was one", then things line up to half again.
I can think of two interpretations of the setup you're describing here, but for both interpretations, Beauty does the right thing only if she thinks Heads has probability 1/3, not 1/2.
Note that depending on the context, a probability of 1/2 for something does not necessarily lead one to bet on it at 1:1 odds. For instance, if based on almost no knowledge of baseb... (read more)
I agree that you can make a betting/scoring setup such that betting/predicting at 50% is correct. Eg,suppose that on both Monday and Tuesday, Beauty gets to make a $1 bet at some odds. If she's asleep on Tuesday then whatever bet she made on Monday is repeated. In that case she should bet at 1:1 odds.
Let's work it out if Beauty has 1/3 probability for Heads, with the 2/3 probability for Tails split evenly between 1/3 for Tails&Monday and 1/3 for Tails&Tuesday.
Here, the possible actions are "don't bet" or "bet on Heads", at 1:1 odds (let's say win $... (read more)
There are rules for how to do arithmetic. If you want to get the right answer, you have to follow them. So, when adding 18 and 17, you can't just decide that you don't like to carry 1s today, and hence compute that 18+17=25.
Similarly, there are rules for how to do Bayesian probability calculations. If you want to get the right answer, you have to follow them. One of the rules is that the posterior probability of something is found by conditioning on all the data you have. If you do a clinical trial with 1000 subjects, you can't just decide that you'd... (read more)
Yes, I can't get it either.
The possibility of a broken link (now or in the future) is one of several reasons why it would have been a good idea to mention the title of the paper and the authors' names.
Nope. Doesn't work this way. There is an important difference between a probability of a specific low probable event happening and a probability of any low probable event from a huge class of events happening.
In Bayesian probability theory, it certainly does work this way. To find the posterior probability of Heads, given what you have observed, you combine the prior probability with the likelihood for Heads vs. Tails based on everything that you have observed. You don't say, "but this observation is one of a large class of observations that I've decided t... (read more)
Any other event, that the beauty is rightfully confident to be low probable but possible for both Monday and Tuesday, happens.
And since this happens absolutely every time she wakes, Beauty should always assess the probability of Heads as 1/3.
There's always fly crawling on the wall in a random direction, unlikely to be the same on Monday and Tuesday, or a stray thought about aardvarks, or a dimming of the light from the window as a cloud passes overhead, or any of millions of other things entering her consciousness in ways that won't be the same Mond... (read more)
I'm not trying to be disrespectful here, just trying to honestly state what I think (which I believe is more respectful than not doing so).
If I understand your position correctly, you think Beauty should rationally think as follows:
This opens a tangent discussion about determinism and whether the amnesia is supposed to return her to the exactly the same state as before, but thankfully we do not need to go there.
To briefly go there... returning her to the exactly same state would violate the no-cloning theorem of quantum mechanics. See https://en.wikipedia.org/wiki/No-cloning_theorem
I'm saying that some of her probabilities become meaningful, even though they were not before. Tails&Monday, Tails&Tuesday, Heads&Monday become three elementary outcomes for her when she ... (read more)
You don't contradict the claim "you can't lawfully use probability theory in this setting" by showing that you will be able to use probability theory if you break the setting up.
The setting is only "broken up" when she decides to leave the room, but she can think about probabilities before that. Are you saying that once she decides to leave the room, her probabilities for aspects of the external world should change?
It is completely typical for people to form probability judgements regarding what would happen if they were hypothetically to do somethin... (read more)
(Heads, Tuesday) does not occur in conjunction with her waking up, assuming the experimenters behave in the way they say they will. So when she wakes up, it is not a possibility for what her current situation is. Which is what we're talking about.
you can't lawfully use probability theory in this setting assuming that Tails and Monday, Tails and Tuesday and Heads and Monday are three exhaustive and exclusive outcomes which probabilities are supposed to be added to 1
Suppose for simplicity that Beauty knows that the coin is flipped on Sunday, on a table in a room next to her bedroom, and that the flipped coin is left on the table, with either Heads or Tails showing.
Now, suppose Beauty wakes up (and knows it's not Wednesday, since otherwise she would have been told immediately tha... (read more)
You haven't mentioned the strongest argument for not updating your beliefs: You aren't dead yet.
The beliefs you hold now have not had fatal consequences (so far, at least). The same is not guaranteed if you change them.
Corollary: If you see death coming, or e.g. you have a near miss and know it was only by chance that you survived, then now’s a good time to change your beliefs. Which, actually, seems to be a thing people do. (Though there are other reasons for that.)
There are two ways one might try to figure out these probabilities. One is that, in whatever final situation is being considered, you figure out the probability from scratch, as if the question had never occurred to you before. The other is that as you experience things you update your probability for something, according to the the likelihood ratio obtained from what you just observed, and in that way obtain a probability in the final situation.
When figuring out the probability of Heads from scratch on Wednesday, I think everyone agrees that it should be ... (read more)
Let's look at the scenario where Beauty remembers her last awakening.
When Beauty is woken and not told it's Wednesday, she should think Heads has probability 1/3, Tails 2/3. She knows that if the coin landed Heads, it is Monday, and that she will next wake up on Wednesday without forgetting anything. She knows that if the coin landed Tails, then with probability 1/2 it is Monday and her memory will soon be erased, and with probability 1/2 it is Tuesday and she will next wake up on Wednesday without forgetting anything.
So waking up on Wed... (read more)
What's Beauty credence for Heads when she wakes on Wednesday and doesn't remember any of her awakenings on Monday/Tuesday? If it's 1/2 what is the reason for the change from 1/3? The fact that she has forgotten the information about her awakening during the experiment?
Because of the memory erasure, I think it's best to regard Beauty on Monday, on Tuesday (if woken), and on Wednesday as different people, all "descended" from Beauty on Sunday. But if you want to think of them as the same person, the change from 1/3 to 1/2 is a consequence of the new in... (read more)
I think it may not be correct to shuffle this off into a box labelled "adversarial example" as if it doesn't say anything central about the nature of current go AIs.
Go involves intuitive aspects (what moves "look right"), and tree search, and also something that might be seen as "theorem proving". An example theorem is "a group with two eyes is alive". Another is "a capture race between two groups, one with 23 liberties, the other with 22 liberties, will be won by the group with more liberties". Human players don't search the tree down to... (read more)
I think the cyclic group exploit could have been found by humans. The idea behind it (maybe it gets confused about liberties when a group is circular) would probably be in the top 1000 ideas for exploits that a group of humans would brainstorm. Then these would need to be tested. Finding a working implementation would be a matter of trial and error, maybe taking a week. So if you got 100 good human go players to test these 1000 ideas, the exploit would be found within ten weeks.
The main challenge might be to maintain morale, with th... (read more)
Did you think it was interesting when AIs became better than all humans at go?
If so, shouldn't you be interested to learn that this is no longer true?
My favourite (done just now, with the July 20 version of chatGPT4):
Why is gold denser than uranium?
There seems to be a confusion in your question. Actually, uranium is denser than gold.
As of my last training cut-off in September 2021, the density of uranium is approximately 19.1 g/cm³, while that of gold is around 19.3 g/cm³. These densities can vary slightly depending on the exact conditions (like temperature and pressure), but generally, gold is not denser than uranium.
Density of a substance is determined by its atomic structure ... (read more)
Like a number of people (including Elga), you're converting an almost-doable thought experiment into one that may be impossible in principle. If the experiment is done with some not-yet-invented but plausible memory-erasing drug, but is otherwise realistic, Beauty will not have the same experiences when woken Monday and when woken Tuesday. Various aspects of her sensed environment, as well as internal thoughts, will be different for the two awakenings. We just assume that none of these differences allow her to infer th... (read more)
I'm not sure what you mean by "Beauty's expected experience ... on Wednesday". There are two possibilities. On Sunday, she will expect that they have equal probabilities. But that says nothing more than that on Sunday she expects Heads and Tails to be equally likely. When woken on Monday or Tuesday, here expectation of what Wednesday will be like isn't necessarily the same.
I also can't figure out what you mean by saying that 1/3 is "absolutely correct", while also saying that there could be another answer.
I've reviewed again the argument in Section 2 of Elga's paper (https://www.princeton.edu/~adame/papers/sleeping/sleeping.pdf) and it seems valid to me. It doesn't have the same simplicity of some other arguments for 1/3, though, such as the betting arguments, and my argument in https://glizen.com/radfordneal/anth.abstract.html that the probability of anyone having your detailed experience on awakening is twice are large if you are woken twice.
There are many good arguments for 1/3. The arguments for 1/2 are all bad, and defending them forces peo... (read more)
Presumably the aim is to enhance human intellectual capabilities, not necessarily the level of innate human intelligence. Looking at it that way, improvements to education seem like a much more promising approach (which isn't to say that one shouldn't do both, of course).
One might object that people have been trying to improve education for millennia, so why would one think there's any low-hanging fruit to be had here? There are two reasons. One is that enhancing intellectual capabilities has not been the only goal of education (or even the pri... (read more)
Ahh, but without a purpose, how can one tell what the question actually is?
You could be asking about who you get by tracing back births and matings over the last 300,000 years that led to you. But do you then count people more than once if they show up more than once in this back trace?
Or are you really asking where your genetic material came from? It's quite possible that none of your genetic material came from one of your great-great-great-great-great-great-great-grandfathers. Genes aren't selected independently to be included or not in... (read more)
Providing a sensible answer is dependent on arriving at a sensible interpretation of the question. I'll assume that it is aimed at understanding to what degree farming or non-farming lifestyles have had an influence on the selection of genes that you carry. I assume that "farming lifestyle" includes people who don't actually farm, but obtain food from farmers, one way or another.
On that basis, and assuming you are a typical inhabitant of a society that hasn't recently engaged in much hunting/gathering (maybe some fishing, but not dominant), I w... (read more)
I don't think you can separate these phenomena like this. Thugs who aren't official police can intimidate political opponents of the government, and then not be prosecuted by the government. Thugs can cause chaos that somehow goes away when an organization or community stops opposing the government, or pays money to associates of the thugs, with the government again not prosecuting the thugs for extortion. In fact, I find it hard to imagine a democratic government becoming authoritarian without it employing some extra-governmental coercio... (read more)
How many people have been killed in the US by right-wing protestors in the last four years? How many have been killed by left-wing protestors?
I can't say with any certainty what exactly happened. Neither, it seems, can anyone else, to judge by https://www.forbes.com/sites/siladityaray/2022/02/23/canada-begins-to-release-frozen-bank-accounts-of-freedom-convoy-protestors/?sh=2745dbcc6364
Note that there is no reason to take statements by government officials at face value.
Even if you believe that the government intended only to freeze accounts of people actually at the protests, this is still a dangerous expansion of government power, considering that there were no court hearings, where peopl... (read more)