It is not a fee-for-service relationship. The price system in medicine has been mangled beyond recognition. Patients are not told prices; doctors avoid, even disdain, any discussion of prices; and the prices make no rational sense even if and when you do discover them. This destroys all ability to make rational economic choices about healthcare.
I think pricing of medical services faces somewhat of a breakdown in the normal price-setting mechanism of markets. For some random good like a sandwich or whatever the buyer can at least have a reasonable sense of how much they want it, the seller understands their costs to produce it, and the price gets established by this balance. But how is someone who seeks medical care really supposed to know how much they value a particular medical service? They would presumably have to rely on their provider, who is on the opposite side of the transaction. Insurers could somewhat serve this role, but I think people often look down upon this, and also it seems likely to be a difficult and imperfect process.
Computing p-values is what Mr. Frequentist is all about.
For once I'd like to the bayesian/frequentist debate see the return of maximum likelihood vs maximum a-posteriori. P-values absolutely are not the only aspect of frequentist statistics! Yes they are one of the prominent so certainly fair game, but the way people talk about them its like they are all that matter. People have general problems with p-values beyond them being frequentist. To me the fact that they feature so prominently raises the question of how much certain commitments to "bayesianism" reflect actual usage of bayesian methods vs a kind of pop-science version of bayesianism.
Bayesian likelihood ratios
Is this meant to refer to a specific likelihood ratio method, or to suggest that likelihood ratios themselves are "bayesian"? Yes, "the likelihood principle" is a big source of criticism of p-values, but I don't see why likelihood ratios themselves are bayesian? I think Andrew Gelman once said something to the effect of both bayesian and frequentist methods need a likelihood, and often times that makes more of a difference than the prior. There's nothing strictly bayesian about "updating". I'm curious how often things that are identified as "bayesian" actual use Bayes' rule.
The frequentist approach, even if flawed in certain respects, still serves as a valuable heuristic. It teaches us to be wary of overfitting to outcomes, to ask about the process behind the numbers, and to maintain a healthy skepticism when interpreting results. Its insistence on method over outcome protects us from the temptation to rationalize or cherry-pick. I'd rather a scientist work with p-values than with their intuition alone.
I think I largely agree with the spirit here. I definitely think p-values have issues and in particular they way they have arguably contributed to publication bias is a highly reasonable criticism. That said, I think people like to make these "methodological" critiques more for philosophical than statistical reasons. In practice, we definitely should expect the application of all methods is going to have issues and be highly imperfect. So I agree that it makes sense to have a practical, "all methods are flawed, some are useful" view of things.
Encode successfully navigated this, by not offering facts (who did what, and when), since they don't have any first-hand knowledge of the facts. What they offered, according to their brief (which is attached as a sechedule to the "main document" for document 72), was their philosophical and technical perspective, particularly as a public body concerned with AI safety vis a vis the change in structure of OpenAI.
Didn't the Encode brief do stuff like quote public opinion polls? Sure they characterize it as offering a philosophical perspective (how it would be called "technical" I'm not sure) but to me it came across as basically asserting policy rather than legal arguments. Sure there is a consideration of the public interest for the preliminary injunction but the overall feel to me was very much policy rather than legal arguments. I also don't think you're necessarily applying this standard evenly to both briefs. The ex-OpenAI brief I think can be seen in a similar way, it just brings in additional pieces of evidence to make that case.
If you read through the two proposed briefs, they are night and day. Encode describes the interest that the public might have in OpenAI continuing under its present structure, compared to transitioning to a for-profit enterprise, the risk of AI, and why it should be avoided. The employee brief recounts meetings, memos, and who was making promises.
In my view the high-level arguments of both briefs are the same in that they argue that having actual control of future AI systems residing with a non-profit is in the public interest. It's just that the ex-OpenAI brief brings in more information for purposes of suggesting that such a belief was not uncommon among OpenAI employees and that we might reasonable view OpenAI to have committed to such a thing and understood this to be consistent with its charitable purpose. I could see how that might not be relevant to the case since it doesn't necessarily go to Musk's reliance, so perhaps it makes sense to not muddy the waters with it, but I don't think its the case that the ex-OpenAI brief somehow lacked any relevance if we assume the Encode brief was relevant.
In a very abstract way, Encode is basically saying that the transition shouldn't proceed because it would be bad for society and humanity. This is a perspective that isn't captured by either Musk or OpenAi/Microsoft.
This is relevant because its a factor for preliminary injunction purposes. I haven't gone back and read all the documents, but it would be very surprising to me if Musk didn't argue that the for-profit transition was contrary to the public interest. Also it seems to me like the ex-OpenAI brief also casts their arguments in these same terms.
The employees tried to say that OpenAI and Altman made promises to them, and those promises should be kept, which is almost entirely factual.
I think the brief is trying to argue that these facts go to the very point you identify the Encode brief as addressing.
The conceptual reasoning is simple and compelling: a sufficiently sophisticated deceptive AI can say whatever we want to hear, perfectly mimicking aligned behavior externally. But faking its internal cognitive processes – its "thoughts" – seems much harder. Therefore, goes the argument, we must rely on interpretability to truly know if an AI is aligned.
I think an important consequence of this argument is that it potentially suggests that it is actually just straight-up impossible for black-box evaluations to address certain possible future scenarios (although of course that doesn't mean everyone agrees with this conclusion). If a safe model and extremely dangerous model can both produce the exact same black-box results then those black-box results can't distinguish between the two models[1]. Its not just that there are challenges that would potentially require a lot of work to solve. For the interpretability comparison, its possible that the same is true (although I don't think I really have an opinion on that), or that the challenges for interpretability are so large that we might characterize it as impossible for practical purposes. I think this is especially true for very ambitious versions of interpretability, which goes to your "high reliability" point.
However, I think we could use a different point of comparison which we might call white-box evaluations. I think your point about interpretability enhancing black-box evaluations gets at something like this. If you are using insights from "inside" the model to help your evaluations then in a sense the evaluation is no longer "black-box", we are allowing the use of strictly more information. The relevance of the perspective implied by the quote above is that such approaches are not simply helpful but required in certain cases. Its possible these cases won't actually occur, but I think its important to understand what is or is not required in various scenarios. If a certain approach implicitly assumes certain risks won't occur, I think its a good idea to try to convert those implicit assumptions to explicit ones. Many proposals offered by major labs I think suffer from a lack of explicitness about what possible threats they wouldn't address, vs arguments for why they would address certain other possibilities.
For example, one implication of the "white-box required" perspective seems to be that you can't just advance various lines of research independently and incorporate them as they arrive, you might need certain areas to move together or at certain relative rates so that insights arrive in the order needed for them to contribute to safety.
As I see it we must either not create superintelligence, rely on pre-superintelligent automated researchers to find better methods, or deploy without fully reliable safeguards and roll the dice, and do as much as we can now to improve our odds.
Greater explicitness about limitations of various approaches would help with the analysis of these issues and with building some amount of consensus about what the ideal approach is.
Strictly speaking, if two models produce the exact same results on every possible input and we would categorize those results as "safe" then there is no problem, we've just said the model is proven safe. But practically there will be a distributional generalization issue where the results we have for evaluation don't cover all possible inputs.
I had a similar thought, and that would make sense to me, but I just don't know enough about the standards to say what the correct interpretation is. To an extent I feel like its kind of tea-leaf reading and maybe isn't a good idea, but at the same time I feel like these dynamics could be relevant to how views on AI safety develop among groups that are exposed to those ideas in these formats. I definitely think this won't be the last court case by far that implicates AI issues, so I feel its worth thinking about how different courses of action could play out.
Going through the proposed brief, it seems that nearly all of the substantive content is factual, being descriptions of processes, statements, and general history. Plainly speaking, this is just inappropriate and has rightly been dismissed.
This definitely seems reasonable, but in that case, why accept the Encode brief? Obviously they are different documents, but my read of the Encode one was that it had similar issues, but (unless I'm missing something?) it appears that it was accepted?
Editing to add: Not sure if the diff is whether the parties all agreed to the Encode brief? I recall the ex-OpenAI employee one noting they didn't have consent of all parties, I don't remember if this was mentioned one way or the other in the Encode one.
There are also some prominent law professors involved in the ex-OpenAI employee brief, Lawrence Lessig appears to be representing them and Eugene Volokh seems to have served as local counsel. Although I have no special knowledge of either of them, while I could see Lessig as someone who might "yolo-sendit" on something like this, the Volokh inclusion to me would make it a bit surprising if the brief were just straight-up inappropriate. But I don't really know how the dynamics of being "local counsel" for something like this work?
Part of my reason for writing my previous post on this case is that I think the reaction in the AI safety community has somewhat misread the situation, in fact a highly upvoted comment on an EA forum post related to this case explicitly suggests the idea of submitting amicus briefs. I had originally drafted a section about amicus briefs but decided to exclude it because the fact that the Encode brief got accepted made me wonder if I myself was misreading the judge's receptivity to such briefs.
Regardless, I think if what you suggest is true it supports the idea that some AI safety people haven't necessarily correctly read the judge's disposition, would you be interested in commenting on that?
If Elon wanted these facts to go in, he could have contacted these employees ahead of time and had them provide evidence directly. The employees don't get to perform an end-run around the rules of evidence by putting all of their factual assertions into a brief.
This definitely makes sense to me.
With respect to your question about the request for judicial notice, the Order refers to the filing that had requested that relief (document 104), and the documents that they were requesting the Court to take notice of were the following
Thanks, I will update the post with this info, appreciate you taking the time to look into it.
I think its worth thinking about what you are trying to achieve in any given discussion. Why do you need the person to acknowledge what you believe is their true interest? I do think people often describe there interests as being about finding or demonstrating "the truth" for a lot of topics, but to me there is a large possibility that going down that road mostly gets into semantics.
I'm not entirely sure I understand your point regarding racism/sexism, but I can imagine something like this. Someone has a belief that is considered racist. When confronted about why they talk about this belief so much, they say its because they think its true and its super important to stand up for the truth. My view is, often the person probably does genuinely believe the thing is true, but their degree of focus does come partially from what you say, the desire to have their belief not considered racist. Which one is the "real" reason? I think its kind of hopelessly entangled. They do believe the thing is true but its not like they are going around being really concerned about telling people the sky is blue, even though they also believe that is true. Often times if you are in a discussion related to this you can focus on whether the belief in question is or is not true, whether it is or is not racist, that kind of thing. I think you will often get more productive discussions this way than to go into what the individual person's "real reasons" are, unless you have an interest in them specifically (like they are your friend and you really want convince them of something about the topic). Even in that case, its not clear you can really "untangle" the reasons, but you might want to go more into their psychology if you care about them specifically.
If you know the person you're talking to well, have a shared understanding of what is meant by the phrase, and have a strong sense that you are working together and it won't be taken negatively, then I can see this working out. But I think there are a lot of situations that lack this shared context and that are more adversarial. It seems to me like you can often get across similar info in a more specific way. For example, if you think someone is rationalizing you can focus on the underlying issue and hope to "shake them out of it" by walking through the logic of the issue, or you could identify a more specific "meta-issue" if you want to go to the meta-level. That would depend on exactly how they are "rationalizing", although again if you have a strong common understanding of what "truth-seeking" means, perhaps that is the best way to describe the meta-issue in your case.
I'd go with "I don't think this conversation is helping us ________"
I think this can be fine, especially if in a context where you know the person well and you are working together on something explicitly (e.g. coworker), but I think this often won't work in context where you don't know the person well or that are more adversarial.
If you're having a debate that's mostly for an external audience, then maybe you should just call out that the liar is lying.
Sometimes you should be I think its totally possible you shouldn't (and I think you should have a very high bar for doing so).
If you're trying to work with them, then it's probably better to try to figure out what aspect of the conversation is motivating them to lie and trying to incentivise telling the truth instead. If you can't do that then it doesn't really matter what's going on in their head; you're not going to have a productive conversation anyway.
This will obviously depend on the situation, but I think its totally possible that you can have a productive conversation even when someone is lying, you don't call them out, and you can't motivate them to tell the truth. It just depends what counts as "productive" from your perspective. That should depend primarily on your goals for the conversation, not some cosmic principle about "truth-seeking". If I'm trying to buy a car and the salesperson lies to me about how the one I'm looking at is surely going to be sold today, I can just keep that to myself and use my knowledge to my own advantage, I don't have to try to make the salesperson more honest.
I'm definitely not representative of lesswrong in my my views above I don't think. In fact in some sense I think I'm shadowboxing with lesswrong in some of my comments above, so sorry about any confusion that introduced.
I don't think I've ever seen maximum likelihood vs maximum a-posteriori discussed on lesswrong, and I'm kind of just griping about it! I don't have a references off to top of my head but I recall this appearing in debates elsewhere (i.e. not on lesswrong) like in more academic/stats settings. I can see if I can find examples. But in general it addresses an estimation perspective instead of hypothesis testing.
I think I'm in agreement with you here. My "methodological" was directed at what I view as a somewhat more typical lesswrong perspective, similar to what is expressed in the Eliezer quote. Sure, if we take some simple case we can address a more philosophical question about frequentism vs bayesianism, but in practical situations there are going to so many analytical choices that you could make that there are always going to be issues. In an actual analysis you can always do stuff like look at multiple versions of an analysis and trying to use that to refine your understanding of a phenomenon. If you fix the likelihood but allow the data to vary then p-values are likely to be highly correlated with possible alternatives like bayes factors, a lot of the critiques I feel are focused on making a clean philosophical approach while ignoring the inherent messiness that would be introduced if you ever want to infer things from reasonably complicated data or observations. I don't think swapping likelihood ratios for p-values would sudden change things all that much, a lot of the core difficulties of inferring things from data would remain.