Thief With Access To The Real Diamond

That leaves a gap between cases in which the thief has access to the predictor/planner/reporter (which run into the diagonalization barrier), and cases in which the thief doesn’t even have detailed information about the diamond (in which case we can solve the problem). In between those, the thief has arbitrarily good information about the diamond, but does not have access to the predictor/planner/reporter.

For this case, I do not have a solution, but I do expect that it’s solvable-in-principle.

The main difficulty in these cases is that the $P\left[Obs|do\right(A^{\ast }\left)\right]$ may be exactly the same between action-plans $A^{\ast }$ which result in the diamond being stolen, and action plans which do not. The thief has enough information to make the predicted sensor readings completely identical between the two cases.

The reason the problem might still be solvable-in-principle is that the actions $A^{\ast }$ still give us information about whether the diamond was stolen. But we somehow have to extract that information in a way which does not require any difference whatsoever between the observation-distributions in the two cases.

I was chewing over the ELK problem this morning and had a silly idea here. What if you deliberately introduce a specific inaccuracy in the camera, or the reporter? Have some timestamp or watermark or scratch, something that has to do with the system's perception of the diamond and which would not be known from mere access to the diamond or at least be very expensive to fake.

Thanks for the post! It makes me wonder whether hyperfinite quantification and hyperreal truth values handle Yablo-esque paradoxes too, but I haven't thought about that much at all yet.

This is an important bit of clarification! You can do some entertaining countersignaling with a nice suit jacket and an unbuttoned button-down.

Both of these examples show a similar mental move: (encounter a fantasy) -> (figure out how to bring that fantasy into reality)...

To me, this move feels particularly foundational and load-bearing; it’s a core part of a healthy agentic human mind. It’s the mental move...

This resonates with something in me. Something kinda malnourished and shaky and weirdly shaped but it resonates all the same. Most of the things I like best about myself, that I'm proudest of having done or that have caused other people to be impressed with me or that nourished me to do, were things that had this pattern. I'm trying to do more of that now. Thanks for painting it so clearly.

I tried, but never really figured out how, to get them to understand that they were engaging in process of downloading a consensus to defer to, and why it matters that that's what they're doing.

Here's the thing, as one such person you mentored and who you counseled not to defer: you can realize you're deferring, and recognize that it's not particularly endorsed, and then do it anyway because all the social pressures point that way, and so do all the grantmaking pressures, and all the selection effects. It totally matters that that's what you're doing! But being openly non-deferential is a very good way to paint a target on your back, to take on huge costs that you would never agree to up front. Even being quietly privately non-deferential can and will show up in the kinds of research-flavored conversations you have, and will cause you to miss connections and get excluded. You'll wind up being held to the epistemic standards of the deferrees - much higher than those of your peers; never mind that that's not the standard most other people get held to. You'd better be willing and able to lay out your entire argument, from scratch, for anyone who asks you, or you look worse than merely stupid - you look like a waste of time.

So... the biggest savings here, by far, is the rent. At a guess it's bigger than everything else combined. If you don't have enough friends, or your friends all live in full houses, guess you're screwed here. Hope we're OK with the tyranny of structurelessness?

I'd like to see a breakdown by "years since doing MATS". What's the retention like, basically? Another breakdown I'd like to see, either for the displayed data or the years-since-MATS one - what's the split in AI safety between "(co)founded an org", "joined a new org", "joined an established org/(research/policy) group (technical vs governance)", and "on a grant, no real org", along with the existing "academic" (split by core academic/alt-academia?), "government" (which country?), and for-profit (maybe a breakdown of product type?). In any case, thanks for posting this!

First off, TFTP. I marked some stuff I thought was most relevant. This is helping remind me of some things I think about LLM confabulation and lack of binding/reasoning... I don't have my thoughts fully formed but there's something here about global inconsistency despite local compatibility, and how that cashes out in Problems. Something a little like an inability to define a sheaf, or homology detection, or something like that? I might say more better words later about it.

Thanks for writing this. I've noticed something in the same vein as well - for the last few months I've felt increasingly like a lot of the pieces and systems agent foundations types use have some kind of important commonality to them, though I'm not yet sure what form that could take. Condensation and natural latents and some of Francis Rhys Ward's work in II-MAIDS; ontology mismatch and Bayes nets and imprecise probability and category theory showing up repeatedly. There's something there to construct, but what?

I'm doing Budget Inkhaven: https://tiled-with-pentagons.blogspot.com/search/label/budget inkhaven

Similar to my earlier writing regimen, but shorter, faster, and slightly more daring. I'll crosspost anything I like especially well here.