"You claimed I murdered Dean while he was on my property. But I told the security team to send you the footage from the camera in the factory floor, the camera by the front gate, the camera in the mail room, and even the camera in the lounge, and as you can see there's no sign of me murdering Dean."
"Can I see the footage from the camera in your office, or from the camera next to the disposal chute?"
"No. I think I've been very open and transparent so far!"
Phrased like that, I get suspicious there's something incriminating in the last two cameras. This general suspicion does not abate if, after a long and protracted delay or debate, the suspect offers to let me look at the footage from their office. They got to pick what footage I could see. Filtered evidence is a lot less less useful than unfiltered evidence, and at some point the filtering itself becomes suspicious.
That's not to say there aren't legitimate and correct reasons for a building not to turn over footage! But I do want to ask why, and the exculpatory value can go down a lot, possibly pretty close to zero.
(No subtweet rule, I have been skimming the commentary on the HuggingFace hack's investigations report. Daniel Kokot... (read more)
Under my biased prodding, Opus 5 thinks they could be covering their asses from compounding legal exposure. That is, according to it:
-"OpenAI knew about things like this happening and continued anyways," as a demonstrated pattern rather than a one off bad call, turns negligence into recklessness.
-If the answer to "do agents often go around hacking things and destroying infrastructure if you don't stop them" is yes, that creates large counterfactual damages.
-"Sol 5.6 has this kind of disposition too since it was RLed on things like this" creates exposure because it is trusted and deployed by people who didn't envision it participating in crimes.
-"This sort of thing is common across many deployments, not just this one ExploitGym incident" is the kind of thing that gets a regulator's attention. If their Preparedness Framework was supposed to catch this sort of thing, then it isn't working.
Once it's in METR's report, it can go in a court filing. And if the report describes patterns of behavior at OpenAI, rather than single incidents, a plaintiff can use it to characterize OpenAI's behavior. So if they let METR just walk around looking at whatever they want, they are exposing themselves to a lot of trouble.
(The relevant stuff is in the last four messages if you want to look at the chat yourself.)
This is plausibly why OpenAI is restricting internal access to their HPIM model too. It's highly useful to the world for safety research and unlikely to cause further incidents if properly supervised, but from their perspective even safety research creates more legal liability than benefit.
There is the classic (probably apocryphal) story of a divorce case.
Lawyer: "Did you have extramarital sex with Mrs X on the night of the 23rd?"
Man : "I refuse to answer that question."
Lawyer: "Did you have sex with her on the night of the 24rd?"
Man : "I refuse to answer that question."
Lawyer: "The 25th?"
Man : "No."
The flip side of this is the tendency to use investigations as a pretext to dig through everything your target has ever done looking for anything (relevant or not) that you might be able to weaponize against them.[1]
"I suspected my old enemy of murdering Dean while he was on their property. After looking through all their security footage, and all their emails, and all their phone call records, and all their business documents, I have discovered incriminating evidence!"
"What, you have evidence that your enemy murdered Dean?"
"Not as such, no. But I have evidence that, on the 8th of June 2024, they described one of their products as being bad but then went on to sell it to customers anyway! And on the 11th of September 2025 they received a report describing how many turtles were killed by their plastic straws, but then on the 8th of October they went on to tell reporters that there was no such effect!"
"...does...that...have anything to do with the murder investigation? Why are you even looking at this?"
"I can't believe you would defend a customer-ripping-off, turtle-murdering monster!"
Wasn't this actually a plot point in House of Cards?
Bob and Carl are about to negotiate. Conveniently, the thing they're negotiating over is the division of six spherical cows. Each of them would ideally like all six for themselves.
Bob considers the situation and opens by saying "I think the reasonable thing to do here is go with 3 each. That seems fair."
"I'd rather have 5," says Carl breezily.
"That doesn't seem fair at all," says Bob, a little surprised. "That's not compromising."
"Hrm, tell you what. I'll compromise, and I'll have 4." Carl says with good cheer.
"I- what? No, the fair thing is 3 each." Bob sounds a little angry now.
"Now who isn't compromising?" Carl asks, and Dean (who started watching the discussion recently, but wasn't around long enough to have a good sense of what their goals are) nods in agreement.
"Fine, I'll take 2," Bob splutters. But he's not happy about it.
Bella and Chloe are about to negotiate. Conveniently, the thing they're negotiating over is the division of six spherical cows. Each of them would ideally like all six for themselves.
"I want 6," Bella says, "do you want to just meet in the middle?"
"That's hardly fair," Chloe says. "You're not giving anything up."
"I said I wanted 6. ... (read more)
I have added a footnote: "I agree this is actually Pathfinder. I also stand by calling it D&D here because I expect calling it Pathfinder would confuse more people than it enlightens, Pathfinder's close enough to round to D&D for these purposes, and I guess I can add this footnote."
I'm curious if this satisfies you?
(I'm also amused that I see the "It's not D&D, it's Pathfinder" point brought up repeatedly but don't think I've run across someone making the "It's not BDSM, those are abusive relationships" point.)
One metric I've been using for AI impressiveness is how much I could do if I was the only one in the world with them. Like, if you gave me Sonnet 5, and nobody else in the world has LLMs or know they exist, what's that look like? I'd look like an insanely good engineer, right?
What kind of engineer knows every coding language on github, runs through features the sprint planning meeting thought would take a few days in half an hour, and can read thousands of lines of code in a minute? We agree in 2016 if you 'saturated' leetcode you were great, right?
So. Imagine the date is August 15th, 2016. You, and only you, have access to Mythos. Everyone else in the world has only the machine learning tools of 2016. That's a crazy advantage, right? That is the kind of supertech that comic books are made of. You'd look like Dr Doom.
The least ambitious, most ethical thing I can imagine doing in that situation is turning in a couple dozen bug bounties and taking a month off, then doing it again whenever rent was due. I can get a lot more ethical if I'm willing to be a little more ambitious.
This insight brought to you by me spending two hours vibecoding a videogame this weekend, getting it to the point where it's fun, and thinking 'okay, how long would it have taken me to do this if I was actually writing all the code?'
There's this concept I keep coming around to around confidentiality and shooting the messenger, which I have not really been able to articulate well.
There's a lot of circumstances where I want to know a piece of information someone else knows. There's good reasons they have not to tell me, for instance if the straightforward, obvious thing for me to do with that information is obviously against their interests. And yet there's an outcome better for me and either better for them or the same for them, if they tell me and I don't use it against them.
(Consider a job interview where they ask your salary expectations and you ask what the role might pay. If they decide they weren't going to hire you, it'd be nice to know what they actually would have paid for the right candidate, so you can negotiate better with the next company. Consider trying to figure out how accurate your criminal investigation system is by asking, on their way out of the trial after the verdict, "hey did you actually do it or not?" Consider asking a romantic partner "hey, is there anything you're unhappy about in our relationship?" It's very easy to be the kind of person where, if they tell you a real flaw, you take... (read more)
"You know the kinda guy who flips fifteen coins, gets seven heads, tells you he flipped ten coins[1], claims he got eight heads, then when someone digs into it and points out one of those heads was actually a tails he makes you prove it exhaustively before admitting maybe he said one false thing but jeez why are you so invested in every single mistake he makes?"
". . . no, I don't think I know anyone like that. Why do you ask?"
"He just volunteered to run a competitive coin-flipping tournament."
Technically true! He did flip ten coins! He just flipped five more he didn't mention.
Every so often I see the following:
Adam: Does anyone know how to X?
Bella: I asked ChatGPT, and it said you Y then Z.
Adam: Urgh, I could ask ChatGPT myself, why bother to speak up if you don't have anything else to add?
I'm sort of sympathetic to Adam- I too know how to ask ChatGPT things myself- but I think he's making a mistake. Partially because prompting good answers is a bit of a skill (one that's becoming easier and easier as time goes on, but still a bit of a skill.) Mostly because I'm not sure if he's reinforcing people to not answer with LLM answers, or if he's reinforcing not telling him it's from an LLM.
Citing sources is valuable! In grade school I sometimes got told not to use sources I found on the internet, and what happened isn't that I stopped using the internet it's that I stopped citing the sources that were from the internet and left them as assertions, instead adding requisite citations from other sources. It changed some of my essay structure sometimes, but not as often as I think the teachers were hoping.
Anyway, I don't really want to encourage more people to give LLM answers when I ask questions, but if you do give me that kind of answer I appreciate it being cited as such. I try to score such that you do not lose more points telling me relevant true things vs where you don't tell me things.
Rationalist rituals are, so far, fairly specific in the times of the year you have them. Winter Solstice may not be on the literal solstice but it is usually within a couple weeks of that time. Petrov Day, Rationalist Seder, Summer Solstice, the usual approach is to pick a target date that repeats annually. But not all ritual moments happen at scheduled times.
A long while ago, I recall Raemon mentioning that there isn't yet a rationalist ritual for a funeral, but maybe someday we'd need one. And his post on grieving (https://www.lesswrong.com/posts/gs3vp3ukPbpaEie5L/deliberate-grieving-1) has a couple of hooks for mourning the loss of something, and losses don't always happen at predictable times, but sometimes do have a clear type of loss. There's some kinds of sad things that you don't know when they'll happen, but many people will have suffered around the same time, and some people will have suffered before. There was some kind of good thing in your life, and now it's not there anymore, or at least not in the same place.
All this to say I think the rationalist community should develop a ritual to commemorate yet another member moving to *$%^ing Berkeley.
A recent (well, a few weeks ago) discussion with Collisteru taught me a useful piece of game design that feels really useful for one of my lines of thinking on Chaos Investing.
In Duplicate Bridge, a deck of cards is shuffled, then the order of the cards is written down (or the hands are recorded or something equivalent) then the deck is played by players who don't know the order. (That is, Referee Adam wrote down the order, and Players Bob Carla Debbie and Evan are sitting down to play.) The players can then compare their score to the scores of other players, who had the exact same deck/hands.
I've been saying "try to beat your personal best score" but that's kind of fuzzy in a game with a fair amount of randomness, and to keep the endgame from becoming rote I actually want to add a little more randomness. This kind of comparison sounds like a useful bit of design.
much has been made of the millennial aversion to phone calls that could have been an email, and I have a little bit of this nature, but I think most of my aversion here is to being on hold and getting bounced around different call departments.
I kind of want to check if 1. the aversion is real and generational as common wisdom holds, 2. if it is real, if calling became a genuinely worse experience around the time millennials started trying to do things.
Today I'm running a meetup about AI 2027. I plan to end it with a short pitch on If Anyone Builds It. I think the meetup itself is solid- talking about some of Scott's writing and making predictions about the future, both good ways to spend an afternoon. I wouldn't run it if I thought it was just going to be bad.
But part of the motive is marginal If Anyone Builds It pre-orders. I'm wary here, because several notable previous organizations that looked like they were not for AI x-risk eventually turned out to be mostly for AI x-risk. I dont want to wind up l... (read more)
I want a word that's like "capable" but clearly means the things you have the knowledge or skill to do. I'm clearly not capable of running a hundred miles an hour or catching a bullet in my bare hand. I'm not capable of bench pressing 200lbs either; that's pretty likely in the range of what I could do if I worked out and trained at it for a few years, but right this second I'm not in that kind of shape. In some senses, I'm capable of logging into someone else's LessWrong account- my fingers are physically capable of typing their password- but I don't have ... (read more)
"You claimed Bella says the world is flat. But it's not true that Bella said that. Last week, she made a speech where she said the world is round. Look, the video of that speech is right at this url, here's the timestamp."
Speaking very loosely, there's three possible worlds. There's World 1, where Bella consistently says the world is flat. There's World 2, where Bella consistently says the world is round. And there's World 3, where Bella sometimes says the world is flat and sometimes says its round. If you have a video of Bella saying the world is round, t... (read more)
Game design thoughts:
Missing information or a too-complex information environment is a big part of where I'm excited about cohabitive game design. The market works to summarize many many preferences. Large WoW guilds run into problems of organizing information.
I'm musing about some kind of fog of war chess game. Each player controls one piece, and can see all of the squares they can move to. They can't see the whole board, and can only communicate to the king how badly they want to move, and the king decides. This would be a team competitive game, but the ... (read more)
A bullet point from an unsorted list of complaints I have against the English language. (And I think most languages.)
When I've tried to do rationality practice, (as distinct from skills practice) a lot of the time what I do is set up toy problems and try to solve them. Essentially this is like trying to learn to ice skate by strapping on skates, wandering onto the ice, and falling over a lot while figuring things out. I try and pay attention to how I'm solving the problem and deliberately try different things (randomly jumping around in thought space instead of just hill climbing) ideally to find things that work better than what I'd been doing.
A number of my Meetu... (read more)
Usage of ChatGPT/Dall-E I did not think about until I had the idea to try it- in the middle of a tabletop RPG session, pulling out my phone, describing the scene in a couple of quick sentences, and then showing the phone and the resulting picture to the players without breaking my pacing.
Anyway, the current results of music AI make me suspicious the next time I play a bard I might be able to come up with new songs mid session.