The thing is, bot spam isn't generally high quality yet. On many platforms the tell of name-lotta-numbers is still a viable method for identifying a bot. And part of Anthropic's concern with the military was that AI can de-anonymize people across multiple accounts and platforms. That capability, if Dario is correct that it exists, seems in line with the ability to identify such a distillation attack. Or at least begin to. Once it begins, then RL means the AI is likely to get better at it over time. Or am I giving AI too much credit? I'm not sure.
In respons...
All three of the big US labs have recently accused various Chinese labs of large-scale covert distillation of their models, presenting evidence that the labs in question have been using thousands of fraudulent accounts to cover their tracks.
In the same way that models have learned to detect whether they are being given a real task or a training scenario and alter their responses, is it likely they will do the same for this scenario?
This would be fascinating, as would their responses. Especially if the models respond in different ways.
We used to have this great thing we called Star Trek. Now we don’t, because its shell is run by Alex Kurtzman who thinks he’s here to do deeply lazy social commentary and sci-fi action sequences and maybe some paint-by-numbers things that nominally look Trek-shaped. Which is why he says science fiction isn’t about the future.
The best Trek currently is the MMO Star Trek Online. Yes it has as many plot holes and oddities as TOS, TAS, and TNG but it also is remarkably true to them. It has admittedly terrible graphics (because it is very old). But it does have...
We have offered to work directly with the Department of War on R&D to improve the reliability of these systems, but they have not accepted this offer.
They note that when code or other artifacts are created by AI, users are less likely to check the underlying logic or identify missing context.
Anecdotes are not data, but I have observed that people create artifacts with AI they cannot create themselves. Therefore, fact checking might be outside the user's capabilities.
I saw this in Excel even before Claude. People ask the office Excel guru for help, then explain help means do it for me, then don't check anything and assume it is perfect. Now they're doing the same thing but with AI.
Everyone can talk, not everyone can make the artifacts.
Derek also notes some people can’t get value out of it, and attributes this to the nature of their jobs versus current tools. I agree this matters, but if you don’t find AI useful then that really is a you problem at this point.
I think both Derek and Zvi underestimate the level of dysfunction in government.
I work in state government, we cannot use AI, it is blocked (the sites are blacklisted in our browsers, and we cannot install any software at all on our computers) from our computers and forbidden by our leadership.
My agency has been trying to get permi...
People have already admitted doing this. The popups requesting authorization came too fast so they stop reading and just grant authority. This includes executives at AI companies. Again, last year, not in the future.
creating real world events, ‘showing up as real humans’ and forming real relationships.
I've been saying for years (think I'm the original source actually):
How it started: pics or it didn't happen
How it's going: IRL or it didn't happen
There appears to be a window of opportunity for people to become known as legitimately human. Given the speed of improvement in AI influencers, actors, etc. I predict that window will close in less than 2 years at which point it may be all but impossible to "prove" you are human online.
I really wonder how muc...
What were the big hits for applied AI this year? Were any of the big medical discoveries helped by AI tools? Not theoretical or "new study shows this will maybe happen eventually" but we're there any actual, tangible, AI-driven life improvements for normies this year?
We can steal [China’s] very best manufacturing engineers, and put them to work here.
I am not confident this strategy is viable. Given the Wikipedia page has a list of over 40 convictions (the accusa ions list is far longer) for espionage primarily in tech, and the rate of discovery of such people is increasing, it seems more likely that a high risk is being taken by any company that tries to hire defectors from China. I am insufficiently skilled to weigh the risk against potential gains given the uncertainties involved, but from the scope of the pro...
On #3, Not having read Mo's book what helps get me thinking the same way (though not as often as I would like) is "what does it feel like to be wrong. It doesn't feel bad, that's for when you know you are wrong. While you are still in the act of being wrong it feels exactly like being right." Which I first encountered as a meme and I don't know the source to appropriately attribute credit. But remembering the concept has helped me quite a bit. Mo's phrasing seems good, I shall add it to my box. The tool I am still working on is remembering to ask myself.
On...
Not the way you think. You are seeing the mask and trying to understand it, while ignoring the shoggoth underneath.
The topic has gotten a lot of discussion, but from the relevant context of the shoggoth, not the irrelevant point of the mask. Every post about how do we know when AI is telling the truth vs repeating what is in training data, the talk of p-zombies, etc. that is all directly struggling with your question.
Edit: to be clear, I am not saying the fact ais make mistakes shows they're inhuman, I am saying look at the mistake and ask yourself what do...
Yes, the data backs you up. In 2022 studies were showing that people had limited trust in AI, and even that varied by field. In 2023 the study came out showing that in blind trials patients overwhelmingly preferred AI chat it's over human doctors. (https://bytefeed.ai/ai-chatbots-bedside-manner-preferred-over-conventional-doctors-by-shocking-margin-according-to-blind-study/) In 2024 & 2025 we got the studies showing AI outperforms human doctors but patients still didn't trust it so long as they knew it was AI. Again, in blind studies patients prefer AI...
What is the latest on how the enteric nervous system plays into cognition and memory (being relevant here to your topic)? I have seen a lot of research on the role in behavior, especially disorders like anorexia and bulemia and gambling addiction, etc.. Given it is half the number of neurons as the brain, one thing that makes me hesitant about cryonics and digital personalities is the thought that what if people are only getting 2/3 or less of themselves because only the brain is being considered. But data on it is not my specialty.
I see three issues with your argument, two that don't change anything meaningful and one that does.
The two minor points:
You presume a world without preference for human made works. I posit this is a world that cannot exist under any circumstances where humans also exist. We are a species that pays a premium for art made by elephants and other animals. That shows off photos of our children and their accomplishments to people who we know don't care. We had pet rocks. The drive to value that which is valueless is, for whatever reason, deeply embedded in us. I...
Thanks for the confirmation on the navy. But you really, really need to check your data on transit costs.
Thanks!
Thank you!
Don't have access on desktop. Is there a way to format via mobile?
The other replies gave you good examples of how to resolve this. Let me take a stab at your mistake. From a high level, you are assuming the information contained in the scam is the only information the Bayesian has available to use.
As shown a Bayesian has probably got priors about the way the market works, the way people advertise, the existence and nature of scams, etc. The information in the predictions is applied against those priors, not just against itself.