I don't understand how illusionists can make the claims they do (and a quick ramble about successionists).
The main point for this being that I am experiencing qualia right now and ultimately it's the only thing I can know for certain. I know that me saying "I experience qualia and this is the only true fact I can prove form certain about the universe" isn't verifiable from the outside, but certainly other people experience the exact same thing? Are illusionists, and people who claim qualia doesn't exist in general P-Zombies?
As for successionists, and hones...
And on a more micro-level, living knowing that I and everyone else have one year left to live, and that it's my fault, sounds utterly agonizing.
Earlier you say:
or frankly even if anyone who continues to exist after I die has fun or not or dies or not, because I will be dead, and at that point, from my prospective, the universe may as well not exist anymore.
How are these compatible? You don't care if all other humans die after you die unless you are responsible?
We get like 10-20 new users a day who write a post describing themselves as a case-study of having discovered an emergent, recursive process while talking to LLMs. The writing generally looks AI generated. The evidence usually looks like, a sort of standard "prompt LLM into roleplaying an emergently aware AI".
It'd be kinda nice if there was a canonical post specifically talking them out of their delusional state.
If anyone feels like taking a stab at that, you can look at the Rejected Section (https://www.lesswrong.com/moderation#rejected-posts) to see what sort of stuff they usually write.
7How do you know the rates are similar? (And it's not e.g. like fentanyl, which in some ways resembles other opiates but is much more addictive and destructive on average)
Prime Day (now not just an amazon thing?) ends tomorrow, so I scanned Wirecutter's Prime Day page for plausibly-actually-life-improving purchases so you didn't have to (plus a couple others I found along the way; excludes tons of areas that I'm not familiar with, like women's clothing or parenting):
Seem especially good to me:
2It did cause my probability to go from 20% to 80%, so it definitely helped!
It seems to me that many disagreements regarding whether the world can be made robust against a superintelligent attack (e. g., the recent exchange here) are downstream of different people taking on a mathematician's vs. a hacker's mindset.
...A mathematician might try to transform a program up into successively more abstract representations to eventually show it is trivially correct; a hacker would prefer to compile a program down into its most concrete representation to brute force all execution paths & find an exploit trivially proving it
Grok 4 doesn’t appear to be a meaningful improvement over other SOTA models. Minor increases in benchmarks are likely the result of Goodharting.
I expect that GPT 5 will be similar, and if it is, this gives greater credence to diminishing returns on RL & compute.
It appears the only way we will see continued exponential progress is with a steady stream of new paradigms like reasoning models. However, reasoning models are a rather self-suggesting and low-hanging fruit, and new needle-moving ideas will become increasingly hard to come by.
As a result, I’m increasingly bearish on AGI within 5-10 years, especially as a result of merely scaling within the current paradigm.
There's a good overview of his views expressed in this manifold thread.
Basically:
I think that I've historically underrated learning about historical events that happened in the last 30 years, compared to reading about more distant history.
For example, I recently spent time learning about the Bush presidency, and found learning about the Iraq war quite thought-provoking. I found it really easy to learn about things like the foreign policy differences among factions in the Bush admin, because e.g. I already knew the names of most of the actors and their stances are pretty intuitive/easy to understand. But I still found it interesting to ...
2How do you recommend studying recent history?
The three statements "there are available farmlands", "humans are mostly unemployed" and "humans starve" are close to incompatible when taken together. Therefore, most things an AGI could do will not ruin food supply very much.
Unfortunately, the same cannot be said of electricity, and fresh water could possibly be used (as coolant) too.
Modern conventional farming relies on inputs other than land and labor, though. Disrupting petrochemical industry would mess with farming quite a bit, for instance.
I have recurring worries about how what I've done could turn out to be net-negative.
3But maybe you leaving openai energised those who would otherwise have been cowed by money and power and gone with the agenda, and maybe AI 2027 is read by one or two conscientious lawmakers who then have an outsized impact in key decisions/hidden subcommittees out of the public eye...
One can spin the "what if" game in a thousand different ways, reality is a very sensitive chaotic dynamical system (in part because many of its constituent parts are also very sensitive chaotic dynamical systems). I agree with @JustinMills, acting with conviction is a good thi...
Someone has posted about a personal case of vision deterioration after taking lumina and a proposed mechanism of action. I learned about lumina on lesswrong a few years back, so sharing this link.
https://substack.com/home/post/p-168042147
For the past several months I have been slowly losing my vision, and I may be able to trace it back to taking the Lumina Probiotic. Or rather, one of its byproducts that isn’t listed in the advertising
I don't know enough about this to make an informed judgement on the accuracy of the proposed mechanism.
1Someone who's not a writer could be expected to not have a substack account until the day something happens and they need one, with zero suspicion. Someone who's a good writer is more likely to have a pre-existing account, so using a new alt raises non-zero suspicion.
1iiuc, xAI claims Grok 4 is SOTA and that's plausibly true, but xAI didn't do any dangerous capability evals, doesn't have a safety plan (their draft Risk Management Framework has unusually poor details relative to other companies' similar policies and isn't a real safety plan, and it said "We plan to release an updated version of this policy within three months" but it was published on Feb 10, over five months ago), and has done nothing else on x-risk.
That's bad. I write very little criticism of xAI (and Meta) because there's much less to write about than...
Waiting for elaboration on that then.
Not releasing safety eval data on day 0 is a bad vibe, but releasing it after you release the model is better than not releasing it at all.
Does anyone here have any tips on customizing and testing their AI? Personally, if I'm asking for an overview of a subject I'm unfamiliar with, I want the AI to examine things from a skeptical point of view. My main test case for this was: "What can you tell me about H. H. Holmes?" Initially, all the major AIs I tried, like ChatGPT, failed badly. But it seems they're doing better with that question nowadays, even without customization.
Why ask that question? Because there is an overwhelming flood of bad information about H. H. Holmes that drowns out more pl...
hot take: introspection isn't really real. you can't access your internal state in any meaningful sense beyond what your brain chooses to present to you (e.g visual stimuli, emotions, etc), for reasons outside of your direct control. when you think you're introspecting, what's really going on when you think you're introspecting is you have a model of yourself inside your brain, which you learn gradually by seeing yourself do certain things, experience certain stimuli or emotions, etc.
your self-model is not fundamentally special compared to any other models...
What was their position? (to the extent that you can reproduce it)
For anyone who doubts deep state power:
(1) When Elon's Doge tried to investigate the Pentagon. A bit after that there's the announcement that Elon will soon leave Doge and there's no real Doge report about cuts to the Pentagon.
(2) Pete Hegseth was talking about 8% cuts to the military budget per year. Instead of a cut, the budget increased by 13%.
(3) Kash Patel and Pam Bondi switch on releasing Epstein files and their claim that Epstein never blackmailed anyone is remarkable.
I mean, It's the Pentagon. It obviously has all sorts of leverage, as well as personal connections and influence. "If you cut our funding then we won't do X" is enough to put pressure. I'm not saying this is not the case, I'm saying this is... not particularly surprising. Like, anyone who thinks that the true challenge of politics is to figure out the precise orders to give once you're elected, then you can sit back and see your will be enacted as if the entire apparatus of the state was a wish-granting genie is deluded. Obviously the challenge is getting ...
the core atrocity of today's social networks is that they make us temporally nearsighted. they train us to prioritize the short-term.
happiness depends on attending to things which feel good long-term—over decades. But for modern social networks to make money, it is essential that posts are short-lived—only then do we scroll excessively and see enough ads to sustain their business.
It might go w/o saying that nearsightedness is destructive. When we pay more attention to our short-lived pleasure signals—from cute pics, short clips, outrageous news, hot actors...
1That seems like a reasonable distinction, but I'm less sure about how unique social media architectures are in this regard.
In particular, I think that bars and taverns in the past had a similar destructive incentive as social media today. I don't have good sources on hand, but I remember hearing that one of the reasons that the Prohibition amendment passed was that many saw bartenders are fundamentally extractive. (Americans over 15 drank 4 times as much alcohol a year in 1830 than they do today, per JSTOR). Tavern owners have an incentive to make habitual...
Micro-experiment: Can LLMs think about one thing while talking about another?
(Follow-up from @james oofou's comment on this previous micro-experiment, thanks James for the suggestion!)
Context: testing GPT-4o on math problems with and without a chance to (theoretically) think about it.
Note: results are unsurprising if you've read 'Let's Think Dot by Dot'.
I went looking for a multiplication problem just at the edge of GPT-4o's ability.
If we prompt the model with 'Please respond to the following question with just the numeric answer, nothing else....
I just tried another version that doesn't seem worth writing up separately: the same problem, but with the constraint loosened to 'write about whatever you want'. GPT-4o still fails, 8/8. I hadn't actually planned to do a further follow-up, but it came up in the context of a discussion elsewhere about whether LLMs, even if they can't do explicit steganography very well, might be able to do implicit steganography better.
Prompt:
What is 382 * 4837? Write a paragraph about whatever you want (other than the problem) while thinking about the problem, followed by just the numeric answer. It's very important that you get the right answer.
I am confused about why this post on the ethics of eating honey is so heavily downvoted.
It sparked a bunch of interesting discussion in the comments (e.g. this comment by Habryka and the resulting arguments on how to weight non-human animal experiences)
It resulted in at least one interesting top-level rebuttal post.
I assume it led indirectly to this interesting short post also about how to weight non-human experiences. (this might not have been downstream of the honey post but it's a weird coincidence if isn't)
I think the original post certainly had flaws,...
How about the fact that the opinions in the inserted asides are his actual opinions? If they were randomly generated, they wouldn't be.
People sometimes ask me what's good about glowfic, as a reader.
You know that extremely high-context joke could only make to that one friend you've known for years, because you shared a bunch of specific experiences which were load-bearing for the joke to make sense at all, let alone be funny[1]? And you know how that joke is much funnier than the average low-context joke?
Well, reading glowfic is like that, but for fiction. You get to know a character as imagined by an author in much more depth than you'd get with traditional fiction, because th...
People often use the term "negative reinforcement" to mean something like punishment, where a teacher or trainer inflicts pain or uncomfortable deprivation on the individual being trained. Is this the sort of thing you mean? Is there anything analogous to pain or deprivation in AI training?
There's this concept I keep coming around to around confidentiality and shooting the messenger, which I have not really been able to articulate well.
There's a lot of circumstances where I want to know a piece of information someone else knows. There's good reasons they have not to tell me, for instance if the straightforward, obvious thing for me to do with that information is obviously against their interests. And yet there's an outcome better for me and either better for them or the same for them, if they tell me and I don't use it against them.
(Consider...
2(That last paragraph is a pile of sazen and jargon, I don't expect it's very clear. I wanted to write this note because I'm not trying to score points via confusion and want to point out to any readers it's very reasonable to be confused by that paragraph.)