Popular Comments
Recent Discussion
One of the first things they teach you in algebra is that the letters you use to signify variables are arbitrary, and you can use whatever you want[1]. Like most of the 'first things' students are taught, this is almost entirely a lie: every letter has implicit connotations, and if (for example) you use "n" for a non-integer variable, it'll confuse someone reading your work. More importantly, if you don't know what symbol choices imply, it'll be harder for you to understand what an equation is implicitly communicating, making it even more difficult to grasp the concepts that are actually being laid out.
So I've decided to go through the English alphabet and explicitly explain the connotations of each character as they might be used by a [unusually-bright-highschooler|reasonably-clever-college-student]-level...
z: The default for complex numbers.
My guess is that for now, I'd give around a 10-30% chance to "AI winter happens for a short period/AI progress slows down" by 2027.
Also, what would you consider super surprising new evidence?
I honestly have no idea what you mean. I am not even sure why "(self) statements you hear while on psychedelics are just like normal statements" would be a counterpoint to someone being in a very credulous state. Normal statements can also be accepted credulously.
Perhaps you are right but the sense of self required is rare. Practical most people are empirically credulous on psychedellics.
I've been thinking of writing up a piece on the implications of very short timelines, in light of various people recently suggesting them (eg Dario Amodei, "2026 or 2027...there could be a mild delay")
Here's a thought experiment: suppose that this week it turns out that OAI has found a modified sampling technique for o1 that puts it at the level of the median OAI capabilities researcher, in a fairly across-the-board way (ie it's just straightforwardly able to do the job of a researcher). Suppose further that it's not a significant additional compute expens...
Been trying to put together a framework for analyzing the way people think, process, approach, and prioritize information; for some time now. The same few patterns seem to come up rather consistently. The following is an attempt to systematize them a little bit.
To be clear, a few of these appear to be rather involuntary, naturalistic, and compulsive: less of a "this is the way people are", more of a "way they routinely choose to be".
Nevertheless, the more I think about them, the more they make sense.
Pointers
By the far the most common, most reactive, most emotional, and the most vocal of all. In politics, the first to repost any and all "events" they happen to personally resonate with. Whether towards the positive or negative side of the spectrum,...
Here ya go....
"The exercise of individual autonomy cultivates a sophisticated understanding of self, the external world, and their intricate interplay. However, a lack of rhetorical training leaves one ill-equipped to navigate a world shaped by discourse, unable to persuade effectively or discern manipulation. This deficiency creates a void at the core of their being, a space where self-directed learning, particularly in rhetoric, could have empowered them to overcome this vulnerability."
Tom Davidson did the original thinking; Rose Hadshar helped with later thinking, structure and writing.
Some plans for AI governance involve centralising western AGI development.[1] Would this actually be a good idea? We don’t think this question has been analysed in enough detail, given how important it is. In this post, we’re going to:
- Explore the strategic implications of having one project instead of several
- Discuss what we think the best path forwards is, given that strategic landscape
(If at this point you’re thinking ‘this is all irrelevant, because centralisation is inevitable’, we disagree! We suggest you read the appendix, and then consider if you want to read the rest of the post.)
On 2, we’re going to present:
- Our overall take: It’s very unclear whether centralising would be good or bad.
- Our current best guess: Centralisation is
Your infosecurity argument seems to involve fixing a point in time, and comparing a (more capable) centralized AI project against multiple (less capable) decentralized AI projects. However, almost all of the risks you're considering depend much more on the capability of the AI project rather than the point in time at which they occur. So I think best practice here would be to fix a rough capability profile, and compare a (shorter timelines) centralized AI project against multiple (less capable) decentralized AI projects.
In more detail:
...It’s not clear whethe
TLDR: LessWrong + Lighthaven need about $3M for the next 12 months. Donate, send me an email, DM, signal message (+1 510 944 3235), or leave a comment, if you want to support what we do. We are a registered 501(c)3, have big plans for the next year, and due to a shifting funding landscape need support from a broader community more than in any previous year. [1]
I've been running LessWrong/Lightcone Infrastructure for the last 7 years. During that time we have grown into the primary infrastructure provider for the rationality and AI safety communities. "Infrastructure" is a big fuzzy word, but in our case, it concretely means:
- We build and run LessWrong.com and the AI Alignment Forum.[2]
- We built and run Lighthaven (lighthaven.space), a ~30,000 sq. ft. campus
I donated. I think Lightcone is helping strike at the heart of questions around what we should believe and do. Thank you for making LessWrong work so well and being thoughtful around managing content, and providing super quality spaces both online and offline for deep ideas to develop and spread!
2EVERYONE, CALM DOWN!
Meaning Alignment Institute just dropped their first post in basically a year and it seems like they've been up to some cool stuff.
Their perspective on value alignment really grabbed my attention because it reframes our usual technical alignment conversations around rules and reward functions into something more fundamental - what makes humans actually reliably good and cooperative?
I really like their frame of a moral graph and locally maximally good values to follow as another way of imagining alignment, it is a lot more similar to that which happened during cultural evolution as explored in for example The Secret of Our Success. It kind of seems like they're taking evolutionary psychology and morality research and group selection and applying the results to how to align models...
In my opinion, theoretically, the key to have "safe" humans and "safe" models, is "to do no harm" under any circumstances, even when they have power. This is roughly what law is about, and what moral values should be about (in my opinion)
Cross-posted from my NAO Notebook.
This is an edited transcript of a talk I just gave at CBD S&T, a chem-bio defence conference. I needed to submit the slides several months in advance, so I tried out a new-to-me approach where the slides are visual support only and I finalized the text of the talk later on. This does mean there are places where it would be great to have additional slides to illustrate some concepts
Additionally, this was the first time I gave a talk where I wrote out everything I wanted to say in advance. I think this made for a much less engaging talk, and for future ones I'm planning to go back to speaking from bullets.
...
For something in the range of $10M/y we think you can operate a system capable of detecting a novel pathogen before 1:1000 people have been infected.
Sounds promising! I assume this is for one location, so have you done any modeling or estimations of what the global prevalence would be at that point? If you get lucky, it could be very low. But it also could be a lot higher if you get unlucky.
Have you done any cost-effectiveness analyses? Do you think that many people would be willing to take actions to reduce transmission etc in a case where no one has gotten sick yet?
