Popular Comments
Recent Discussion
In light of reading through Raemon's shortform feed, I'm making my own. Here will be smaller ideas that are on my mind.
Thank you for collecting those links :-)
I've listened to two or three of the interviews (and ~three other talks from a long time ago), and I still have no clue what the central claims are, what the reasoning supporting them is &c. (I understand it most for Zvi Mowshowitz and Sarah Constantin, less for Jessica Taylor, and least for Benjamin Hoffman & Vassar). I also don't know of anyone who became convinced of or even understood any of Michael Vassar's views/stances through his writing/podcasts alone—it appears to almost always happen through in-person interaction.
[If you haven't come since we started meeting at Rocky Hill Cohousing, make sure to read this for details about where to go and park.]
We're the regular Northampton area meetup for ACX readers, and (as far as I know) the only rationalist or EA meetup in the Pioneer Valley area. We started in the 2018 Meetups Everywhere event and have been going ever since, though had a dip during the COVID era. We're currently meeting every other Saturday evening for most meetups (this meetup is an exception, mainly due to the holidays and me having been out of town for some medical tests in Boston).
Please join us for rationalist discussion, drinks and snacks at:
DATE & TIME
Sunday, December 15, 2024, 5:00 PM – roughly 8:00 PM (slightly different...
Cross-posted from my NAO Notebook.
This is an edited transcript of a talk I just gave at CBD S&T, a chem-bio defence conference. I needed to submit the slides several months in advance, so I tried out a new-to-me approach where the slides are visual support only and I finalized the text of the talk later on. This does mean there are places where it would be great to have additional slides to illustrate some concepts
Additionally, this was the first time I gave a talk where I wrote out everything I wanted to say in advance. I think this made for a much less engaging talk, and for future ones I'm planning to go back to speaking from bullets.
...
I assume this is for one location, so have you done any modeling or estimations of what the global prevalence would be at that point? If you get lucky, it could be very low. But it also could be a lot higher if you get unlucky.
We haven't done modeling on this, but I did write some a few months ago (Sample Prevalence vs Global Prevalence) laying out the question. It would be great if someone did want to work on this!
Have you done any cost-effectiveness analyses?
An end-to-end cost-effectiveness analysis is quite hard because it depends critically on ...
Someone I know wrote this very nice post explaining the core intuition around Shapley values (which play an important role in impact assessment and cooperative games) using Venn diagrams, and I think it's great. It might be the most intuitive explainer I've come across so far.
Incidentally, the post also won an honorable mention in 3blue1brown's Summer of Mathematical Exposition (I'm really proud of having given input on the post :).
Explaining the Shapley value in terms of the "synergies" (and the helpful split in the Venn diagram) makes much more intuitive sense than the more complex normal formula without synergies, which is usually just given without motivation. That being said, it requires first computing the synergies, which seems somewhat confusing for more than three players. The article itself doesn't mention the formula for the synergy function, but Wikipedia has it.
I'd say that we'd have a 70-80% chance of going through the next decade without causing a billion deaths if powerful AI comes.
My other explanation probably has to do with the fact that it's way easier to work with an already almost-executed object than a specification, because we are constrained to only think about a subset of possibilities for a reasonable time.
In other words, constraints are useful given that you are already severely constrained, to limit the space of possibilities.
In an attempt to get myself to write more here is my own shortform feed. Ideally I would write something daily, but we will see how it goes.
This is good. Please consider making it a top level post.
One of the first things they teach you in algebra is that the letters you use to signify variables are arbitrary, and you can use whatever you want[1]. Like most of the 'first things' students are taught, this is almost entirely a lie: every letter has implicit connotations, and if (for example) you use "n" for a non-integer variable, it'll confuse someone reading your work. More importantly, if you don't know what symbol choices imply, it'll be harder for you to understand what an equation is implicitly communicating, making it even more difficult to grasp the concepts that are actually being laid out.
So I've decided to go through the English alphabet and explicitly explain the connotations of each character as they might be used by a [unusually-bright-highschooler|reasonably-clever-college-student]-level...
You're right. I'll delete that aside.
We present gradient routing, a way of controlling where learning happens in neural networks. Gradient routing applies masks to limit the flow of gradients during backpropagation. By supplying different masks for different data points, the user can induce specialized subcomponents within a model. We think gradient routing has the potential to train safer AI systems, for example, by making them more transparent, or by enabling the removal or monitoring of sensitive capabilities.
In this post, we:
- Show how to implement gradient routing.
- Briefly state the main results from our paper, on...
- Controlling the latent space learned by an MNIST autoencoder so that different subspaces specialize to different digits;
- Localizing computation in language models: (a) inducing axis-aligned features and (b) demonstrating that information can be localized then removed by ablation, even when data
I think this approach can be combined with self-other overlap fine-tuning (SOO FT, see Self-Other Overlap: A Neglected Approach to AI AlignmentI'm part of the SOO team[1], now an ICLR submission). The difficult part of SOO is to precisely isolate the representation of self and other, and I think it should be possible to use ERA to get a tighter bound on it.
- ^
I'm part of the SOO team.
