Quick Takes

A lot of people seem to share the belief that as individual investors, investing in publicly traded big tech AI companies (e.g. Google, Microsoft, Meta) won't contribute meaningfully to AI risk. I've already outlined some of my thoughts on this in a recent post on ethical investing in light of AI risk, but here I'd like to further challenge a particular argument that I've seen made in favour of investing in big tech still being an ethical option. The argument is that these companies would be able to spend as much as they would like to on the AI build out even without borrowing, given they have such large cash war chests. So any support to the share price by your purchase would have no impact on total spending given no borrowing was required. I've seen this view often put forward in some of the top voted AI investment related post and comments. An example is given below from this post on how to ethically make money off AGI. "But Microsoft and Google are not like that. They already have infinite war chests and are not going to be spending more or less money on AI based on your investment. Nor is Nvidia going to do anything but maximize along its capacity and scaling constraints, money isn't a factor. I think buying such stocks is fine." I think the recent announcements around big tech bond issuance to pay for the AI build out should cause people to update their views here. It's projected that big tech firms will raise hundreds of billions of dollars this year. Even those with huge cash positions such as Google are raising money. Clearly even big tech cannot build enough data centres without borrowing. The link between stock purchase, higher share price, greater borrowing and greater spend on AI build out seems more concrete this year than it did previously.

I hear Fable 5 is a much better writer than Opus 4.8. I generated 18 fiction samples with each, and could not reliably tell the difference. Curious if others here can do better: quiz

As I write this, there are around 3 hours left before polls close for this years's New York's 12 District Democratic Primary. If you're a registered democrat in NY-12, you can still vote.[1] But for those of us who reside elsewhere, there's little to be done but to wait with bated breath. Will Alex Bores, author of the RAISE act, manage to overcome the millions of dollar spent against him by Leading the Future and demonstrate that AI regulation is not just politically viable but a winning issue? Or will the establishment favorite (and favorite from the start of the race) Micah Lasher succeed in succeeding his mentor Nadler? I don't know. As of writing, the prediction markets (Kalshi, PredictIt) have Bores winning at around 28% and Lasher at 72%. If you think you do know the answer, you should go make some money on these markets! One thing I'm worried about is that people will learn too much from the binary outcome of Bores or Lasher and not on the details of the race. I'm writing this in haste to get it out before the polls close, and we start seeing the outcome, so as to preregister my thoughts. If Bores loses, some might claim that AI regulation remains politically toxic, and that LTF's spending was decisive. (I imagine LTF certainly will.) But this is a mistake: win or lose, Bores's demonstrated that passing AI regulation will not just leave you facing down millions of dollars of Super PAC spending alone. Instead, millions of dollars of Super PAC money was spent on ads championing Bores (in fact, more than what LTF spent!), as well as hundreds of thousands of dollars of donations from AI Safety-concerned individuals. I know many people who've donated to Bores's campaign, and who are invested in his victory. If he were to lose -- especially by a large margin -- it might seem tempting to dismiss the whole enterprise of political donations entirely. Similarly, if (somehow) he were to win by a large margin, it might feel like the marginal donation was useless. B

Recently an interviewer asked me how I got to be such a good forecaster, and I replied by saying something humble. In retrospect it was a bad answer because I should have instead used the opportunity to give actual advice on how to forecast AI well. Here's a stream-of-consciousness attempt to do that: 1. The heuristic that things which sound weird and sci-fi are less likely to happen in reality, is bad. I suspect that's what really going on is that things which sound weird and sci-fi put you at risk of being judged a weirdo if you talk about them which is not the same thing as are unlikely to happen. Repeat to yourself the mantra that some weird sci-fi things really do happen, and others don't, and you have to take them on a case by case basis. 2. Trend extrapolation is your friend. Your best friend. Don't let anyone tell you otherwise. I've actually only rarely see someone extrapolate a trend too credulously; more often, people have a trend staring them in the face and extrapolate it a tiny bit into the future and then are too timid to keep extrapolating it. In general I think it's reasonable to extrapolate a trend as far into the future as it extends into the past. Now obviously, trend extrapolation is just the beginning of the forecasting process, it's not the end. For each trend you should ask yourself whether it makes sense for it to continue like that and if not why not etc. You'll usually end up with some sort of sophisticated view about how the trend will probably continue but bend downwards a bit and then ultimately plateau around X level. 3. Explicit models are also your friend. Things like Bio Anchors, the AI Futures Model, takeoffspeeds.com, etc. The process of making your own complicated model like this, and engaging with the models made by others, is... well, I think it's pretty edifying. I'm not sure why but I could speculate. Maybe something about teaching you to be appropriately humble (e.g. when the model output is sensitive to a parameter you

The first podcast episode I've ever participated in has been released, if anyone wants to get an update on what I've been thinking about recently, in audio form. (A transcript is also available.) Thanks to Fin Moorhouse @fin for the conversation and handling the logistics/production. BTW, related to the theme of humans being bad at philosophy and also just not caring very much about it, I recently finished The Good Place (spoiler alert), and it bothered me a lot that it ends with Chidi, the philosopher character, deciding to end his (after)life instead of using the infinite time and resources available to solve philosophy, or to participate in such a project. Worse, I can't find any existing objection to this, including from actual philosophers, who mostly seem happy just to have any representation at all in pop culture. Meanwhile in AI safety, our best plan is still to defer to humans on questions like "What should ASI do with our lightcone?"

ChangeDiaperBench, PlanInvasionBench, ButcherHogBench, ShipConnBench, BuildingDesignBench, SonnetBench, AccountBalanceBench, WallBuildBench, BoneSetBench, ComfortDyingBench, OrderTakeBench, OrderGiveBench, CooperateBench, ActAloneBench, SolveEquationsBench, AnalyzeProblemBench, ManurePitchBench, ComputerProgramBench, TastyCookingBench, EfficientFightingBench, GallantDyingBench

it's kind of crazy thinking about the fact that i will basically never write code ever again in my life. the models are actually good at code now (tbf, i haven't tried the models on anything super hard yet, so don't update too much on my take here.)