Supported by Rethink Priorities
This is part of a weekly series summarizing the top (40+ karma) posts on the EA and LW forums - you can see the full collection here. The first post includes some details on purpose and methodology.
If you'd like to receive these summaries via email, you can subscribe here.
Podcast version: prefer your summaries in podcast form? A big thanks to Coleman Snell for producing these! Subscribe on your favorite podcast app by searching for 'Effective Altruism Forum Podcast'.
Author's note: I'm currently travelling, so there won't be any summary post next week. It'll be back the week after with the summaries for those 2 weeks of content at a higher karma bar.
FTX
FTX filed for bankruptcy on 11th Nov. The FTX Future Fund, financed by FTX and its founder SBF (Sam Bankman-Fried), had been a major funder in EA since its launch in Feb this year.
What happened: Binance, a competitor, sold their stake in FTX’s primary coin - this caused it to drop in value, and led to the equivalent of a bank run ie. a large number of clients attempting to withdraw funds. FTX didn’t have enough money to pay them, and there have been claims this is due to SBF misusing customer funds to prop up his trading company Alameda Research.
This has meant thousands of people with funds in FTX have likely lost them. If you need someone to talk to, even just to vent, CEA’s community health team is available or you can access peer support set up by Rethink Wellbeing and the Mental Health Navigator here.
FTX FAQ by Hamish Doodles provides a good overview of the situation as of Sunday 13th November.
Other key points from posts:
This is an evolving situation, with many posts on the topic. Those with 40+ karma from the past week are below, with the most recent at the top:
Proposals for reform should come with detailed stories by Eric Neyman
Effective Peer Support Network in FTX crisis (Update) by Emily, Inga
AI Safety Microgrant Round by Chris Leong, Damola Morenikeji, David_Kristoffersson
NY Times on the FTX implosion's impact on EA by AllAmericanBreakfast
Wrong lessons from the FTX catastrophe by burner
How the FTX crash damaged the Altruistic Agency by Markus Amalthea Magnuson
The FTX Situation: Wait for more information before proposing solutions by D0TheMath
Announcing Nonlinear Emergency Funding by Kat Woods, Emerson Spartz, Drew Spartz
SBF, extreme risk-taking, expected value, and effective altruism by vipulnaik
Thoughts on legal concerns surrounding the FTX situation by Molly
Hubris and coldness within EA (my experience) by James Gough
Will MacAskill's role in connecting SBF to Elon Musk for a potential Twitter deal by dyj34650
Noting an unsubstantiated belief about the FTX disaster by Yitz
In favour of compassion, and against bandwagons of outrage by Emrik
A personal statement on FTX by William_MacAskill
After recent FTX events, what are alternative sources of funding for longtermist projects? by CarolineJ
CEA/EV + OP + RP should engage an independent investigator to determine whether key figures in EA knew about the (likely) fraud at FTX by Tyrone-Jay Barugh
How could we have avoided this? by Nathan Young
IMPCO, don't injure yourself by returning FTXFF money for services you already provided by EliezerYudkowsky
Thoughts on FTX and returning to our ideals by michel
Under what conditions should FTX grantees voluntarily return their grants? by sawyer
My reaction to FTX: appalled by Robert_Wiblin
Resorting constantly to bets can be bad for optics, especially in the current situation by EdMathieu
For the mental health of those affected by the FTX crisis… by Daystar Eld
We must be very clear: fraud in the service of effective altruism is unacceptable by evhub
Some comments on recent FTX-related events by Holden Karnofsky
Community support given FTX situation by Julia_Wise
The FTX Future Fund team has resigned by Nick_Beckstead, leopold, ab, ketanrama
Money Stuff: FTX Had a Death Spiral by Elliot Temple
FTX will probably be sold at a steep discount. What we know and some forecasts on what will happen next by Nathan Young, NunoSempere, Stan van Wingerden, Juan Gil
FTX.com has probably collapsed by Charles He
by Bob Fischer
Second post in the moral weight project sequence. Assesses the likelihood of trait possession for each of 11 different species on >90 empirical proxies that might provide evidence of variation in valenced states. (Ie. The proxies provide evidence on if the species best and worst potential experiences would be very different in ‘goodness’). These included both hedonic and cognitive proxies. Some examples: ‘joy like behavior’, ‘concept of death’, ‘reward based learning’, ‘cooperative behavior’, ‘individual differences / personality’. This is presented visually in a (super cool) graph, and available in excel also.
Key meta-results included that there are a lot of unknowns, a decent amount of cases with positive evidence for a trait, and very few cases where we have evidence a species doesn’t have the trait. The most information was known about terrestrial vertebrates (eg. pigs) and the least about invertebrates (eg. silkworms). In terms of traits, a lot was known about some (eg. cooperative behavior, communication, parental care) and very little about others - particularly affective states like guilt-like behavior, sympathy-like behavior, or shared intentionality.
Naïve vs Prudent Utilitarianism
by Richard Y Chappell
Naive utilitarianism is acting whenever the most salient first-order consequences are positive. These calculations are unreliable, and violating people’s rights is almost guaranteed to be of negative expected value, even when the first order effects look positive. This is well known by utilitarian theorists, and so prudent / rational utilitarians tend to abide by cooperative norms - making them more trustworthy than critics equating their moral philosophy to naive utilitarianism would think.
Opportunities that surprised us during our Clearer Thinking Regrants program
by spencerg, Clare_Diane
Clearer Thinking evaluated >630 project proposals as part of their Regrants program. They share key learnings that updated them toward there being many promising opportunities where 10K-500K gifts could make a big difference.
Unexpected room for funding of well-known orgs
Object-level learnings
Instead of technical research, more people should focus on buying time and Ways to buy time
by AW
Delaying AGI timelines by 1 year gives the entire alignment community an extra year to solve the problem. ‘Buying time’ interventions typically involve convincing AI labs that AI x-risk is an important concern, and providing feasible actions for them to reduce it. Eg:
The author thinks ~40-60% of alignment researchers should work on this instead of technical research (particularly if they would be a better fit for it), and 20-40% of AI Safety community builders should also switch to this focus. This is particularly the case if you are able to progress an intervention that buys time at the end, i.e. when we know more and have more tools to help with alignment.
[Links post] Economists Chris Blattman and Noah Smith on China, Taiwan, and the likelihood of war
by Stephen Clare
Summaries of two recent posts by prominent economists on the likelihood of a China / US war over Taiwan, using economic frameworks. Both think it’s relatively likely. Details below:
The prospects for war with China: Why I see a serious chance of World War III in the next decade by Chris Blattman
Applying the bargaining framework he developed in his book ‘Why We Fight’, Chris argues that as China grows its economy and military, war becomes more likely. While negotiated settlement is theoretically preferable, there may be principles (e.g. democracy vs autocracy) that are non-negotiable, and China harmed its reputation for sticking to settlements with its crackdown on Hong Kong.
Why I think an invasion of Taiwan probably means WW3 by Noah Smith
Uses game theory to predict the most likely war scenarios, via:
Given his assumptions, the equilibrium solution is for China to invade and attack the US to maximize its chances of victory, nearly assuring the outbreak of a major great power war. However the assumptions might not be great, and he discusses how they don’t take into account eg. misinformation, or the ‘US resists’ outcomes being too negative for both countries for either to take that path.
Tracking the money flows in forecasting
by NunoSempere
A list of 27 forecasting organisations (within and outside EA), including description and rough estimates of monetary value and social value.
Does the US public support radical action against factory farming in the name of animal welfare?
by Neil_Dullaghan
A US nationally representative survey by Rethink Priorities found 15.7% of respondents (N=2,698) supported a ban on slaughterhouses when presented with arguments for and against, and asked to explain their reasoning. Previous surveys by the Sentience Institute in the US found ~39-43% of people supported banning slaughterhouses, highlighting a large discrepancy.
Rethink Priorities suggests that previous polls which determined attitudes in response to broad questions (e.g. “I support a ban on slaughterhouses”) may not be accurate indicators of support for certain policies. These findings are notable for animal advocates as previous findings had been cited as support for bold reforms.
Neil also notes that for future research, it might be useful to test a radical ask (ban factory farming) and a moderate ask (labelling for cage-free eggs, say) each with a radical message ("meat is murder") versus a moderate one ("human/consumer welfare") somewhat similar to this paper.
How to change a system from the inside
by weeatquince
A guide on how to change your organization’s rules, protocols, decision-making, culture or ethos from the inside. Originally written by UK civil servants, the author is most confident it applies in that context, but has adapted the post to give advice more broadly to staff from juniors to middle managers in large organisations. The process suggested is:
Change is slow, and can fail or be limited by external factors - but giving it a go can be good for learning and career capital even if it fails.
Apply now for the EU Tech Policy Fellowship 2023 by Jan-WillemvanPutten, Cillian Crosson, Training for Good, SteveThompson
8-month programme to catapult grads into high-impact career paths in EU policy, mainly working on the topic of AI Governance. Includes remote PT study July/Aug, 2x week-long policy trainings in Brussels, and choice of ~5 month placement at a host org or support applying to other EU policy jobs. Open to EU citizens, Apply by Dec 11.
GiveWell is hiring a Research Analyst (apply by November 20)
by GiveWell
GiveWell is hiring for a Research Analyst for their core interventions team, which investigates and makes funding decisions about programs they’re already supporting at scale. All locations welcome, as long as willing to join meetings in the California time zone. No specific experience or degrees needed. Apply by Nov 20th.
by Bradley Tjandra, Nathan Sherburn
A lot of EA aligned work being done in Australia involves working remotely with the international community, but there’s also a growing list of projects at least partially led by Australian residents. This post compiles them together and provides summaries, who’s working on it, links, and requests / calls for action for each.
Includes: AI Safety Australia & New Zealand, AI Safety Support, EA Pathfinder, Foundations for Tomorrow, Giving What We Can, Good Ancestors Project, High Impact Engineers, High Impact Recruitment, Insights for Impact, Lead Exposure Elimination Project, Quantifying Uncertainty in Givewell CEAs, Ready Research, and Sentience Institute.
Google Scholar is now listing (some) EA Forum posts
by PeterSlattery
When typing ‘effective altruism’ into Google Scholar, some EA forum posts show up - although without proper titles. DAOMaxi explains in the comments why this would happen, and how it’s possible to either manually add articles to Google Scholar or for the forum team devs to include tags that will do it automatically (and with correct details). Currently 48 forum posts are indexed on Google Scholar.
Some advice on independent research
by mariushobbhahn
You might want to do independent research as a side / transition project, if there aren’t positions open in the area you want to research, or because you value independence / flexibility.
The author has several tips for doing this successfully:
Doing Ops in EA FAQ: before you join (2022)
by Vaidehi Agarwalla, Alexandra Malikova, ES
Guide by Pineapple Operations on things to consider before entering Ops work at an EA org. Includes:
What is Ops, and how to get into it?
What Ops is like
They also list further resources and orgs specializing in operations.
The 8-week mental health programme for EAs finally published
by tereziekosik, Kristyna Stastna, Sylvie Wagnerová
Mental Health Navigator has published an 8-week mental health programme for EAs. This includes an 8-chapter workbook and resources for facilitators to run weekly workshops.
AI Safety groups should imitate career development clubs
by Joshc
If you want talented ML students to learn about AI safety, offer them what they find valuable ie. projects and skill-building that create career capital. This is the model of ML @ Berkeley, which is run unpaid, requires 15 hour p/w commitment from participants, is extremely selective (~7% get in) and still has 50 students. Many groups are more discussion-focused, and could benefit from this approach.
by Bob Jacobs
The author designed symbols for a utilitarianism flag, their local EA group, and common EA mindsets which they share here. They’ve also created a lot of banners, thumbnails, and images used elsewhere on the forum.
How could we know that an AGI system will have good consequences?
by So8res
Alignment proposals are answers to the scenario: ‘Imagine you’re about to launch AGI, and think there is >50% chance it will end the acute risk period in a good way. Why do you think that?’
Current proposals fall into three buckets, which the author believes are doomed for the reasons indented below each:
Applying superintelligence without collusion
by Eric Drexler
A lot of AI safety research assumes a monolithic AGI, with the argument that if there were multiple superintelligent-level systems they would inevitably collude and act as one.
Factors that make collusion less likely include a large number of actors, sensitivity to defectors, diversity among actors, constrained communication, single-move decision-processes, and lack of shared knowledge. The author argues these conditions are supported by current architectures and incentives (eg. making multiple diverse models improves quality / reliability of answers). Applying multiple potentially untrustworthy superintelligent-level systems to problems can improve rather than degrade safety by thwarting collusion. They call for greater attention on this prospect.
I Converted Book I of The Sequences Into A Zoomer-Readable Format
by dkirmani
The author has converted book 1 of the sequences into machine-read audio overlaid on unrelated Subway Surfers gameplay footage. Similar videos are often recommended on TikTok, so this format may be highly engaging for some people. The videos are all linked in the post.
"Rudeness", a useful coordination mechanic
by Raemon
Rudeness is a way of spending down social capital, which you accumulate doing high status respectable things. Different communities and cultures have different norms of ‘rude’ eg. belching at a meal might be rude in one culture, and rude not to do in another. This allows groups to fine-tune what they optimize for - making some actions more socially expensive and so occur less often.
Trying to Make a Treacherous Mesa-Optimizer
by MadHatter
The author builds a toy model to try and empirically prove the possibility of treacherous mesa-optimizers ie. optimizers that try to look aligned in training.
They created a model to follow the X = Y line in a graph up to Y = 5, to know humans can’t control it after that, to learn via simulation, and to have a loss model that is different from the ‘true’ loss model we want. They show in this case the model sometimes veers away from X = Y after the Y = 5 point. Code, graphs, and commentary are provided.
Instrumental convergence is what makes general intelligence possible
by tailcalled
Author’s tl;dr (lightly edited): General intelligence is possible because solving real-world problems requires solving common subtasks. Common subtasks are what give us instrumental convergence (definition: the tendency for most sufficiently intelligent beings to pursue similar sub-goals). Common subtasks are also what make AI useful; you want AIs to pursue instrumentally convergent goals. Capabilities research proceeds by figuring out algorithms for instrumentally convergent cognition. Consequentialism and search are fairly general ways of solving common subtasks.
A philosopher's critique of RLHF
by ThomasW
Shares a transcription of a short Q&A between Brian Christian (Author of The Alignment Problem) and Yale Philosophy Professor L.A. Paul. The latter suggests that the issue with RLHF (reinforcement learning from human feedback) is new scenarios where humans can’t distinguish what is better, or even bucket it into an existing category. The phrasing is crisp and tackles the problem clearly, despite Professor Paul having limited background in AI Safety. The author recommends watching the full recording.
by Metacelsus
Genetics is the study of genes ie. sequences of genetic material that encode functional products. Epigenetics is the study of modifications to this genetic material that don’t affect the sequence, but control which genes get expressed. In mammals, these are DNA methylation and histone modifications. Newly copied DNA lacks these modifications and the modifications can be read, written, and erased by specialized proteins (scientists can also do this via CRISPR).
Speculation on Current Opportunities for Unusually High Impact in Global Health
by johnswentworth
The Sahel region is close to the Malthusian equilibrium (where all production is used for sustenance). This means many die in an economic downturn. Support is likely neglected due to government corruption, necessitating charities to deliver support in person, with one source stating antibiotic imports of the entirety of Mali amounted to $53k in 2020. Givewell top charities also tend to physically distribute goods. Based on this, the author suggests even flying in with a backpack of antibiotics to give away might be highly impactful.
by Mati_Roy
Suggests exam-only universities as a way to allow people to learn at their own pace / from wherever they want, and for testing to become more standardized across institutions. It also removes bias from lecturers offering exam hints during classes, and the inconvenience and cost of attending classes.
What it's like to dissect a cadaver by Alok Singh