Zach Stein-Perlman's Shortform

Zach Stein-Perlman

Decreasing or increasing returns in the size of the field?

Suppose a random 1% (or 50%) of the AI safety ecosystem quits tomorrow (not including you). This is bad news, but does it make your personal impact higher or lower?^[1]

Higher: there's diminishing returns in the size of the field. With a smaller field, you pluck lower-hanging fruit. If there are N people, all equally skilled, each person produces less than 1/N counterfactual value, since N-1 people would still pluck the low-hanging fruit. (If there's an upper bound on the value of the field, no matter how large the field is—e.g., solving alignment and making the transition to superintelligence go well—then clearly at some point the field must become sublinear.) Also maybe large fields have issues.
Lower: with a larger field, your work improves more people's work and more other people improve your work. (You discovering that a particular line of work is promising or not is more effective; more other people discover that a particular line of work is promising or not and thus help you; there's more infrastructure and each piece of infrastructure is more valuable; probably more.) Maybe there's increasing returns to effort within projects. Maybe there's synergies between projects or many projects act as multipliers on other projects — maybe there's positive feedback loops between the field doing lots of good research and new people wanting to join the field, maybe government-policy work depends on technical-safety work, etc.

More potential considerations: Claude (mediocre).

I think it's generally assumed that there's diminishing returns because "higher" is a bigger effect. I want a quantitative estimate in the case of AI safety — this is a parameter for quantifying the value of growing the field, as part of my prioritization work.

There's probably a bunch of economics work on what affects R&D/innovation; see Claude.

(A separate phenomenon is that if you grow the field, on average the new people will be lower-quality than the old people.)

^{^}
Or: is it less bad or more bad than magically decreasing the impact of the field by 1%/50%?
And you can consider the opposite question: increase the field by 1%/50%/100%, maintaining the current distribution of people-quality.

Zach Stein-Perlman6mo150

Showing 3 of 5 replies (Click to show all)

2Garrett Baker6mo

Somewhat of a nitpick, but the correct question to ask is about the marginal 1% of the AI safety ecosystem, no? This is a very different distribution than a random subset.

Zach Stein-Perlman6mo42

Here I'm exploring separating two parameters: effect of growing the field holding quality-distribution constant and marginal quality vs average quality. (And there's also value of the field and cost to grow the field by 1%.)

2Zach Stein-Perlman6mo

Yeah, separating "quality-adjusted field size --> output" and "output --> impact" might be helpful. But I think "parallelizing" is the wrong frame — it doesn't account for how larger fields have more shared infrastructure, more synergy, etc. And you have to be careful to account for the "low-hanging fruit plucked first" or "infinite work produces bounded value" phenomenon exactly once.