Venki's Shortform
Feb 12, 20251
Venki
37
1
5
Related thread from Emmett Shear that I've appreciated on the topic: https://x.com/eshear/status/1660356496604139520, esp the framing:
> “power corrupts” is actually shorthand for “power corrupts your world model”
Replying toThe Inkhaven Residency
Tangent: I was curious about your estimates of the top 1% and top 0.1% figures - so I looked into it. Somewhat also thinking about Erik Hoel's essay about successful authors being about as rare as billionaires. [link]
- Some estimate ~45 Substacks at >$1M/y [link] - that roughly fits with Substack's reported total subscription revenue at ~$450M/y. (Officially reported at 30 in 2024 [link])
- Reasonable to estimate 100-1000 Substacks at >$100k/y
- How many "bloggers" exist?
- Substack reports 50,000 Substacks w/ at least one paid sub [link]
- Very closely supports your 0.1% and 1% estimates TBH!
- There are about ~500M wordpress blogs on the Internet (???)
- I'm not sure what the right vibe is here: I could buy anything
- Substack reports 50,000 Substacks w/ at least one paid sub [link]
What are all the high-level answers to "What should you, a layperson, do about AI x-risk?". Happy to receive a link to an existing list.
Mine from 5m of recalling answers I've heard:
- Don’t work for OpenAI
- Found or work for an AI lab that gains a lead on capabilities, while remaining relatively safe
- Maybe work for Anthropic, they seem least bad
- Don’t work for any AI lab
- Don’t take any action which increases revenue of any AI lab
- Mourn
- Do technical AI alignment
- Don’t do technical AI alignment
- Do AI governance & advocacy
- Donate to AI x-risk funds
- Cope
- Don't perform domestic terrorism
This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.
Labor magnification as a measure of AI systems.
Cursor is Mag(Google SWE, 1.03) if Google would rather have access to Cursor than 3% more SWEs at median talent level.
A Mag(OpenAI, 10) system is a system that OpenAI would rather have than 10x more employees at median talent level
A time based alternative is useful too, in cases where it's a little hard to envision as many more employees.
A tMag(OpenAI, 100) system is a system that OpenAI would rather have than 100x time-acceleration for its current employee pool.
Given that definition, some notes:
1. Mag(OpenAI, 100) is my preferred watermark for self-improving AI, one where we'd expect takeoff unless there's a sudden & exceedingly hard scaling wall.
2.... (read more)
Related: Rule Thinkers In, Not Out