bgold's Shortform

Ben Goldhaber

This is a special post for quick takes by Ben Goldhaber. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

AI for improving human reasoning seems promising; I'm uncertain whether it makes sense to invest in new custom applications, as maybe improvements in models are going to do a lot of the work.

I'm more bullish on investing in exploration of promising workflows and design patterns. As an example, a series of youtube videos and writeups on using O3 as a forecasting aid for grantmaking, with demonstrations. Or a set of examples of using LLMs to aid in productive meetings, with a breakdown of the tech used and social norms that the participants agreed to.
- I think these are much cheaper to do in terms for time and money.
- A lot of epistemics seems to be HCI bottlenecked.
- Good design patterns are easily copyable, which also means they're probably underinvested in relative to their returns.
- Social diffusion of good epistemic practices will not necessarily hapepn as fast as AI improvements.
- Improving the AIs themselves to be more truth seeking and provide good advice - with good benchmarks - is another avenue.

I imagine a fellowship for prompt engineers and designers, prize competitions, or perhaps retroactive funding for people who have already developed good patterns.

2ozziegooen2y

Happy to see thinking on this. I like the idea of getting a lot of small examples of clever uses of LLM in the wild, especially by particularly clever/experimental people. I recently made this post to try to gather some of the techniques common around this community. One issue that I have though is that I'm really unsure what it looks like to promote neat ideas like these, outside of writing long papers or making semi-viral or at least [loved by a narrow community] projects. The most obvious way is via X/Twitter. But this often requires building an X audience, which few people are good at. Occasionally particularly neat images/clips by new authors go viral, but it's tough. I'd also flag: - It's getting cheaper to make web applications. - I think EA has seen more success in making blog posts and web apps than we did things like [presenting neat ideas in videos/tweets]. - Often, [simple custom applications] are pretty crucial for actually testing out an idea. You can generate wireframes, but this only tells you a very small amount. I guess what I'm getting at is that I think [web applications] are likely a major part of the solution - but that we should favor experimenting with many small ones, rather than going all-in on 2-4 ideas or so.

Good points! I agree that actual prototyping is necessary to see if an idea works, and as a demo it can be far more convincing. Especially w/ the decreased cost of building web apps, leveraging them for fast demos of techniques seems valuable.