I think something like what you're sketching here, viz "harnessing technology to make people and civilization saner", is probably highly valuable and possibly quite neglected. [1] Thank you for working on this.
A class of infrastructure/technologies that seem very important, but which I didn't see mentioned in this post: infrastructure for creating common knowledge of better equilibria and coordinating transitions to them. [2] Do you (have plans to) address anything matching that (vague) description anywhere?
Low-hanging dignity points! ↩︎
I.e., something that would solve problems of form "there exists a much better equilibrium, but getting there would require lots of people to have common knowledge of that better equilibrium, and also coordinate and sufficiently-credibly commit to near-simultaneously taking action that would be detrimental to them if they took it alone". Some examples: move from frequentist stats to Bayesian stats; make it easier for AI labs to (conditionally) stop racing; US voters coordinate to vote for a less sociopathic party-independent candidate (or to replace first-past-the-post with a saner voting system entirely); abolish all JavaScript forever, refactor the Internet to use a non-insane language; kill Elsevier; almost everyone simultaneously leaves (at least the more toxic platforms of) social media (and move to a less toxic new platform); journals/researchers commit to preregistering studies and publishing negative results; etc. ↩︎
Yeah, I like that. For me, it comes under coordination tech, which admittedly is a particularly thorny part of the map, and difficult. Ability to network and form coalitions, deliberate on models and prospects, and implement or commit to changes come under coordination, for me. Often there'd be an aspect of institution-installment or -adaptation, which deserves special care (forthcoming 'charter tech' discussion, name tbc). You'd also naturally want (individual foresight and collective) epistemics to be improved as an important support to that.
We don’t think that humanity knows what it’s doing when it comes to AI progress. More and more people are working on developing better systems and trying to understand what their impacts will be — but our foresight is just very limited, and things are getting faster and faster.
Imagine a world where this continues to be the state of play. We fumble our way to creating the most important technology humanity will ever create, an epoch defining technology, and we’re basically making it up as we go along. Of course, this might pan out OK (after all, in the past we’ve often muddled through), but facing the challenges advanced AI will bring without a good idea of what’s actually going on makes us feel… nervous. It’s a bit like arriving in your space ship on a totally unknown planet, and deciding you’ll just release the airlock and step outside to see what it’s like. There might be air on this planet — or you might asphyxiate or immediately catch a lethal alien pathogen.
If you survive, it’s because you were lucky, not because you were wise.
Now imagine a more sensible world. In this world, we get our act together. We realise how clueless we are, and that this is a huge risk factor — so we set about creating and harnessing tools, using AI as a building block, to help us to reason and coordinate through the transition.
Seamless tech helps us to track where claims come from and how reliable they are. Highly customised tools help individuals to understand themselves better and make decisions they endorse. We vastly uplift forecasting and scenario planning, massively improving our strategic awareness, and democratizing access to this kind of information. Coordination tools help large groups to get on the same page. Privacy-preserving assurance tech helps people trust agreements that are currently unenforceable, and negotiation assistance helps us find win-win deals and avoid the outcomes nobody wants.
There’s certainly still a lot of room for things to go wrong in this more sensible world. There might still be very deep value disagreements, and progress could still be very fast. But we’d be in a much better position to notice and address these issues. To a much greater extent, we’d be navigating the transition to even more radical futures with our eyes open, able to make informed choices that actually serve our interests. The challenges might still prove insurmountable! — but because that’s the nature of reality, not because of unforced errors or lack of awareness.
We think that a more sensible world should be achievable, soon — and that more should be done to help us get there.
This series of design sketches tries to envision more concretely how near-term AI systems could transform our ability to reason and coordinate. We’re hoping that these sketches will:
Of course these are just early sketches! We expect the actual technologies that make most sense could in some cases look quite different. But we hope that by being concrete we can help to kickstart more of the visioning process.
Below is an overview of all the technologies we cover in the series, grouped into some loose clusters.[1]
Collective epistemics
Tools for collective epistemics make it easy to know what’s trustworthy and reward honesty.
The technologies we discuss are:
Angels-on-the-shoulder
‘Angels-on-the-shoulder’ are customised tools that help people make better decisions in real time, narrowing the gap between how well people could make decisions, and how well they actually do.
The technologies we discuss are:
Tools for strategic awareness
Tools for strategic awareness deepen people’s understanding of what’s actually going on around them, making it easier for them to make good decisions in their own interests.
The technologies we discuss are:
Coordination tech
Coordination tech makes it faster and cheaper for groups to stay synced, resolve disagreements, identify coalitions, or negotiate to find win-win deals.
The technologies we discuss are:
Assurance and privacy
Assurance and privacy tech allow people to verifiably share information with trusted intermediaries without disclosing it more broadly, or otherwise have greater trust in external processes. This can unlock deals and levels of transparency which are currently out of reach.
The technologies we discuss are:
This article was created (in part) by Forethought. Read the original on our website.
A few caveats: