This is a revised version of a paper I put out late last year (including a new table about capabilities lacking in current AI and how we might get them.) Interested in reactions and potential improvements, as I'm contemplating various possibilities for either expansion or distillation/repackaging of this basic picture,...
Overview The idea of this post is to describe, discuss, and if warranted understand how to create, a model of crypto-fed computation.[1] The basic idea is that high-powered GPU (or other ML-specialized) hardware could be equipped with in-chip hardware locks such that the computational cores require a steady stream of...
(crossposted to EA forum here.) Although I have not seen the argument made in any detail or in writing, I and the Future of Life Institute (FLI) have gathered the strong impression that parts of the effective altruism ecosystem are skeptical of the importance of the issue of autonomous weapons...
This is a revised version of a paper I put out late last year (including a new table about capabilities lacking in current AI and how we might get them.)
Interested in reactions and potential improvements, as I'm contemplating various possibilities for either expansion or distillation/repackaging of this basic picture, for a variety of audiences.
Matthew I think you're missing a pretty important consideration here, which is that all of these policy/governance actions did not "just happen" -- a huge amount of effort has been put into them, much of it by the extended AI safety community, without which I think we would be in a very different situation. So I take almost the opposite lesson from what's happening: concerted effort to actually try to govern AI might actually succeed -- but we should be doubling down on what we are doing right and learning from what we are doing wrong, not being complacent.
(comment crossposted from EA forum)
Very interesting post! But I'd like to push back. The important things about a pause, as envisaged in the FLI letter, for example, are that (a) it actually happens, and (b) the pause is not lifted until there is affirmative demonstration that the risk is lifted. The FLI pause call was not, in my view, on the basis of any particular capability or risk, but because of the out-of-control race to do larger giant scaling experiments without any reasonable safety assurances. This pause should still happen, and it should not be lifted until there is a way in place to assure that safety. Many of the things... (read more)
If anyone would like to be funded to do actual high quality research on this topic, I strongly encourage application to FLI's Humanitarian Impacts of Nuclear War grant program. For decades there have been barely any careful studies because there is barely any research funding or support. It's quite possible the effects are not as bad as currently predicted, but it's quite possible they are worse — the modern nuclear winter studies fund that things are worse than the early ones in the 80s (though fortunately the arsenals are much smaller now.)
It seems quite important to me to have a clear-eyed view of what the results of "small" and "large"... (read more)
I think this depends a lot on the use case. I envision for the most part this would be used in/on large known clusters of computation, as an independent check on computation usage and a failsafe. In that case it will be pretty easy to distinguish from other uses like gaming or cryptocurrency mining. If we're in the regime where we're worried about sneaky efforts to assemble lots of GPUs under the radar and do ML with them, then I'd expect there would be pattern analysis methods that could be used as you suggest, or the system could be set up to feed back more information than just computation usage.
The purpose of the COMPUTE token and blockchain here would be to provide a publicly verifiable ledger of the computation done by the computational cores. It would not be integral to the scheme but would be useful for separating the monitoring and control, as detailed in the post. I hope it is clear that a token as a tradeable asset is not at all important to the core idea.
Very cool, thanks for the pointer!
The idea of this post is to describe, discuss, and if warranted understand how to create, a model of crypto-fed computation.[1] The basic idea is that high-powered GPU (or other ML-specialized) hardware could be equipped with in-chip hardware locks such that the computational cores require a steady stream of cryptographic keys in order to continue performing. In the absence of such continually supplied keys the hardware would downgrade to a small or zero fraction of its nominal capability.
Deploying such hardware would allow three key things:
There's no single metric or score that is going to capture everything. Metaculus points as the central platform metric were devised to —as danohu says — reward both participation and accuracy. Both are quite important. It's easy to get a terrific Brier score by cherry-picking questions. (Pick 100 questions that you think have 1% or 99% probability. You'll get a few wrong but your mean Brier score will be ~(few)*0.01. Log score is less susceptible to this). You can also get a fair number of points for just predicting the community prediction — but you won't get that many because as a question's point value... (read more)
I am not an expert on the Outer Space Treaty either, but by also by anecdotal evidence, I have always heard it to be of considerable benefit and a remarkable achievement of diplomatic foresight during the Cold War. However, I would welcome any published criticisms of the Outer Space Treaty you wish to provide.
It's important to note that the treaty was originally ratified in 1967 (as in, ~two years before landing on the Moon, ~5 years after the Cuban Missile Crisis). If you critique a policy for its effects long after its original passage (as with reference to space mining, or as others have the effects of Section 230 of the... (read more)
You're certainly entitled to your (by conventional standards) pretty extreme anti-regulatory view of e.g. the FDA, IRBs, environmental regulations, etc., and to your prior that regulations are in general highly net negative. I don't share those views but I think we can probably agree that there are regulations (like seatbelts, those governing CFCs, asbestos, leaded gasoline, etc.) that are highly net positive, and others (e.g. criminalization of some drugs, anti-cryptography, industry protections against class action suits, etc.) that are nearly completely negative. What we can do to maximize the former and minimize the latter is a discussion worth having, and a very important one.
In the present case of autonomous... (read more)
(crossposted to EA forum here.)
Although I have not seen the argument made in any detail or in writing, I and the Future of Life Institute (FLI) have gathered the strong impression that parts of the effective altruism ecosystem are skeptical of the importance of the issue of autonomous weapons systems. This post explains why we think those interested in avoiding catastrophic and existential risk, especially risk stemming from emerging technologies, may want to have this issue higher on their list of concerns.
We will first define some terminology and do some disambiguation, as there are many classes of autonomous weapons that are often conflated; all classes have some issues of concern, but some... (read 5619 more words →)
Great article Garrison!
Bravo.