I think that human level capabilities in natural language processing (something like GPT-2 but much more powerful) is likely to occur in some software system within 20 years.

Since human level capabilities in natural language processing is a very rich real-world task, I would consider a system with those capabilities to be adequately described as a general intelligence, though it would likely not be very dangerous due to its lack of world-optimization capabilities.

This belief of mine is based on a few heuristics. Below I have collected a few claims which I consider to be relatively conservative, and which collectively combine to weakly imply my thesis. Since this is a short-form post I will not provide very specific lines of evidence. Still, I think that each of my claims could be substantially expanded upon and/or steelmanned by adding detail from historical trends and evidence from current ML research.

Claim 1: Current techniques, given enough compute, are sufficient to perform par-human at natural language processing tasks. This is in some sense trivially true since sufficiently complicated RNNs are Turing complete. In a more practical sense, I think that there is enough evidence that current techniques are sufficient to perform rudimentary

• Summarization of text
• Auto-completion of paragraphs
• Q&A
• Natural conversation

Given more compute and more data, I don't see why there would be a fundamental stumbling block for current ML models to scale to human level on the above tasks. Therefore, I think that human level natural language processes could be created today with enough funding.

Claim 2: Given historical data and assumptions about future progress, it is quite likely that the cost for training ML systems will continue to go down in the next decades by significant amounts (more specifically: an order of magnitude). I don't have much more to add to this other than the fact that I have personally followed hardware trends on websites like videocardbenchmark.net and my guess is that creating neural-network specific hardware will continue this trend in ML.

Claim 3: Creating a system with human level capabilities in natural language processing will require a modest amount of funding, relative to the amount of money large corporations and governments have at their disposal. To be more specific, I estimate that it would cost less than five billion dollars in hardware costs in 2019 inflation adjusted dollars, and perhaps even less than one billion dollars. Here's a rough sketch for an argument for this proposition:

• The cost of replicating GPT-2 was $50k. This is likely to be a large overestimate, given that the post noted that intrinsic costs are much lower.
• Given claim 2, this cost can be predicted to go down to about $5k within 20 years.
• While the cost for ML systems does not scale linearly in the number of parameters, the parallelizability of architectures like the Transformer allow for near-linear scaling. This is my impression from reading posts like this one.
• Given the above three statements, the cost of running a Transformer with the same number of parameters as the high estimate for the number of synapses in a human brain would naively cost about one billion dollars.

Claim 4: There is sufficient economic incentive such that producing a human-level system in the domain of natural language is worth a multi-billion dollar investment. To me this seems quite plausible, given just how many jobs require writing papers, memos, or summarizing text. Compare this to a space-race type scenario where there becomes enough public hype surrounding AI such that governments are throwing around one hundred fifty billion dollars, which is what they did for the ISS. And relative to space, AI at least has very direct real world benefits!

I understand there's a lot to justify these claims. And I haven't done much work to justify them. But, I'm not presently interested in justifying these claims to a bunch of judges intent on finding flaws. My main concern is that they all seem likely to me, and there's also a lot of current work in out-competing companies to be first on the natural language benchmarks. It just adds up to me.

Am I missing something? If not, then this argument at least pushes back on claims that there is a negligible chance of general intelligence emerging within the next few decades.