Bjartur Tómas


Shouldn't there be a Chinese translation of Human Compatible?

After looking into the PISA scores and finding they implied about 20x more 3-sigma people are in China than America, I emailed Stuart Russel about translations. This was his reply:


The vast majority of Chinese CS researchers are publishing in English.

For the broader policymaking class, it might be useful to have literature in Chinese.

My book Human Compatible will appear in Chinese shortly.

Slaughterbots has already appeared with Chinese subtitles.

For English-language government and think-tank documents,

I assume Chinese policy makers have access to translation resources as needed.


Are we in an AI overhang?

One thing we have to account for is advances architecture even in a world where Moore's law is dead, to what extent memory bandwidth is a constraint on model size, etc. You could rephrase this as how much of an "architecture overhang" exists. One frame to view this through is in era the of Moore's law we sort of banked a lot of parallel architectural advances as we lacked a good use case for such things. We now have such a use case. So the question is how much performance is sitting in the bank, waiting to be pulled out in the next 5 years.

I don't know how seriously to take the AI ASIC people, but they are claiming very large increases in capability, on the order of 100-1000x in the next 10 years, if this is a true this is a multiplier on top of increased investment. See this response from a panel including big-wigs at NVIDIA, Google, and Cerebras about projected capabilities: On top of this, one has to account, too, for algorithmic advancement:

Another thing to note is though by parameter count, the largest modern models are 10000x smaller than the human brain, if one buys that parameter >= synapse idea (which most don't but is not entirely off the table), the temporal resolution is far higher. So once we get human-sized models, they may be trained almost comically faster than human minds are. So on top an architecture overhang we may have this "temporal resolution overhang", too, where once models are as powerful as the human brain they will almost certainly be trained much faster. And on top of this there is an "inference overhang" where because inference is much, much cheaper than training, once you are done training an economically useful model, you will almost tautologically have a lot of compute to exploit it with.

Hopefully I am just being paranoid (I am definitely more of a squib than a wizard in these domains), but I am seeing overhangs everywhere!

Open & Welcome Thread - June 2020
What would be a good exit plan? If you've thought about this, can you share your plan and/or discuss (privately) my specific situation?'

+1 for this. Would love to talk to other people seriously considering exit. Maybe we could start a Telegram or something.

Open & Welcome Thread - June 2020

I am getting worried about something like a cultural revolution here, too. Am considering prepping for leaving the country. This really could go uncomfortably far. The "Spanish Inquisition" with modern surveillance and stylometry is a terrifying thought. Struggle sessions condensing out of Twitter into reality, state subsidized elite overproduction for the last few decades combined with a significant portion of youthful elites in massive debt, this amplified by demographic changes further restricting supply of elite positions, Thucydides trap. I feel it may be prudent to get out now. I am not a well-traveled person. Never bothered getting a passport. I will apply for one now, but they are hard to get these days. Shulman's link for purchasing citizenship is an interesting hedge.

For someone like yourself, doing vital work, such hedges are even more important.

Interview with Aubrey de Grey, chief science officer of the SENS Research Foundation

>He's 57, and looks older than many in their 70s.

I think you are underestimating how much the beard contributes to this.

human psycholinguists: a critical appraisal

They already assigned >90% probability that GPT-2 models something like how speech production works.

Is that truly the case? I recall reading Corey Washington a former linguist (who left the field for neuroscience in frustration with its culture and methods) claim that when he was a linguist the general attitude was there was no way in hell something like GPT-2 would ever work even close to the degree that it does.

Found it:

Steve: Corey’s background is in philosophy of language and linguistics, and also neuroscience, and I have always felt that he’s a little bit more pessimistic than I am about AGI. So I’m curious — and answer honestly, Corey, no revisionist thinking — before the results of this GPT-2 paper were available to you, would you not have bet very strongly against the procedure that they went through working?

Corey: Yes, I would’ve said no way in hell actually, to be honest with you.

Steve: Yes. So it’s an event that caused you to update your priors.

Corey: Absolutely. Just to be honest, when I was coming up, I was at MIT in the mid ’80s in linguistics, and there was this general talk about how machine translation just would never happen and how it was just lunacy, and maybe if they listened to us at MIT and took a little linguistics class they might actually figure out how to get this thing to work, but as it is they’re going off and doing this stuff which is just destined to fail. It’s a complete falsification of that basic outlook, which I think, — looking back, of course — had very little evidence — it had a lot of hubris behind it, but very little evidence behind it.

I was just recently reading a paper in Dutch, and I just simply… First of all, the OCR recognized the Dutch language and it gave me a little text version of the page. I simply copied the page, pasted it into Google Translate, and got a translation that allowed me to basically read this article without much difficulty. That would’ve been thought to be impossible 20, 30 years ago — and it’s not even close to predicting the next word, or writing in the style that is typical of the corpus.