Tomás B.

Occasionally think about topics discussed here. Will post if I have any thoughts worth sharing.

Wiki Contributions

Comments

GPT-5 with a context window that can fit entire code bases is going to be very scary. Particularly if you think, as I do, that agency is going to start to work soon. I really do think at least "weak recursive self improvement" of the form of automating AI research/training loops is on the table relatively soon. 

I would like to register a prediction. I believe a GPT-4-level model that has been RLHFd for humour will be super-human or near superhuman at humour.  At least in the 99th percentile of professional comedians. My intuition is humour is much easier than people think, and current models fail at it mostly because the forms of RLHF existing models use pushed them into humourlessness . 

Conditional on this being true, he must be very certain we are close to median human performance, like on the order of one to three years.  I don't think this amount of capital can be efficiently expended in the chips industry unless human capital is far less important than it once was. And it will not be profitable, basically, unless he thinks Winning is on the table in the very near term.

I feel 5 trillion must be a misprint. This is like several years worth of American tax revenues. Conditional on this being true I would take this as significant evidence that what they have internally is unbelievably good. Perhaps even an AI with super-persuasion!

It is such a ridiculous figure, I suspect it must be off by at least an OOM. 

My take on self-driving taking forever is driving is near AGI complete. Humans drive roughly a million miles between fatal accidents; it would not be particularly surprising if in these million miles (where you are interacting with intelligent agents) you inevitably encounter near AGI-complete problems. Indeed, as the surviving self-driving companies are all moving to end-to-end approaches, self-driving research is begining to resemble AGI research more and more.

I bought index funds. I would say it has the advantage of being robust to AGI not happening, but with birth rates as they are I am not so sure that's true! If we survive, Hanson's economic growth calculations predict the economy will start doubling every few months. Provided the stock market can capture some of this, I guess learning how to live on very little (you really want to avoid burning your capital in this future, so should live as modestly as possible both so you can acquire capital and so you can use as little as possible until the market prices in such insane growth) and putting everything in index funds should be fine with even modest amounts of invested capital.  However, I doubt property rights will be respected.

Any evidence for it working? Seriously doubt.

Nope. Sadly. And if there were, your intellect would not be impressive for such tools would reach fixation.

If it’s any consolation, all the brilliant people able to make many multiples of your salary due to being born with a better brain - while almost to a man being incredibly smug about it - will soon be losing intellectual death matches with toaster ovens.

And OpenAI has explicitly said this is what they want to do! Their Superalignment strat looks suspiciously like "gunning for RSI".

Load More