Rodrigo Heck


Sorted by New

Wiki Contributions


AI won't have wishes or desires. There is no correlation in the animal kingdom between desires and cognitive function (the desire to climb on the social hierarchy or to have sex is preserved no matter the level of intelligence). Dumb humans want basically the same things as bright humans. All that suggests that predictive modeling of the world is totally decoupled from wishes and desires.

I suppose it is theoretically possible to build a system that also incorporates desires, but why would we do that? We want Von Neuman's cognitive abilities, not Von Neuman's personality.

No. It performs much worse than AI systems.

Can you predict the shape of a protein from the sequence of its aminoacids? I can't and I suspect no human (even with the most powerful non-AI software) can. There is so much we are unable to understand. Another example is how we still seem to struggle to make advances on Quantum Physics.

It seems less obvious to me that human grokking looks like 'stare at the same data points a bunch of times until things click'

Personally, I don't think it's that different. At least for language. When I read some unrecognizable word in a foreign language, my mind tries first to retrieve other times I have seen this word but haven't understood. Suppose I can remember 3 of these instances. Now I have 3 + 1 of these examples in my mind and, extracting the similar context they share, I can finally deduct the meaning.

A possible avenue to explore is to expand these models to multilingual data. There are perhaps a lot of high quality text uniquely available in other languages (news, blogs, etc.).  Anyways, IMO this effort should probably be directed less on acquiring the largest amount of data and more on acquiring high quality data. Chinchilla's scaling law doesn't include quality as a distinctive property, but we have reasons to believe that more challenging text are much more informative and can compensate low data environments.

I don't understand why and how the authors reached the conclusion that IQ is not that important from this study on Swedish CEOs. They are 1.5 standard deviations above the mean. This is a huge effect.