Faustine Li
49
7
Faustine Li has not written any posts yet.
Faustine Li has not written any posts yet.
The best response I've heard against the simulation hypothesis is "If we're simulated, why aren't we asked to do meaningful work?" Think about why we would want to simulate personalities in our near to medium term future: to answer questions, to help us work faster, to entertain us, to be our companions, etc.
So why aren't we asked to complete the equivalent of Mechanical Turk questions? Is compute is so unfathomably cheap that they there's no economic reason we would need to "pay rent" so to speak? Even if that's the case, why don't our creators want to interact with us directly? In other words, if we're all video game characters, who is... (read more)
Let me go one further, here: Suppose an GPT-N, having been trained on the sum total academic output of humanity, could in-principle output a correct proof of the Riemann Hypothesis. You give it the prompt: "You're a genius mathematician, more capable than any alive today. Please give me a complete and detailed proof of the Riemann Hypothesis." What would happen?
Wouldn't it be far simpler for GPT-N to output something that looks like the proof of the Reimann Hypothesis rather than the bona-fide thing? Maybe it would write it in a style that mimics current academic work far better than GPT-4 does now. Maybe it would seem coherent on first glance. But there's... (read more)
I liked the format, but let me pick on a particular point. What makes you confident that in seven years China will be meaningfully ahead of the West? My intuition is that the West still has the best education and economic centers to drive R&D and those have significant moats that don't get shaken up that quickly. You're pretty vague about your justifications other that impressive levels of progress. I see it as a "rising tides float all boats" situation where progress is being accelerated everywhere by open sharing, an economically conducive environment for AI research, and availability of compute.
This took a weird turn in an article that I thought would explore the basic scientific challenges of nanotech and why we don't have it yet. I do think that the erosion of religion has some negative externalities in modern society (ie. lack of easy meaning and direction in life and downfall of local community kinship), but no I don't think that is the main reason why we don't have nanotech specifically. I don't even think that's the primary reason we are more polarized politically now (my current thoughts lean towards changes in information consumption, communication, and trust in institutions).
But specifically nano-printers? Of course people want that, as much as people want quantum computing, fusion energy, brain-computer interfaces, and life-extension. The benefits are obvious, from the money-making opportunities alone. Maybe reality is just disapointing: it's a harder problem than people originally expected without any economically viable intermediate steps (the kind that bolster AI reasearch now) so progress is stuck in a quagmire.
I agree that most of the recent large model gains have been due to the surplus of compute and data, and theory and technique will have to catch up eventually ... what I'm not convinced on is why that would necessarily be slow.
I would argue there's a theory and technique overhang with self-supervised learning being just one area of popular research. We haven't needed to dip very deeply yet since training bigger transformers with more data "just works."
There's very weak evidence that we're hitting the limits of deep learning itself or even just the transformer architecture. Ultimately, that is the real limiter ... certainly data and compute are the conceptually easier problems to solve. Maybe in the short-term that's enough.
Sutton and Barto's Reinforcement Learning textbook: http://incompleteideas.net/book/the-book-2nd.html
Elements of Statistical Learning: https://hastie.su.domains/ElemStatLearn/ as well as the more introductory version: https://www.statlearning.com/
Deep Learning textbook: https://www.deeplearningbook.org/
Bayesian Data Analysis: http://www.stat.columbia.edu/~gelman/book/
Mongolian BBQ is completely divorced from the ethnic cuisine of Mongolia, which is heavy in meat and fat and (within my limited knowledge) not very flavorful. I've watched a food vlog and ... yeah, it's fatty lamb boiled in plain broth. Food is subjective, but I'm sure that wasn't what you were thinking.
What we know as Mongolian BBQ was actually invented in Taiwan on the 1950's and given that name for a pretty arbitrary reasons. (As an aside how many food names with a country name are actually from that place? Usually it's tenuous connections at best like how Hawaiian pizza was named because of pineapples and actually invented in Canada).