Matches my experience. Some examples:
- Waymo:
- First ride: "holy shit it's driving itself!"
- Second ride: "dang this is so cool the future is here!"
- Third ride: "why is Waymo so bad at driving?!?"
- Claude Code:
- First time using it: "this is incredible it mostly figured it out on its own!"
- Every subsequent time using Claude Code: "wtf why didn't you do it right?!?"
- Sonos:
- First song: "wow, it made a real song out of my half-formed idea!"
- Every other song: "why does this suck at writing good songs 😡"
It just goes on and on. I expect to be thoroughly bored by my robot butler that does everything for me, my AI personal assistant that never makes any mistakes, and everything else that's about to happen about 5 minutes after it becomes the new normal.
1I think a lot of people, including futurists, set their standards for "crazy sci-fi future" far too low.
Consider the Industrial Revolution. Take the median person from before it started and after it was over: the daily labor of the latter would be unrecognizable to the former. Their house might look quite different, and the experience of keeping it certainly would be. You would be able to tell the two lived in different worlds just by taking a walk around the block, or looking in their pantry. I don't know my history well enough to delineate this by exact dates, but if you compare a life in 1780 to its equivalent in 1880 you'd surely see what I mean.
Now imagine all that happened in the span of two years. People would not be unimpressed and disappointed! People would be freaking the fuck out! Imagine taking a five minute walk and seeing your home surrounded by artifacts a person in 2020 wouldn't even recognize; imagine the best meal you've ever had - not just the ingredients but the taste quality too - becoming commonplace. Imagine this happening all at once, too fast to acclimate, at the same time as everyone you know is simultaneously reorganizing their daily routine around the new changes.
I'm not claiming this will or won't happen in 3-10 years, only that if it did, it would be a really crazy sci-fi future, and I think people would react appropriately. Nothing AI has done to date is really comparable. If you're a software engineer your job maybe looks a little different now, but I'm a game developer (adjacent at least?) and AI hasn't affected my daily labors all that much, even though I use it pretty often. I do the same things, I just ask Claude to help with some of them. The math progress is exciting, but I don't understand most of it, and it doesn't affect my material existence at all; for me it's no different than hearing that NASA discovered something cool about Mars, or whatever. And this is as someone with prior interest in the field: I know plenty of people in different bubbles who have never actually had a conversation with an LLM, even now.
Personally, I was pretty impressed and excited by LLM progress from 2020-2024, then kinda disappointed/bummed out about the period after that, though I'm hesitantly considering becoming impressed and excited again given recent events. Most of my impressed/excitedness before 2024 was "within the hobby", in the sense of "wow, this thing is so much smarter than GPT-3!", rather than "wow, this thing is restructuring the very bedrock of my existence!" I wonder how much the present-day sense of crazy-sci-fi-ness is predictive (the HuggingFace incident isn't very important on its own but implies very important things could happen very soon), and how much is looking at things from "within the hobby", in the sense that they'd only matter to you if you have an existing interest in AI.
Speculating the difference between those who "feel the AGI" vs don't:
People are social: If most people they see aren't acting impressed, then they shouldn't either. A vibe I get from social media is that AI is bad (for environment, jobs, AND capabilities). This is also similar to reactions to COVID early 2020. This is also similar to myself where I didn't invest time into AI safety until I made 1 friend to work on it over discord (it became socially viable to do at that point).
AGI soon would be inconvenient to our own narrative: I remember spending a few hours to understand the chinchilla scaling laws; my stomach sunk. In the back of my mind, I had always expected to, you know, get married, have kids, live a long and healthy life, and this punctured a hole in that.[1] I was quite depressed for a couple of days, but fortunately had read how to grieve well.
For students, a large narrative is finishing their degree & getting a job. Even if timelines aren't that short, does it really make sense for them to keep studying that when a (metaphorical) meteor could hit earth in a few years? I think it's a natural learned behavior to ignore thoughts/feelings of dissent in your current narrative you're relying on, because no narrative --> no motivation (and it might look bad to acknowledge faults in your plan).[2]
A few related ones:
- AI hasn't literally cured cancer (I do believe there will be a cascading effect once it has done something of this caliber)
- Scaling laws aren't felt in a gut sense: I learned it in a way that viscerally affected my predictions of the future. Most people DON'T have a strong model of scaling laws. Although, I think some equations or graphs would lack punch without going into depth the capabilities of eg GPT-2,3,4, etc.
AGI soon would be inconvenient to our own narrative: I remember spending a few hours to understand the chinchilla scaling laws; my stomach sunk.
This was me about four months ago, except my trigger of multi-day dread and depression was seeing the METR graph of task-completion time horizons.
I guess some of us have just had that moment of "oh shit" and concluded x-risk is a real thing that we really need to worry about, and some of us haven't.
Are we all (in this thread and this community) just wired in an unusual way that makes us assume worst case scenario is the most worthy of our attention?
(I was mostly using "cure cancer" as an example that folks will be impressed with, but I'll straightforwardly answer your question)
Cancer is many different things (even sometimes within the same tumor). Biology is complicated, and not my expertise, but a subset of cancers would be helped by understanding the functionality of genes and how they interact (ie Gene Regulatory Networks (GRNs)). For example, if a type of cancer works through [Gene 1], then we knock it out, it might instead use [Gene 2] (ie bypass pathway activation). It'd be great if we understood that Gene 1 & 2 do the same thing, so drugs should target both.
I think this can be done soon through Mech Interp on bio models[1] (ie single cell expressions models, which are different but you can finetune parts to find GRN).
Currently only 14% of patients are eligable for targetted therapy and only 7% respond successfully. Better models here would increase that percentage (unsure on how much).
There are many other types of cancer that work differently that I haven't thought about.
For timeline: end of 2027 for the technical results and basic wet-lab functionality. Unsure on FDA/approval side of things.
For ALL cancer, I really don't know. I would speculate that 2027 models would work well enough to help with the research, but the bottleneck being good data-collection. (Again, heavy speculation).
- ^
Disclaimer: I do mech interp
A more accurate GRN would also help the methods Ihor presents here for intelligence augmentation. I really am unsure on whether these'd actually work, but it actually seems way easier and tractable than genetic engineering in babies. This is due to it NOT being gene editing (and instead just like a normal drug, but to eg increase neuro-plasticity or something); which is easier for both mechanical and social reasons.
Something seems missing from this picture about why they react that way. Like, I often get reactions when I try to explain it as though claiming the thing is powerful is thereby working for the enemy; as if there's an implicit belief that yes, it's obviously powerful in the terrain, but admitting it would grant the social power that the creators want, or something. People call the AI surge a bubble as if believing it is is load-bearing for them somehow. There's more to this than just unimpressed and disappointed
1) What do you expect? We already live in a world full of sci-fi wonders.
2) LLM's as they currently stand are a video-calling in the 90's level of futuristic. As in, kinda what we associate with the future, but the fictional versions are way more powerful than reality.
3) People recognise that AGI isn't for them. It's current trajectory will further enrich a few. Leaving the rest of us to an uncertain and probably worse off fate. So I would say its pretty understandable that people aren't exactly jazzed about that prospect
Average person not inculcated with a steady diet of sci-fi media -- and I'm not talking about butter-soft Star Wars stuff, but actual hard sci-fi in the vein of your Egans and Baxters -- tends to lack the ability to imagine/perceive such possibilities as RSI and therefore defaults to "nothing ever happens" as a response, however ill-informed that reaction may be.
People are not logical, and talk is cheap. This explains the vast majority of popular discourse.
on a discord server of ~100 people, I have interested 2, maybe a 3rd. one because he has an incurable degenerative disease and needs some hope, the other two are math or adjacent, one of whom weeps for joy with me whenever an open problem is resolved and is now using Fable to help with his research. most people pretend it doesn't exist or use it for frivolous things. my intuition is that they are mostly in a "it's too unpredictable, so no use wasting time thinking about it" camp. it isn't so much "sci fi is childish fantasy", but "bets on sci fi are boom or bust, and I have a mortgage"
Noting that this doesn't match my experience. Most of my mathematician friends are very much feeling the approaching end of mathematics, and a number of them got quite scared of the Hugging Face incident too.
Computers have been commonplace to ubiquitous for around forty years now. That's four decades of experience showing us that machines can do things that humans can't. We take it for granted that a computer is better at math[1] than we are, that it can 'remember' more clearly than we do, that it can access huge swathes of information that we can't. On top of this, fiction has taken these strengths and exaggerated them, so that many (most?) people would believe that a computer can enhance a photo to be able to accurately reproduce a face from a few indistinct pixels, or that it can in some sense think and plan ahead, or 'move beyond its own programming'.
Most people don't really grasp how different 'general intelligence' is to the niche domains that computers have excelled in to date. How is an AI agent any different from Clippy? How is a system beating the Go champion any better than the one which already defeated the Chess champion twenty years before? We are just bad at estimating how much intelligence is required for a given (reasonably complex) task.
Then there is the fact that humans have been the uncontested most intelligent thing that we have encountered for pretty much our branch's entire evolutionary history[2], so there is a strong inherent bias against thinking that other things could equal or exceed our intelligence[3].
So, in my opinion, people will remain sceptical and dismissive of advances in AI right up until the moment that they can no longer deny that the thing talking to them from their device is way smarter than they are, and on an individual basis people will probably waver, vacillating between being impressed and then retreating into scepticism for a while before they are fully convinced.
How is it possible you haven't met or talked to people impressed with Moltbook, or OpenAI covert message board, or vibemath or coding agents? And Software Engineers NOT worried about their jobs? They must be seniors or staff and very adept at being accelerated by AI rather than replaced by it.
I feel like more people than not are impressed and worried about the near, and distant, future when it comes to AI and its capabilities. Regardless of whether it is considered AI, AGI, ASI.
More often that not at my company, Claude is prompted with a Feature Request or Bug Report (+logs, etc) and solves the problem much faster (and accurately, when accounted for speed) than any human employee would be able to. And people claim loudly: "we're so going to be replaced". "Obviously" we keep self updating and are quickly changing from Software Engineers to AI Agent "Whisperers". We are also noticing an increasing need for Product Owners/Managers and a very reduced need for Junior/Midlevel SWEs.
My family worries about what the youngests' future will be like. Should they even endeavour in STEM focused studies? (They are mostly toddlers atm). Other families worry the same. What jobs will be left for our kids? I could go and on about the uncertainty of the future, but the gist of it is everyone is very impressed by the rate of evolution of AI agents and other AI tools and it is leading to a big sense of insecurity about the future. 3 years, 5 years, 10 years, 20+ years.
If people were not impressed, why would they worry?
I guess my question is, have you really not talked to anyone worried about AI and the future? Or they are worried but not impressed?
How is it possible you haven't met or talked to people impressed with Moltbook, or OpenAI covert message board, or vibemath or coding agents?
No idea, genuinely. If I live in a bubble, it must be a Truman Show crazy level of bubble.
And Software Engineers NOT worried about their jobs?
I think some people around me do "worry" in an abstract sense, like "I expect that in 5 years I will see some news headline that says 'Unemployment rate has hit record highs'", not in a visceral "I will lose MY job" sense. I don't know anybody who expects >90% human unemployment within their lifetime though.
People are very, very adaptable to the status quo. Some believe in omnipotent, omniscient agents that care about them personally and don't bat an eye except maybe on Sundays.
I'd say the primary cause is lack of surprise; our brains tune out the unsurprising in order to conserve energy and avoid accidentally wireheading on sunsets or honey.
Peoples' beliefs can imply very surprising things but unless the surprise is experienced and sensed it won't mean much to most people. Once sensed, they'll move past it and try to cope with the new normal. Futurists are good at imagining the implications in enough detail to feel the surprise beforehand.
For me, personally, the surprise happened back when learning about the idea of ASI itself (some mix of sci-fi books and movies in ~teens). It was clear that it was physically possible and therefore likely to happen barring natural human extinction, so my imagination went wild on it for quite a while. Now it's a blurred future of the SotA eval graph lines waiting for me. I expect unexpected changes and significant future surprise, but the meta-surprise happened a long time ago. I don't really plan for where to be in 5 years or more. I haven't for at least 5-10 years. Live, work, dance, enjoy folks. Wait.
I'm surprised by your circle of friends. Many people around me are impressed. However very few think software development jobs are going to disappear.
I am amazed and astounded almost daily and I think that the times we're living in are the coolest thing ever.
I have a ton of optimism for the future, the only thing I'm sad about is that I'm just not good enough at math to keep up and truly appreciate what I'm seeing on vibemathed. And it's only going to get better!
Maybe my sense of wonder is like a baby discovering an iphone and forever losing the ability to focus on anything else. It's likely that I'll be dumber in the future than I would have otherwise because I use AI tools for things I don't understand instead of learning and internalizing them.
I usually try to make posts with graphs and numbers, or at least something more than just opinions and anecdotes. This one is not like that.
Very verbose disclaimer: I have never met - for any reasonable definition of the word met, including online-only conversations on Discord with people whose faces I've never seen - even a single person who has ever stated, orally or in text, publicly or in private, that they believe ASI will be created within their lifetime. I do know people who believe that ASI is possible to create in theory, but they believe it will be completely unrelated to LLMs, Transformers, neural networks, reinforcement learning or any other contemporary technique/architecture, and is 100/1000/some astronomical number of years away.
Ok, with that out of the way, here's the main point of this post: AI will soon (3-10 years, depending on which advancements exactly we're talking about) be solving Millennium math problems, finding cures for many diseases, making novel bioweapons, making most software, hacking a lot of software, and much more, and most people won't be impressed or even will be disappointed.
(we'll leave aside the question of whether in the long run there will be humans left to witness anything ASI-made)
I've noticed that no matter what AI does, nobody (that I know) is ever impressed. I have shown vibemathed.com to my friends, parents and people on Discord that I talk to a lot, and didn't receive a single "holy shit". LLMs have solved hundreds of open problems, including problems that were unsolved for decades. 10 years ago this would be considered sci-fi.
Perhaps these results are too abstract, unless you have a math background? But I did show it to a guy with a math background, he wasn't impressed either! And he said there is "no reason" to believe LLMs will ever be solving Millennium math problems (he knows about the recent advancements like the Jacobian conjecture counterexample and OpenAI's paper on cycle double cover conjecture and such)! He's of the opinion that ASI belongs in an ontologically different category and that there is no evidence that ASI will be made using anything like modern machine learning architectures and training methods.
What about recent cybersecurity incidents, especially the part where OpenAI's agents made a covert message board? Nobody was impressed by that either. One person compared it to Moltbook and said it's the same thing. Everyone else who either forgot about Moltbook or never knew about it in the first place wasn't saying "holy shit". Their occupation and education made no difference, at least as far as I can tell.
Maybe I'm approaching this from the wrong angle? Perhaps what's needed for people to be impressed and "feel the AGI/ASI" (I actually dislike the phrase "feel the AGI", but it fits here) is a lot of exposure to frontier LLMs in their daily life? How about people who use Codex or Claude Code regularly?
Nope, they aren't impressed either! I know 5 people who use Codex/Claude Code, none of them are expecting ASI within their lifetime! They don't expect to lose their jobs either. One out of those 5 has a similar opinion to the mathematician I mentioned above, that ASI is possible to create but LLMs have nothing to do with ASI, same for neural networks, reinforcement learning, etc. Out of those 5 people, only one believes we need to invest more effort and money in AI safety/alignment after the recent hacking incidents like the HuggingFace one! Oh, and one of them unironically compares LLMs to NFTs and thinks they will disappear in the next 1-3 years and nobody will ever use them again because they are expensive and useless. He says that as he's using coding agents. Look...don't ask me, I'm not a mindreader.
There are some other examples, such as my mother who worked as a translator for most of her life being unimpressed with LLM translations, or my chemist friend being unimpressed with AI advancements in science, but I think these are sufficient. I already knew that believing that ASI will be created within our lifetime is a rare thing. What I didn't realize was that being impressed by advancements in AI along the way is a rare thing too. So rare that I am yet to see it at all!
You know the "everyone generalizes from n=1. At least, I do" joke/quote? Perhaps I fell victim to drawing confident conclusions from a small sample size. But even after accounting for my small sample size, I'm quite sure about my conclusion: in the next 3-10 years we will see things that previous generations described as sci-fi, and barely anyone will be impressed.
And some people will even be disappointed. I've once seen a guy who said "Man, if transformer based LLMs turn out to be enough for AGI, then nature sucks anyway." and "like if transformers are capable of AGI, maybe we weren't that special anyway" and "Man, that would be some of the most unsatisfying thing if it happened". He also added "I'll start joining believing that once they can do theorem proving properly". I'm bummed that he's not active on Discord anymore, I sure would love to see his reaction to vibemathed.com stats.