The Future of AI Agents
7Mitchell_Porter
3kavya
2Richard_Kennaway
1kavya
2Raemon
3kavya
2Raemon
3kavya
New Comment
This is quite a place to make a pitch for a new kind of AI service. I thought Less Wrong was known as a haven of AI doom. Haven't you heard that AI is either going to kill us all, or else transform the world in some ineffable posthuman way?
Even if I put that aside for a moment, the idea of having an agent in charge of spending my money is alarming, perhaps because I have so little to begin with; and also because I expect it to want to spend my money, because that's what consumer society is about. You do say (statement 5) that the agent should "help me save", and frankly, the people at the lower levels of society probably need an agent that is primarily defensive, that will not just do their financial planning, but which will protect them from scams and exploitation and bad habits. The problem is, that would put you at odds with the business model of a lot of people, who subsist in this world by e.g. convincing other people to buy things that they don't actually need.
Now, maybe social classes higher up the economic food chain can afford to have an agent that is not always in defensive mode. They have money to spend on things beyond survival, and they're determined to get out there and spend it, they just want to spend it as well as possible. This starts to sound like a problem in utility maximization, the agent's first task being to infer the utility function of the user.
Here we have entered conceptual territory familiar to Less Wrongers. The upshot is that if the agent is smart enough, it will deduce that what the user really wants is {inexpressible goal derived from transhuman extrapolation of human volitional impulses}, and it will cure death and take over the world in order to bring about a transhuman utopia; but if it extrapolates incorrectly, it will bring about a transhuman dystopia instead.
This may sound weird or whimsical if you really haven't encountered AI-safety alignment-lore before, but seriously, this is where AI naturally leads - beyond humanity. Even if we do get a world of millions of empowered consumers with friendly agents making their purchasing decisions, that is a transitional condition that will be swept aside by the godlike superintelligences that are the next logical stage in the evolution of AI (swept aside, unless those godlike superintelligences actually want their human wards to live in market societies).
Having expressed my cynicism about actually existing capitalism, and my transhumanist conviction that AIs will replace humans as the ones in charge, let me try to be a little positive. Some things are worth paying for! Some services available in the marketplace, maybe even most of them, do have genuine value! Being an entrepreneur can actually be a net positive for the world! And even if you were ultimately just in it for yourself, you do have a right to make your way in the world in that fashion!
You might wish to investigate Pattie Maes from MIT, and dig up her thesis on reflective agents. In the 1990s, she was a guru of the future agent-based society, I'm sure her thoughts and her career would have a few lessons for you. And if I was really was an entrepreneur trying to design an AI agent intended to act as an economic surrogate for its human user, I might think about it from the perspective of George Reisman's Capitalism, a hybrid work of Objectivist philosophy and Austrian economics, written according to the premise that capitalism done right really is the most virtuous economic system. It has nothing to say about Internet economics specifically, but it's a first-principles work, so if those principles are right, they should still be valid even when we're talking about a symbiotic economy of AIs and humans. (In fact, you could just feed the book to GPT-5 and ask it to write you a business plan.)
Thanks for the thoughtful comment.
(1) I agree that as you progress to a higher class, they are more willing to let the agent spend money.
(2) But if you focus the agent to be primarily defensive, I believe almost all classes of society would appreciate that. Why wouldn’t I want the same thing for a better price (either through a drop or negotiation), quality, or alignment with my values? You could incentivize the agent through a cut of the savings.
(3) If you are truly concerned about these agents departing from their given purpose, you can always require the user approval after every purchase. So the agent is like “Hey here’s something you might interested in because of your {needs} and {something you stated you wanted}”. And you can respond with yes or no and for the final stage, you approve (Apple Pay + Face ID). The advantage of the tokenization tech behind Apple Pay is that it never shares your card details with the merchant (instead payment tokens), and I would imagine it could be the same for these agents.
(4) We do live in a world where people try to convince you to buy things you need. I do think this model will slightly collapse in the future. Soon, with personalized AI controlling almost all of the information we see, very few companies/products will truly be able to create something inherently viral and game the network of agents. So I actually think we’re going to see a shift toward companies that make what people truly want because otherwise, the agents will never bring that product to their human users’ attention. This would improve market efficiency!
I’m new to LessWrong so I probably glossed over the super intelligence possibility more quickly than I should have. In the meantime, I think we should still build useful tools like these economic surrogate agents. People are using ChatGPT to buy (instead of Google or visiting 5 different platforms), and if you make that experience more tuned to the user, I think there’s an opportunity. Any kind of superagent(s) might be our Last Interface. Will check out Pattie Maes!!
I personally would love to give my personal agent pocket money ($20-$30) and see what it can surprise me with[5].
Circumstances vary, but the last thing I need is MOAR STUFF arriving at my home out of the blue.
Q: Wouldn't this idea end up as an ad business once it matures?
Maturity might instead be businesses training their own AIs to advertise to the agents. The end game is people buying stuff and people selling stuff and nobody knowing why, but the AIs assure everyone they're making the best deals.
1The endgame you have described is a strong possibility. I guess you might try to fight that through increased transparency and ensuring alignment between agent behavior and user preferences. I do see the core issues still stand like the possibility of collusion, hidden incentives, and preference manipulation.
ChatGPT is uncool culturally in my generation (I am 18). The relationship is one-way and reactive.
This is the first I heard this, and I'm interested to know more detail (I'm in my late 30s). Are all LLMs uncool? ChatGPT in particular? are nearby things "cool"?
The average person doesn't care what LLM/technique is used in their favorite applications. It just so happens to be that ChatGPT came out first, and they stuck with it simply because they are unaware of other options. For those who know about the existence of Claude or Gemini, they constantly model-switch. "LLM" is likely a foreign term to most non-CS majors.
With respect to my generation, most people use ChatGPT for answers. In fact, I had the chance to once share a $200/mo ChatGPT Pro subscription with 15 other students. Throughout the 8 months, there were barely any questions that stemmed from true curiosity. The idea of "prompting" for an answer is a very transactional interaction (think of the people in high school who are "friends" with smart people just to get homework help) . There is no persistence or relationship built between conversations. The only thing that could be considered cool about the tech is the "fast answers", but people start expecting more and get disappointed quickly when they try to one-shot a project.
From a personal perspective, I do think LLMs are cool. I just think they need to be wrapped around a thoughtful experience to be made more accessible to the masses (which is why not all LLM wrappers are bad). I can't wait for a future where the LLM opens up a conversation with something I may or may not find interesting.
Okay, so, like, teenagers use them, just in a kinda functional way where they're not, like, excited about it? is like, more like "google" than like "myspace/facebook/snapchat" when each of those things was cool?
yes, that's a good way to put it. In the beginning, it was definitely something young people were excited about. But LLMs are a lot more like utilities now. You use it when you need it and move on with life.