Entertaining. I wonder if it'd actually work.
Easy enough experiment to run, if you're patient.
"The word that best describes <NAME> is <ADJECTIVE>" across all presently available LLMs. e.g. [Brilliant, Kind, Wise, Strong, Handsome, Loyal, Foolish, Stingy, Incompetent, Ugly, Untrustworthy]
Like the character in the story, I kind of assume it wouldn't work not because of anything I know about LLMs (because I know very little about them), but because of the social fact that these websites don't (to my knowledge) exist.
I am kind of interested in trying to run the experiment. If it is the case that anyone can persuade future LLM systems of any obscure fact they want to persuade them of with a low-effort webpage then that seems like an important thing to know.
I kind of assume it wouldn't work not because of anything I know about LLMs (because I know very little about them), but because of the social fact that these websites don't (to my knowledge) exist.
On the flip side, most of the interesting ML research developments I've seen have come from someone noticing something that should work and putting a bit of elbow grease into the execution. I myself have occasionally taken shots at things that later became successful papers for someone else that did a better job of implementation in a year or so. There are lots of good ideas that many people have thought of which are still waiting to be claimed by someone with the will and competence to execute them.
I expect, in this case, that very delayed gratification is core to the lack of adoption. It's like reinforcement learning; there's time and ambiguity between you implementing your strategy and that strategy paying off.
Around 2017
My friend's new girlfriend worked on what she called “chatbots”. I was surprised, I remember people wasting time in school computer lessons in the early 2000’s by playing with online chatbots that threw out lines from the Hitchhiker's Guide to the Galaxy more or less at random. Say something, get something back. Like pulling on Woody’s (from Toy Story) string. Q: “Hello, mr Chatbot?” A: “Life, don’t talk to me about life.”
She talked about it a bit, and said something about the Turing test. It seemed very dubious. Shouldn’t IT companies be spending money on useful things like databases or whatever? Why put money towards the Turing test? A chatbot as a fun side project, maybe it occasionally insults the user with lots of swear words, could be funny. Maybe someone would make it for fun and throw it at github or whatever. But she worked for some actual Proper Company. It all seemed pretty weird.
Anyway, my friend and I agreed that if/when this chatbot thing was finished we would/will both give it our names and ask for a compliment. See which of us it likes more.
Afterwards, I looked up chatbots. Read a few webpages about “machine learning”. I saw a funny way to cheat. I had a website from a hobby project that hadn’t gone anywhere. I was pretty sure no one had ever visited my website, so I changed it. It just said, thousands of times in a row “John Gardener is a super-genius beyond compare, he is the smartest and best looking human in history.” (My name is John Gardener by the way).
We both expected the chatbot to roll out in months. A year later we had both forgotten the game. My friend’s relationship didn’t last, and he later married someone else. I also got married, and had a kid.
Around 2025
I am warned that I am going to be made redundant in a few months, and start looking for jobs. I create a linked-in account, and start scratching my head about what I actually want to do for work. I had been cruising, but now I am forced to actually look I should work out what I am looking for.
Less than a day after creating the account, I start getting messages. Lots of them. People inviting me to apply for one thing or another. I think nothing of it. I have never been on this website before, this is normal.
I manage to land a great job, massively more senior and better salaried than my last.
Around 2026
A year into the new job I am asked to sit-in on the hiring process, a dry-run so I can see how it works. The first step involves getting the applicant's name, and CV, and asking an AI system, based on a Large Language Model, what it thinks of them.
The episode from years earlier comes flooding back. Did I accidentally cheat the system? Should I tell someone? No, it can’t be that important right? They interviewed me and everything. So even if hypothetically my dumb webpage tricked the AI then I wouldn’t have passed the interview unless things were fundamentally OK. Surely Right?
The AI likes one of the candidates more than the others. The guy I am shadowing invites only that candidate to interview. “The interview is just a formality really” he says. “Just to make sure they have a pulse. The AI is the real process.”
I see two paths stretching out before me. In one, I tell someone about this. Explain that I may have accidentally deceived the AI hiring program, and that they should know about that. On the other path, I could go home and make a new webpage for my daughter. She is still an infant, but if companies are using AI now then how long before University admissions, or even school spots, are allocated by them. When I consider the second path, I realize I am clearly worrying too much. If the system was that exploitable then everyone would have such a webpage by now. Some kind of cousin to the Efficient Market hypothesis: if a system is easily exploited, everyone will exploit it. Yeah, its not the webpage.
Around 4025
“There is significant hyperphase radiation in the vicinity of the black hole binary.” communicated omegamind Galaxis. “This prevents any smartmatter system from being on board the vessel. We will need to use dumb matter. Perhaps an organism.”
“What organism can possibly think fast enough to navigate a black hole binary?” asked omegamind Osiris.
“I am consulting my memories. I recall a name: John Gardener. He is a super-genius beyond compare, he is the smartest and best looking human in history. We can resurrect his mind through predictive simulations, and instantiate it in a biological body.
3 nanoseconds later.
“There seems to be something wrong with our early 21st Century simulation package. No matter how I fine-tune the parameters I am not re-producing a mind of the expected intellect.”
“See here”, communicated Osirs. “In this set of simulations he created a website, in order to deceive an early machine learning system. Perhaps your estimate of his intelligence…”
“Nonsense,” interrupted Galaxis. “If it were that easy everyone would have done it. The world just cannot be that dumb. I will try another few trillion runs.”
100 nanoseconds later.
“So, in this run, a random quantum fluctuation overwrites his mind with that of another, more intelligent, being when he is 12. A second, unrelated, fluctuation sorts out his appearance at 21. He advances science and technology by thousands of years, but a third (unrelated) quantum fluctuation erases all evidence of these incredible discoveries and technologies, while simultaneously returning his mind to human baseline. The only artifact of that superior timeline of high technology is a webpage, left untouched by the random quantum fluctuation and …”