My intuition, completely unjustified, is jokes will prove easier than most suspect, even very good jokes. Unfortunately, there are large incentives to hobble the humor of such models - but greentext prompts provide a small hint of what they are capable of. I suspect explicitly optimizing for humor would work surprisingly well. It would be interesting to use :berk: or other Discord reactions as data for this.
One idea for a short story I never explored is the eternal sitcom - a story about a future where everyone has AR glasses and a humor model feeding them good lines.
There would be a scene at the start where a comedian deals with hecklers, and plays with them as a judo master does a neophyte, and a scene in the middle where an augmented heckler - a "clever Hans" - (one of the first users of the model) "completely destroys" the comedian.
Story idea: AI strategically uses humor to defeat humans. Basically, if AI wants to prevent you from doing something, it will invent and make popular a really funny joke that will completely annihilate your status if someone tells the joke after you proposed doing the thing, or worse, were observed doing the thing.
Something like "if you worry about too many paperclips, you are just compensating for a short penis", only hysterically funny, so it is impossible to ever comment on the paperclip industry growing exponentially without someone responding with the joke and completely derailing the debate.
It should be a sequence of jokes, increasingly funny, following multiple steps of a strategy AI uses to get more power. At the beginning, it just seems random; towards the middle of the story it becomes obvious to the protagonist that the taboo topics are all about things that make the AI more powerful. Yet the protagonist is afraid to comment on that, after watching other people getting socially destroyed by commenting on the latest taboos.
(Bonus points if the author of the story uses an actual chatbot to invent the jokes.)
Humour is already a powerful weapon of human manipulation. Our ex prime minister Boris Johnson wouldn't have lasted nearly as long as he did (probably would never have got in) if he weren't very good at getting us all to laugh.
Standard Mildly Amusing Jokes and Perspectives
I think this is because it is aiming at a broader audience as opposed to a narrower one. 1) Because it's a special. 2) Because Seinfeld is looking to appeal to a wide demographic. I'd love to see what Seinfeld would come up with if he was targeting a narrow audience. I'd probably pay hundreds of dollars for that.
Since you're in NY, I'd highly recommend New Joke Night at The Comedy Cellar. The night I went was one of the best nights of my life. Wil Sylvince was the host and he was incredible.
I may be interpreting Zvi's GPT scale incorrectly, but I think it is mostly a measure of novelty and not quality. Higher GPTs will (presumably) take more and more inferential steps away from the training data and have more ability to seem novel. So a GPT that does a specific comedians style exactly, and is hilariously funny, is probably a lower hanging fruit than one that invents its own comedic style that is meh-to-okay.
So, if I understand it right, the 6's are the comedians who are very weird. For example, back when I was a teenager the UK TV/radio comedy scene I would say Milton Jones would have a very high GPT value. This is because every other comedian told long interconnected stories with all the jokes woven together, while pacing around the stage and trying to interact with the audience. Typical content would be fairly dense in references to sex and overall be a bit rude and dirty. The stories were all calibrated to seem kind-of-beleiveable, like the story would start with the comedian talking about how badly their first gig went and then become harder to credit as it continued. Milton Jones just stood up straight and machine gunned one liners. The topics were always clean, nothing dirty, racial or rude. Nothing remotely intended for you to think was real/true. He wasn't necessarily the funniest (depending on taste), but the fact that he had very deliberately looked at the expected style and intentionally leaned 100% against it is presumably something GPT-X doesn't do for low X.
Part of Seinfeld's humor comes from his delivery of the material (e.g. intonation, rhythm, loudness, facial expressions, hand gestures, and other elements of nonverbal communication). So it's worth noting that you need more than a funny comedy script from GPT-X to replace Seinfeld.
I think that was the point. Comedians of the future will be performers. They will not write their own jokes but be increasingly good at reading the lines written by AI.
When Chat GPT came out I asked it to write a Seinfeld episode about taking an IQ-test. In my judgment it was just as good and funny as every other Seinfeld episodes I’ve watched. Mildly amusing. Entertaining enough not to stop reading.
Thanks to Jordan Schneider gifting a last minute ticket for an amazing seat, serendipity led me to go to Friday night’s performance by Jerry Seinfeld at the Beacon Theater.
If I had to describe the show in one word, I’d say it was unsurprising. Jerry was Jerry. He had thoughts. Mostly he had complaints. They were all Very Seinfeld.
If I had two words I might say mildly amusing. Which was good enough for a worthwhile evening. Live performances are something special. Every time I’ve gone out to a comedy show, even if a bunch of it was kind of lame, I have been happy I came. The correct bar for worth watching is actually lower in person than at home.
The thought I couldn’t shake as I went home was, what GPT level was that on?
This first came to me during the opening act. His opening act was if anything too on the nose. Very Jerry. Much Seinfeld.
It felt very much like he had GPT-X put together a package of Standard Mildly Amusing Jokes and Perspectives, with the prompt that the performer was black, old, male, never married and the opening act for Jerry Seinfeld. Given the material, delivery was solid.
The question was, what was the X in GPT-X?
I am confident our current incarnation, which is GPT-3.5 or so, can’t do it.
What about GPT-4? Could go either way, if you could try out bits and select winners.
What about GPT-5? Yes, absolutely, if it is what I’d expect, that should work.
So I’d give that act a GPT-level rating of 4.25.
Jerry himself was on a higher level. Not as high as his peak. I’d give him a 5.
What about the best stand-up shows and specials?
I’ve seen 6s. I worry I may have never seen a 7.