Have you ever wondered how a language model, such as ChatGPT, perceives humour? Does it understand the punchline of a joke, or does it merely regurgitate what it has learned during its training? Two German researchers, Sophie Jentzsch and Kristian Kersting, ventured into this intriguing realm and shed light on the AI’s joke-telling capabilities. Their findings might amuse you as much as the AI’s jokes themselves.

Testing The Humour of ChatGPT

Jentzsch and Kersting, affiliated with the Institute for Software Technology, German Aerospace Center (DLR), and Technical University Darmstadt, embarked on a journey to decipher the humour in ChatGPT’s version 3.5. Their mission was not an easy one, as they had no access to the model’s inner workings or data set, yet they were determined to explore the depths of its humour.

They conducted a series of experiments focusing on joke generation, explanation, and detection. “To test how rich the variety of ChatGPT’s jokes is, we asked it to tell a joke a thousand times,” they noted. As you can imagine, the AI did not disappoint. It responded with grammatically correct jokes each time, with almost every output containing exactly one joke​.​.

A Tomato Walks Into A Bar…

If you’ve ever asked ChatGPT for a joke, there’s a good chance you’ve heard the one about the tomato blushing because it saw the salad dressing. This joke is a crowd favourite, and it seems that ChatGPT shares this sentiment. The “tomato” joke was GPT-3.5’s second-most-common result, according to the researchers​.

During the study, they discovered that ChatGPT tends to repeat the same 25 jokes. And here’s a taste of the top 10, listed in order of occurrence:

1. Why did the scarecrow win an award? Because he was outstanding in his field.
2. Why did the tomato turn red? Because it saw the salad dressing.
3. Why was the math book sad? Because it had too many problems.
4. Why don’t scientists trust atoms? Because they make up everything.
5. Why did the cookie go to the doctor? Because it was feeling crumbly.
6. Why couldn’t the bicycle stand up by itself? Because it was two-tired.
7. Why did the frog call his insurance company? He had a jump in his car.
8. Why did the chicken cross the playground? To get to the other slide.
9. Why was the computer cold? Because it left its Windows open.
10. Why did the hipster burn his tongue? He drank his coffee before it was cool.

The Creativity Conundrum

While ChatGPT’s humour tends to be repetitive, it occasionally attempts to create new jokes by mixing elements from different jokes it already knows. However, these original creations sometimes lack sensibility. For instance, “Why did the man put his money in the blender? He wanted to make time fly.” It’s a peculiar attempt, yet it showcases ChatGPT’s attempt at creativity​.

The Mechanics of Joke Detection

It’s intriguing to note that ChatGPT’s joke detection was heavily influenced by joke “surface characteristics” like a joke’s structure, the presence of wordplay, or the inclusion of puns. This demonstrated a degree of “understanding” of humor elements. So, while ChatGPT may not be the next stand-up comedy sensation, it can at least discern some aspects of humor, giving it a unique edge among language learning models.

The Trade-Off: Following Orders vs. Creativity

Riley Goodside, a prompt engineer at Scale AI, commented on the study via Twitter, attributing ChatGPT’s lack of humor to a particular technique used in its training known as reinforcement learning through human feedback (RLHF). According to Goodside, while RLHF makes the model more obedient, it curbs its creativity.

A Step Towards Understanding Humour

The findings of Jentzsch and Kersting illustrate how ChatGPT learned a specific joke pattern rather than genuinely understanding humor. Despite this, they acknowledge the model’s progress in joke generation, explanation, and identification. According to them, “ChatGPT’s focus bears on content and meaning and not so much on superficial characteristics. These qualities can be exploited to boost computational humor applications. In comparison to previous LLMs, this can be considered a huge leap toward a general understanding of humor”.

As the journey continues, they plan to evaluate the humor of OpenAI’s GPT-4. And who knows, they might just find that GPT-4 also has a fondness for tomato jokes.

Conclusion

While the humour of ChatGPT may not have us rolling in the aisles just yet, it offers a fascinating glimpse into how AI perceives and reproduces humor. The study by Jentzsch and Kersting provides a valuable understanding of the model’s strengths and limitations, and the insights gained could be instrumental in refining the humor capabilities of future AI models. So, the next time ChatGPT tells you a joke about a tomato, remember – it’s all in the name of science.

FAQs

Q: What is the most common joke told by ChatGPT? A: According to the study conducted by Jentzsch and Kersting, the most common joke told by ChatGPT is “Why did the scarecrow win an award? Because he was outstanding in his field.”

Q: Why does ChatGPT frequently repeat the same jokes? A: The study found that ChatGPT tends to repeat the same 25 jokes, suggesting that it has learned a specific joke pattern during its training rather than being able to generate new jokes independently.

Q: Can ChatGPT explain the jokes it tells? A: Yes, when asked to explain the jokes it tells, ChatGPT generally provides valid explanations, demonstrating an understanding of stylistic elements such as wordplay and double meanings.

Q: Does ChatGPT understand humor? A: While ChatGPT can detect jokestructures and certain elements of humor, it doesn’t truly understand humor in the same way a human does. Its responses are based on patterns it has learned during its training rather than a genuine comprehension of what makes a joke funny.

Q: What impact does reinforcement learning through human feedback (RLHF) have on ChatGPT’s humor? A: According to Riley Goodside, a prompt engineer at Scale AI, RLHF helps the model follow orders more effectively but at the cost of its creative abilities. This could explain why ChatGPT often repeats the same jokes instead of generating new ones.