Have you ever faced the frustration of writer's block, staring at a blank screen? In such moments, seeking assistance from a chatbot might seem appealing. The rise of ChatGPT has highlighted the potential of generative AI, which can craft screenplays, poems, and jokes that often rival human creations.
However, a groundbreaking study reveals that while AI can surpass the creativity of an average office worker, it still cannot compete with the highest levels of human imagination.
A collaborative research effort from the Université de Montréal, Concordia University, and the University of Toronto has published a comprehensive comparison between human creativity and advanced AI capabilities.
The study uncovers an intriguing paradox: while AI models like GPT-4 excel in standard creativity assessments, they struggle against exceptionally creative individuals.
This research serves as a reality check amidst the ongoing AI hype, emphasizing that a significant number of people lack creativity. This underlines the importance of supporting artists, musicians, and writers, who are currently facing challenges from the advancing AI landscape.
The Semantic Distance Game
To objectively assess creativity, the researchers employed the Divergent Association Task (DAT), a straightforward test where participants must generate ten unrelated words.
For instance, a low score might result from listing words like "cat" and "dog," while a high-scoring individual might come up with "galaxy, freedom, harmonica, and photosynthesis."
Over 100,000 human participants and several major AI models, including GPT-4, were tested. The scoring was based on "semantic distance," a method that measures how conceptually distant two words are.
The study's findings were striking. GPT-4 outperformed the average human score, while other models like GeminiPro matched human performance.
"Our research indicates that some AI systems can now exceed the creativity of average humans on specific tasks," states Professor Karim Jerbi, the lead author.
However, the term "average" carries significant weight. When isolating the top performers, the results flipped; the most creative 50% of humans outperformed all tested AI models. The gap widened even further among the top 10% of humans.
In essence, AI has elevated the baseline of creativity but has not broken through the upper limits. "Even the most advanced AI systems still fall short compared to the most creative humans," Jerbi adds.
The "Ocean" Problem and Temperature Control
The study also revealed peculiar patterns in AI creativity. While humans produced a diverse array of words, AI models often repeated specific terms excessively.
For example, GPT-4 frequently used the word "microscope" in 70% of its responses and "elephant" in 60%. The newer GPT-4-turbo model was even more repetitive, with "ocean" appearing in over 90% of its outputs.
In contrast, human participants used common words like "car" and "dog" in only about 1% of their responses, suggesting that AI's apparent creativity often stems from predictable patterns in its training data.
This phenomenon relates to a parameter in large language models known as "temperature," which influences how adventurous the AI is in word selection. A lower temperature yields predictable responses, while a higher temperature encourages more creative and varied outputs.
Researchers discovered that adjusting this temperature could enhance AI creativity. At higher settings, GPT-4's performance improved, surpassing approximately 72% of human participants.
"The way instructions are framed significantly impacts creativity," the study notes. For instance, instructing the AI to use an "etymology strategy" led to notable improvements in its scores.
From Words Lists to Literature
The research team extended their investigation beyond word lists, challenging the AI to compose haikus, movie summaries, and short stories. Here, the limitations of AI became even more evident.
While AI could generate competent text, humans consistently excelled in "Divergent Semantic Integration," showcasing an ability to blend diverse ideas into cohesive narratives. Visual data analyses indicated that human writing occupied a distinct "region of meaning" compared to machine-generated text.
Importantly, while increasing the "temperature" improved AI storytelling, it did not aid in producing haikus, indicating that structured poetic forms demand a level of intentionality that AI struggles to replicate.
This study arrives at a pivotal moment for writers and artists, yet its conclusions are encouraging. AI is not replacing quality art; rather, it is presenting new challenges.
"Despite AI achieving human-level creativity on certain tasks, we must move beyond a misleading sense of competition," Jerbi emphasizes. "Generative AI has emerged as a powerful tool to enhance human creativity."
The findings suggest a future where AI serves as a baseline generator, assisting creators in overcoming average brainstorming hurdles, allowing them to focus on reaching the heights of originality that machines have yet to achieve.
