Just before Christmas, I was sitting with a group of academic colleagues, munching on mince pies and discussing a topic that’s becoming impossible to ignore: the use of AI in academia. Concerns about plagiarism, copyright, and academic integrity dominated the conversation, but what struck me most was the confidence with which some of my colleagues claimed they could tell, just by reading, which parts of a student’s assignment had been generated by AI. I couldn’t resist gently pushing back. AI tools are evolving so rapidly, and people use them in such varied ways, that the idea of reliably “spotting AI” no longer holds water. More importantly, should it even matter? When assessing a piece of work, isn’t the quality of the output more significant than the tools used to create it?
Consider this: if you see a nail hammered neatly into a piece of wood, can you tell whether it was driven in by a nail gun or a hammer? Does it even matter, as long as the nail does its job? I have this conversation often in filmmaking, where I argue that audiences don’t care whether a film was shot on an Arri Alexa or an iPhone. They respond to the story, the performances, and the emotional impact — not the equipment. The same principle applies to AI-generated content. What matters is whether the output achieves its purpose — plagiarism and copyright infringement notwithstanding. Yet, some persist in believing they can differentiate between human and AI-created work. This belief, however, says more about human psychology and bias than it does about the capabilities of AI.
The Psychology of Overconfidence
Humans have a remarkable ability to overestimate their skills and expertise. Psychologists call this the Dunning-Kruger effect: the less someone knows about a subject, the more likely they are to overrate their expertise (Kruger & Dunning, 1999). This cognitive bias provides an explanation for why so many people are overconfident about their driving abilities — and, crucially, why some are equally confident in their ability to detect AI-generated content when, in reality, they can’t.
This is supported by research which shows that when participants are asked to identify whether a piece of text, an image, or even a piece of music was created by AI or a human, their accuracy hovers around 50–60% — barely better than flipping a coin (Zhu et al., 2024). Worse still, their confidence levels often far exceed their actual performance. The result? A false sense of certainty that clouds judgement.
This overconfidence isn’t harmless. In an era where AI is increasingly integrated into creative and professional workflows, the insistence on separating “human” from “machine” risks overshadowing far more important conversations about quality, ethics, and accountability.
The Rise of AI’s Sophistication
A significant reason for our inability to distinguish AI-generated content from human work is that today’s AI systems are incredibly sophisticated. Models like GPT and other large language models, as well as tools for generating images, music, and even videos, are trained on massive datasets encompassing human language, creativity, and cultural nuance. These datasets allow AI to produce content that is coherent, contextually appropriate, and often indistinguishable from human-created material.
For example, a study by Brown et al. (2021) tested professional editors — experts trained to spot errors, inconsistencies, and patterns in writing — and asked them to differentiate between human-written essays and those generated by AI. The results were telling: even these trained professionals struggled to reliably identify which was which. If the experts can’t do it, what chance does the average person have?
AI’s adaptability also plays a key role in its effectiveness. These tools can produce content in a wide range of styles and tones, shifting effortlessly from formal academic writing to casual conversational language. This versatility makes it increasingly difficult to assess authorship based solely on the “feel” of the content.
The Role of Bias
Our inability to reliably distinguish human from AI-generated content is compounded by bias. Imagine being shown two identical pieces of writing: one labelled “written by a human,” the other “generated by AI.” Which would you rate as more creative, meaningful, or high-quality? Most people instinctively favour the “human” text, even when the two are identical (Zhu et al., 2024).
I’ve experienced this bias firsthand. Not long ago, I created a music video using RunwayML, an AI tool for generating video. When I showed the video to an audience, the initial feedback was overwhelmingly positive. People praised the video’s creativity and emotional resonance. But once I mentioned that AI had been used in its creation, the tone of some in the audience shifted. Suddenly, the same video was dismissed as “soulless” and “lacking originality.” Other filmmakers I’ve spoken to have faced similar reactions, with some even receiving hate mail or threats after disclosing their use of AI.
One probable source for this resistance to AI-generated content is a deeply ingrained belief that creativity is uniquely human. When AI intrudes on this domain, it challenges our sense of what makes art, writing, and other creative outputs “authentic.” Yet, as these biases reveal, our perceptions of quality are often shaped more by preconceptions than by the intrinsic value of the work itself. I remember many years ago seeing somebody throwing a number of prints of Picasso’s paintings into a skip — not the sort of thing you’d hang on your wall at home…
The Use of AI in Creative Workflows
Adding to the difficulty of distinguishing human from AI-generated work is the sheer variety of ways in which people use these tools. AI tools aren’t just content generators; they’re increasingly being used as tools to streamline and refine the creative process. Some rely on AI for brainstorming ideas, others to generate a rough draft or enhance clarity in their writing. The most recent releases of Adobe Creative Suite have AI deeply embedded in creative workhorse tools such as Photoshop and Premiere Pro. These uses make it challenging to assess where AI ends, and human effort begins.
Moreover, AI’s accessibility means that individuals with varying levels of skill and expertise are using it. From students crafting essays to filmmakers creating visual effects, the integration of AI into everyday workflows ensures that its influence is both pervasive and subtle.
Why It’s Time to Move On
The obsession with trying to distinguish human from AI-generated content is not just misplaced — it’s counterproductive. AI is already deeply embedded in industries ranging from filmmaking and journalism to marketing and education. Instead of fixating on whether a piece of content was created by a human or an AI, we should shift our focus to more meaningful questions: Was the content created ethically? Does it serve its intended purpose? Is it accurate, meaningful, and original?
This shift in perspective is particularly important in education. Rather than policing the use of AI, educators could adapt their assessment methods to emphasise skills and understanding that AI cannot easily replicate. Oral defences, reflective essays, and collaborative projects can all help demonstrate a student’s grasp of the material.
In creative industries, the focus should be on maintaining ethical standards and ensuring that AI-generated content aligns with the goals and values of the project. Transparency about AI’s use can help foster trust, but it should not detract from the quality or impact of the final product.
As AI becomes an increasingly powerful tool in creative and professional domains, the debate over whether content is human or AI-generated is losing relevance. The evidence is clear: we are not as skilled at distinguishing between the two as we think, and our biases often cloud our judgement to the detriment of all.
References
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., … & Amodei, D. (2021). Language Models Are Few-Shot Learners. Retrieved from https://arxiv.org/abs/2005.14165
Kruger, J., & Dunning, D. (1999). “Unskilled and Unaware of It: How Difficulties in Recognising One’s Own Incompetence Lead to Inflated Self-Assessments.” Journal of Personality and Social Psychology, 77(6), 1121–1134.
Zhu, T., Weissburg, I., Zhang, K., & Wang, W. Y. (2024). “Human Bias in the Face of AI: The Role of Human Judgement in AI Generated Text Evaluation.” arXiv preprint arXiv:2410.03723. Retrieved from https://arxiv.org/abs/2410.03723
Atchley, P., Symons, C., & Vega, J. (2024). “Human and AI Collaboration in the Higher Education Environment: Opportunities and Concerns.” Cognitive Research: Principles and Implications. Retrieved from https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00547-9
