An Era of Magical AI Thinking

An Era of Magical AI Thinking

An Era of Magical AI Thinking

Is it only the algorithms that are hallucinating or have we lost touch with reality too?

We’ve all heard the cautionary tales. The wild fabrications and hilarious (albeit worrying) falsehoods.
The moments when an AI engine invents facts with a straight face that would put even the most skilled human liar to shame.

But is the most pressing issue really the accuracy of the algorithm? Or is it the unprecedented level of trust we’ve so quickly placed in a new class of tools that feel both wondrously powerful and fundamentally unknowable at the same time?

Recent research out of the University of Melbourne showed that two-thirds (66%) of employees who use AI at work have relied on AI output without evaluating it.

Should we really be worrying about the moments when we catch the algorithm out in a lie… or why we’ve so quickly come to place so much trust in AI in the first place?

THE NATURAL HUMAN BIAS TOWARDS WONDER AND ILLUSORY BELIEF
Long before the recent explosion of AI, humans have been known to fall back on Magical Thinking when faced with uncertainty, complexity, or forces beyond our understanding. We’re primed to see patterns where they don’t exist, to attribute intent and agency whether there is only mechanism, to find comfort in a (false) sense of control, to believe based on appearances even if we don’t understand the mechanics behind.

If we don’t understand how something works, but it appears coherent, authoritative, or familiar, our brains are more than willing to fill in the gaps. AI doesn’t create these biases, it activates them.

A GIANT LEAP MAKING TECHNOLOGY MAGICAL ONCE MORE

For years, digital progress was largely incremental. Then AI arrived and broke the pattern.

At a time when many of us had grown complacent about technological change, generative AI reintroduced wonder into our relationship with machines. Suddenly, technology wasn’t just responsive. It was conversational. Creative. Intuitive. Seemingly human.

Just a few years ago, ChatGPT was making its first ripples in the mainstream. Today, it’s difficult to find a digital tool or system that doesn’t have AI embedded (whether visibly or invisibly).
Like the introduction of personal computing or the early web, technology once again feels magical. Full of limitless possibility – both unimaginable and yet oldy believable at the same time.

This is where things get risky. Because when technology feels magical, scrutiny often gives way to awe.
How much of our trust is warranted? How much is simply a mirage of AI competence?

INNATE HUMAN BIASES AND ILLUSIONS TO WATCH OUT FOR

  • Illusions of Action: the anthropomorphism of technology has long demonstrated how the mirage of a familiar action can give us a false sense of confidence in what’s happening under the digital ‘hood’. But just because it feels like you’re ‘chatting’ with a person, that doesn’t mean Gen AI brings any true human-like judgement.

  • Illusions of Control: for millennia, superstition and rituals have given humans an inflated sense of control in situations beyond our understanding. From harvest rituals to rain dances, we humans are known to seek formulaic rituals when faced with complex systems. AI introduces a modern variant: the belief that if we just phrase the prompt right, we can fully control the outcome.

  • Illusions of Availability: behavioural sciences shows us how easily we place a false sense of trust in readily available and easily recallable information. We trust what’s in front of us and what’s front-of-mind far faster than we question what we might have missed or overlooked. AI outputs arrive instantly, neatly packaged, and linguistically polished - making them feel more reliable than slower, messier human inputs. What’s missing is often invisible and the unseen gaps are rarely questioned.

  • Illusions of Prediction: Perhaps the most seductive illusion of all. When AI generates plausible future scenarios, trends, or recommendations, it can create the impression that uncertainty has been “solved.” But pattern extrapolation is not foresight. Confidence is not accuracy. And probability is not destiny.

EMBRACING AI WONDER, STAYING GROUNDED IN REAL-WORLD REALITY

Ultimately gaps in AI accuracy aren't the problem. Unquestioned belief and trust are.

Embracing AI ethically and strategically means grounding your practice in principles that balance new technological possibility with rigorous human judgement.

  • Collaboration Over Delegation: AI should support human judgement, not replace it. Just as we wouldn’t hand strategic, ethical, or contextual decisions entirely to a junior team member, we shouldn’t fully outsource them to an algorithm.

  • Inspiration Over Answers: Use AI to explore possibilities, test ideas, and provoke thinking. Not to prematurely conclude it. Good questions matter more than fast responses.

  • Interpretation Over Acceptance: AI outputs are stimulus to be interpreted, challenged, and contextualised. Not ‘answers’ to be followed. The true value and meaning still emerges through human sense‑making.

INTERESTED IN LEARNING MORE?

Get in touch