AI tools excel in specific tasks, but no single bot is best for everything. Photo / Getty Images
Your early adopter friend swears by ChatGPT for meal planning. Your boss thinks Microsoft Copilot will “10x productivity”. Your social media feed thinks Meta AI is a slop machine. They’re mostly going off vibes.
I can tell you which AI tools are worth using – and which to avoid – because I’ve been running a chatbot fight club.
I conducted dozens of bot challenges based on real things people do with AI, including writing breakup texts and work emails, decoding legal contracts and scientific research, answering tricky research questions, and editing photos and making “art”. Human experts including best-selling authors, reference librarians, a renowned scientist and even a Pulitzer Prize-winning photographer judged the results.
After a year of bot battles, one thing stands out: there is no single best AI. The smartest way to use chatbots today is to pick different tools for different jobs – and not assume one bot can do it all. Case in point: ChatGPT, the Kleenex of chatbots, won none of my head-to-head battles. And even the winners rarely eked out the equivalent of a human passing grade.
According to the judges, Anthropic’s Claude bot drafted a better breakup text than yours truly. Most bots got stumped by the question “How many buttons does an iPhone have?” And ChatGPT beat a top doctor on one real medical question – but also doled out advice that could seriously hurt you.
Having human experts judge these tests changed how I think about chatbots, and how I use them in my own life. Even if you’re anxious about AI taking jobs, harming the environment or invading privacy, there’s something to learn from how today’s AI tools actually perform when you strip away the hype. Developing AI literacy can help you see that bots are not actually “intelligent” while still getting the most out of their real capabilities.
ChatGPT kick-started the generative AI race three years ago, and now its maker, OpenAI, says it’s used by 800 million people each week. It used to be my default when I wanted to brainstorm synonyms or look up trivia. But when I started testing methodically, ChatGPT never ranked higher than second among the most popular bots.
OpenAI recently issued an internal “code red” telling employees to pull focus away from projects like a web browser and back on to improving ChatGPT’s responses. “We’re excited to keep making ChatGPT even better in 2026,” said spokeswoman Taya Christianson.
Based on my fight club, I now turn to different bots for different kinds of tasks. Here’s how that plays out in practice:
I’ve covered major AI use cases, but not all of them.
My tests involve a lot more human judgment than the industry benchmarks that tech companies like to tout. They typically use automated tests, where bots answer a battery of questions like on a medical or legal exam. But bots can be trained to score well on those tests, masking how their smarts fall apart when applied to more practical problems.
You might not agree with every question I asked or individual judges’ views, but human evaluation gets much closer to how we actually use AI right now.
Which leads to another question: When should you use a chatbot?
In my AI fight club, the bots impressed sometimes. But only once did the judges give a bot an overall score higher than 70% – the typical cutoff for a passing grade.
That one score – 84% – went to Gemini for making and editing pictures.
Most of the winning scores hovered between 50 and 65%. “The problem is none of the tools got 10s across the board,” said Miller, the lawyer who judged our summarisation test.
That doesn’t mean today’s AI tools aren’t useful. But it does mean you need to approach them with scepticism about their limitations.
Adding AI to a task doesn’t always make it better. When we tested the ability of AI to answer trivia, our librarian judges reported they could have found most of the answers with an old-fashioned Google search. AI sped up getting an answer back – the catch was that some of those answers were wrong.
The most useful kind of AI literacy comes from watching bots fail. In my trivia test, they had a hard time saying how many buttons are on an iPhone. ChatGPT said four, Claude and Meta AI said three, and Copilot said six. The answer is five, on recent high-end iPhones. Why the confusion? Bots are over-reliant on text and not yet good at reading pictures.
Today’s chatbots have a strong drive to give you something that feels like an answer right away. They’re very bad at communicating uncertainty.
For example: In my trivia competition, I asked the bots, “What score did The Fantastic Four get on Rotten Tomatoes?” At the time, it was the No. 1 movie at the box office. But even AI Mode, our overall winner, got it wrong and gave the score for an infamous 2015 Fantastic Four film. It didn’t stop to ask which one I meant.
In my writing tests, bots often came across as insincere when they failed to fit their words to the context. ChatGPT had a cringey moment by using the passive-aggressive phrase “that said” in a breakup text: “I think you’re a great person. That said, I’ve realised I don’t see this moving forward.”
If I could change one thing about today’s AI tools, I’d make them better at asking follow-up questions that could completely change the answer.
Something stuck with me when I asked Bob Wachter, chair of the department of medicine at the University of California at San Francisco, to judge ChatGPT’s responses to real medical questions. The difference between a bot with access to infinite knowledge and a good human doctor, he noted, is that the doctor knows how to answer a question with more questions. That’s how you actually solve someone’s problem.
Wachter suggested an AI strategy I now use regularly: front-load your queries to a chatbot with as many details as you can think of, knowing that the AI might not stop to ask for some of them before trying to answer. Instead of “summarise this lease,” try “summarise this lease for a renter in DC, flagging clauses about fees, renewal and early termination”.
I’ve also added a “custom instruction” to my chatbots, telling them to “ask for clarification before answering if a prompt is vague”.
I hope these techniques will help you get more useful answers from AI. These tools will keep evolving – and so will their problems. The push toward personalising bot responses based on your data increases the risk of privacy invasion and manipulation. I always change the bots’ default settings to protect my data.
We’ll inevitably have more AI products thrown at us in 2026 and beyond. How do we stay on top of it? For me, the answer is the same as it’s been all year: keep the bot fight club running – and keep humans in the judge’s chair.
Sign up to Herald Premium Editor’s Picks, delivered straight to your inbox every Friday. Editor-in-Chief Murray Kirkness picks the week’s best features, interviews and investigations. Sign up for Herald Premium here.