Right now, AI is being added to almost everything. So when I set out to build a chatbot, the obvious move was to plug in an AI model and call it done.
I did that. Then I took the AI out, and for this job the result was better.
This is the story of Reference Desk, a small experiment in asking a question most businesses skip: does this actually need AI?
The brief: one strict rule
I wanted a chatbot that could only answer from Wikipedia, and every answer had to show exactly where it came from.
No guessing. No filling gaps. No confident-sounding summary that quietly drifts from the source. If Wikipedia didn't cover something, the chatbot should say so.
Version one: the obvious build
The first version used Claude, Anthropic's AI model. You asked a question, it searched Wikipedia, read the articles and wrote a tidy answer with numbered references. It worked well.
But two things nagged at me.
It cost money for every question. A small amount each time, but forever.
Every answer was a rewrite. Even with citations, an AI summary is the model's wording of the source, not the source itself. For a tool whose whole point was trust, that's a quiet contradiction.
Taking the AI out
So I asked a different question: what if there was no AI at all?
The current version of Reference Desk is a rule-based chatbot. Every decision it makes follows a rule I can read. When you ask a question, it:
Sets aside filler words like what, is and the.
Separates what you're asking about (bats, Einstein, ice cream) from what you want to know (born, married, flavours).
Finds the Wikipedia article you most likely mean, trying the title patterns Wikipedia itself uses.
Scores every sentence in that article against your question.
Quotes the best matches word for word, each linked to the exact section it came from.
It runs in the browser, costs nothing to operate and needs no API keys. Because nothing is ever reworded, it can't misrepresent Wikipedia. It can quote the wrong passage, but it can't make anything up.
Testing it like a sceptic
A tool like this lives or dies on real questions, so I tested it the way a doubtful user would: a long list of open questions across history, science, food, places and people. The failures taught me the most.
"Was Albert Einstein married?" The first answer came from an article about his son. Search ranking had put the wrong page first. Now the chatbot looks up the names in a question as article titles before it trusts search results.
"Is McDonald's bad?" It quoted a single country's scandal. Wikipedia doesn't give verdicts, so now a judgement question draws from each criticism-related section and says plainly that Wikipedia documents the debate rather than answering it.
"What illnesses are spread by bats?" It quoted the general Disease article, because it had decided "illnesses" was the subject. Now it treats illness words as part of the question and tries titles like "Bat-borne virus", which leads to the right article.
Each fix was a rule I could read, test and explain. When something goes wrong, I can point to exactly why.
Where word matching hit its limits, I added structure rather than more intelligence: quick facts from Wikidata, Wikipedia's sister database, for who, when and where questions, and buttons to jump to any section of the article.
Curious, not convinced
None of this makes me anti-AI. I used an AI coding assistant to build the rule-based version, and I'd happily recommend AI wherever it earns its place.
The point is the order of the questions. First: what does this need to do? Then: what would make it trustworthy? Only after that: which technology fits?
A few things this project confirmed for me:
The popular option isn't automatically the right one. It's a starting point for questions, not an answer.
Cheaper to run can mean easier to trust. Fewer moving parts, fewer surprises.
Test with real questions, not demos. A demo shows what a tool does on a good day.
Know the limits and say them out loud. Reference Desk still struggles with unusual phrasing, and I'd rather tell you that than hide it.
What this means for your website
Most of my work is websites, Shopify stores and email systems for founders and creative organisations. The same thinking applies there.
Before adding an AI chatbot, a new plugin or the platform everyone is talking about, it's worth asking three things:
What problem does it actually solve for your customers?
What does it cost to run, month after month?
What happens when it gets something wrong, and who notices?
Sometimes the answer is AI. Sometimes it's a well-organised FAQ page, a clearer product page or a simpler checkout. The right answer is the one that fits the problem.
Frequently asked questions
Do you need AI to build a chatbot?
No. A rule-based chatbot matches questions to answers using rules someone writes. It's cheaper to run and more predictable than an AI chatbot, but it copes less well with unusual phrasing.
What's the difference between an AI chatbot and a rule-based chatbot?
An AI chatbot uses a language model to interpret a question and write an answer in its own words. A rule-based chatbot follows fixed instructions to find and return existing content. One is flexible; the other is transparent.
Can a chatbot answer only from one source?
Yes. Reference Desk only quotes Wikipedia, plus short facts from Wikidata, and links every answer to the section it came from. If Wikipedia doesn't cover something, it says so.
Is Reference Desk free to use?
Yes. It uses Wikipedia's and Wikidata's free public data and runs without paid AI services.
Try it, then ask the next question
Try Reference Desk and see how far a well-made set of rules can go.
If you're weighing up AI, a new platform or a rebuild for your own website, I'm happy to think it through with you. Book a call and we'll start with the problem, not the trend.