LLM chatbots

Why the old chatbots failed, and what changed

It is worth being precise about this, because most people's opinion of chatbots was formed by something genuinely bad.

The previous generation worked by matching what you typed against a list of phrases someone had written in advance. If your question was on the list, you got the right answer. If it was not — and it usually was not, because people phrase things a hundred ways — you got "Sorry, I did not understand that", and after two rounds of that everyone learned to type "agent" immediately.

A language model does not match phrases. It reads the sentence the way a person does, which means "do you deliver to Klang", "can send klang ah" and "delivery Selangor available?" all land as the same question. That single change is what makes the difference between a chatbot customers avoid and one they actually use.

But reading the question well introduces a new risk that the old ones did not have. A language model will produce a fluent, confident answer whether or not it knows anything. Left unchecked it will invent a delivery charge, quote a discount you never offered, or promise a lead time you cannot meet. That is the actual engineering problem, and it is where most of the build effort goes.

Grounding: how it is stopped from making things up

The technique is called retrieval-augmented generation, and the idea is simpler than the name.

Rather than letting the model answer from what it absorbed during training, we give it a library of your own documents — price list, delivery terms, returns policy, warranty conditions, opening hours. When a customer asks something, the system first searches that library, pulls out the passages that actually relate to the question, and then asks the model to answer using only those passages.

The practical effect is that the answer is traceable. When it tells a customer delivery to Johor is three working days, that came from a specific line in a document you wrote and can point at. If you change the document, the answer changes the same day. Nothing needs retraining.

It also means the system knows when it has nothing to work from. If the search returns no relevant passage, the correct behaviour is to say so and offer a person — not to improvise. We test this deliberately, by asking it things your documents do not cover and checking that it declines rather than invents.

This is also why the first deliverable is often just writing your policies down. Many businesses have never put delivery terms or the return window in one place; it lives in the head of whoever has been there longest. That document is useful whether or not you ever ship the chatbot.

What it should never handle

A chatbot that tries to do everything is worse than one with clear limits, and the limits are a design decision made before anything is built.

Complaints go to a person. Someone who is already unhappy does not want an efficient reply, they want to feel heard, and the fastest way to lose a customer permanently is to have software handle their frustration smoothly. The bot's job there is to recognise the tone, apologise briefly, and hand over with the full conversation attached.

Anything involving money it is not certain about goes to a person. Custom pricing, negotiated rates, credit terms, refunds. The rule we use is that if getting it wrong costs you money or a relationship, it does not get answered automatically.

Legal, medical and safety questions go to a person, always, regardless of what your documents say. And when a customer asks directly to speak to someone, that request is honoured immediately — no retention loop, no "let me try to help first". That single behaviour does more for how the bot is perceived than any amount of answer quality.