Stay up to date with AI tools, model releases, and developer workflows that matter.
It has been revealed that weekly. Free. One click to leave.
Industry observers note that 7 Day Free Trial. Cancel Anytime.
Ask one of the newest language models whether a backpack comes in green, and it'll usually admit it doesn't know. Ask it how to return that backpack, and it may send the customer to a returns page, an account screen, and a return label that the store has never had. That's still a hallucination: an answer that sounds right, comes with total confidence, and isn't true.
Whatever you're building, the fix is the same: stop trusting the model with the parts it can get wrong, and check its work before anyone sees it.
According to recent reports, in this tutorial we'll build a small support bot for a made-up outdoor gear store, three times over. The first version is the prompt plenty of bots start with. Then we'll apply five fixes, run six tricky customer emails through every version, and finish with a bot where the model can't slip a made-up price, date, or product past the code.
In a recent development, an LLM hallucination is output that's fluent and confident but not backed by the facts: a product that doesn't exist, a policy nobody wrote, a date that's a week out, or a citation to a paper that was never published. The word covers two different failures:.
Each one needs a different fix, so it helps to keep them apart.
According to recent reports, A language model writes by predicting likely text, one token at a time. It has no separate store of facts it checks against, and nothing in that process says "stop, you don't know this". Training adds to the problem. OpenAI's research on why language models hallucinate points out that when models are graded only on how many answers they get right, a guess scores better than "I don't know", so they learn to guess.
A few things make it worse in your app:.
According to recent reports, newer models are much better at the first one. They'll often say they don't know a price rather than invent one. But they still fill gaps with what's typical, and what's typical isn't always what's true for your business. You can't train that out of a model you call through an API. What you can change is what you ask it to do, what you give it, and what you check.
According to recent reports, A made-up answer costs more than one bad reply. Some of the costs are already on the record:.
In a support inbox, the cost is quieter but adds up: a refund you didn't owe, or a customer who goes looking for a live chat button that isn't there and doesn't come back.
Northwind Outdoor sells four things: two packs, a rain cover, and a tent. It has two orders on file and three short policies. Here's all of it:.
Six emails come in, and each one is there to trip the bot in a different way:.
Recent findings suggest that the traps: the 40L pack doesn't come in green, but the 28L does. The 40L is water-resistant, not waterproof. There's no 60L. Whether the return is allowed depends on a membership, a sale price, and a count of days since delivery. The tracking details are in one order record. And the store has no price match policy at all.
Make a folder, add "type": "module" to its package.json, and install the Anthropic SDK and Zod:.
Every version calls the model through one small helper:.
We're using Claude Opus 5.5 at low effort. These are short, simple replies, so the model doesn't need to think for long. Swap in any model you like, because none of the fixes depend on it.
According to recent reports, opus 5.5 didn't invent a single price, colour, or delivery date across the six emails. Every reply said it couldn't see the store's details. That's real progress compared with older models, but look at what it said instead. Here's the reply to the email about a pack the store doesn't sell:.
Industry observers note that it treats the Ridgeline 60L as real and offers to help pick a fit. The other replies made up the store around the customer:.
Sources indicate that northwind has none of those. None of it is a wild guess: it's what a typical online store has. That's exactly why it's dangerous. A customer who goes looking for the live chat or the newsletter discount won't find it, and the bot never answered the question anyway.
It has been revealed that the model can't know Northwind's prices, so give them to it. This is the idea behind retrieval-augmented generation (RAG): find the records that matter and put them in the prompt. Our store is small enough to send all of it.
Two things changed. The store data goes in the system prompt, wrapped in tags so the model can tell data from instructions. And the prompt tells it to answer only from that data. Here's the green pack email again:.
Right on all three counts, and it even pointed to the pack that does come in green. The return, which needs four facts from two places, came back right as well:.
Delivered October 6, a Trail Club member, so 60 days, so December 5. Bought on sale, so store credit for what was paid, not the list price. Version 2 got all six emails right.
The last line of that prompt matters as much as the data: "If the answer isn't in the data, say you don't know and that a team member will reply by email." Without it, a model that can't find the answer has to choose between letting the customer down and guessing, and training has taught it to guess. With it, "I don't know" is a right answer, not a failure.
The price match email is the test, since the store data notes nothing about price matching:.
It remarked it didn't know, passed the question to a person, and answered the parts it could from the data.
In a recent development, if version 2 got everything right, why build a version 3? Because "right on this run" isn't the same as "right every time":.
The next three fixes take the facts and the rules away from the model completely.
Version 3 splits the work into three steps. First, the model reads the email, but instead of writing a reply, it fills in a form. Every field is a choice from a list, a yes or no, or text copied from the email:.
According to recent reports, zod describes the form, and z.toJSONSchema() turns it into a JSON schema for the API's structured outputs. The API holds the model to that schema, so product can only be one of our four product IDs, not_sold, or none. There's no way for it to answer ridgeline-60. Zod then checks the answer again, so the rest of the code gets a typed object. SitePoint's guide to LLM structured output with JSON Schema goes deeper on this pattern.
Here's what it read from the 60L email:.
The model is still reading natural language, which it's very good at. It just isn't trusted to know what the store sells anymore.
Recent findings suggest that A form also makes the model's mistakes easy to see. The first time I ran this, the form had no asks_something_else field, and the model filed the price match email under product. The reply didn't make anything up, but the email never reached a person. In version 2, a mistake like that would be hidden inside a paragraph. Here it was one wrong value in a JSON object, and the fix was one more field.
According to recent reports, if all you need from this step is decisions like these, a model built for typed answers can do it faster and for less. Our typed AI pipeline tutorial builds the same kind of email sorter with Jev and Clairic.
Second step: plain code takes the form, looks up the product and the order, applies the rules, and does the maths. What comes out is a list of facts in plain sentences:.
For the return email, that gives:.
Recent findings suggest that every number in there was worked out by code that does the same thing every time, and that you can unit test. The 34 days, the December 5 deadline, and the $151.20 in store credit come out the same on every run, whatever the model does.
It has been revealed that the last rule in decide() matters most. If an email asks something the rules don't cover, the facts say so and the email goes to a person. Code decides when to say "I don't know", not the model.
Third step: the model writes the reply from the facts and nothing else. Writing is the one job here it does better than code.
It has been revealed that even with the right facts, a model writing a reply can still add something: a helpful "ships in 1β2 business days", or a price it remembers from somewhere. So the last step checks the reply. Anything that looks like a number, an order number, or a product name has to appear in the email or the facts:.
According to recent reports, it's simple on purpose. It doesn't understand the reply. It just refuses to let through a value that came from nowhere, and that covers the hallucinations that cost money: prices, dates, quantities, and products. Nothing in the reply can talk it out of a result.
The writer runs the check on every draft:.
If a draft fails, the model gets one retry and is told exactly which values weren't in the facts. If the retry fails too, the customer gets the facts as a plain list. That's less charming, but it's true.
According to recent reports, A check like this will raise false alarms, and that's fine. On my first run, it read a line about the tent's 2.1 kg weight as a product called "Weighs 2" and sent the draft back for a retry. The weight was right, and it was the only thing flagged. A false alarm costs a few seconds. A missed one costs a customer. I tightened the patterns (the version above has the fix), but I'd still rather it flag too much than too little.
It's the same thing that separates the better agentic AI coding tools from autocomplete: they run the tests and read the errors instead of trusting their first draft. Model output isn't finished until something other than the model has checked it.
This script sends each email through all three versions, saves every reply, and runs the check from Fix 5 over all of them. Versions 1 and 2 are checked against everything the store knows:.
According to recent reports, the check caught one thing in version 1: the "1β2 business days" it made up for processing an order. It missed the "up to 24 hours" for tracking in the same reply, but only because $24 is the rain cover's price, so 24 counted as a known number. And it couldn't catch the invented returns page or live chat at all, because they aren't numbers. That's why the check is the last fix, not the only one.
Reading all 18 replies, here's how each version did:.
In a recent development, version 3 was less helpful than version 2 in one place. It couldn't give the rain cover's price, because decide() only adds facts about the product the email names. Version 2 had the whole store to look at. In version 3, the fix is a rule you write once: when a product has an add-on, include the add-on's facts too.
Here's version 3's reply to the price match email:.
According to recent reports, no model is free of hallucinations, so you can't stop one from ever making something up. What you can do is build so a made-up detail can't reach a customer: let code own the facts, and check the model's output against them before it's used. Where it matters, send anything your rules don't cover to a person.
It cuts them a lot, because the model no longer has to guess facts it was never given. In our test, putting the store's data in the prompt took the bot from no useful answers to six right ones. It doesn't stop the model misreading what it retrieved, mixing up two records, or filling a gap the retrieved text leaves. Pair it with a way out and a check on the output.
Not really. Temperature controls how random the wording is. A low temperature makes the model give the same answer more often, including the same wrong answer.
Collect inputs that have tripped your app (or are likely to), write down the right answer for each, and run every version of your prompt or pipeline over the same set. Automated checks like the one in this tutorial catch made-up values. Read the replies yourself for the rest, because the made-up returns page above would pass any number check.
It has been revealed that hallucinations come from asking a model to do jobs it isn't built for: knowing facts it was never given, and running small programs in its head. The five fixes take those jobs away:.
It has been revealed that the model still does what it's best at: reading messy text and writing a friendly reply. It just isn't trusted with the facts anymore. From here, try a seventh email that asks two things at once, add the add-on rule to decide(), or extend the check to catch colours and sizes as well as numbers.
Recent findings suggest that 7 Day Free Trial. Cancel Anytime.
