AI rarely invents facts, but it sells yesterday's prices

The core question

How often does AI give a wrong answer about Dutch brands?

Perplexity gives a wrong majority answer on 9.2% of customer questions, ChatGPT on 2.7% and Gemini on 1.1%, measured across three identical runs per question on 5 and 6 July 2026.

The gap between the models is large: a factor 8 between the worst and the best. The confidence intervals of Perplexity (5.8 to 14.3%) and Gemini (0.3 to 3.9%) do not overlap, so that difference is not down to chance. For a business owner it means the odds of a wrong answer about their brand depend heavily on which assistant the customer happens to use.

That 9.2% is a strict figure, too. A question only counts as wrong here when at least two of the three runs were wrong, so that incidental outliers drop out and only the structural error picture remains. Across all 1,917 assessed answers the conclusion is the same: a wrong answer is the exception rather than the rule, and those exceptions concentrate at one of the three models.

The nature of the errors

What goes wrong: does AI invent facts, or does AI lag behind?

AI rarely invents facts: 62% of all wrong answers contained outdated information and only 11% was genuinely fabricated.

The distinction between those two kinds of error is fundamental, and in this study we keep them strictly apart instead of lumping them together as hallucination. Outdated means the answer was once correct but has since been overtaken, for example a price that was right last year. Fabricated means the answer was never correct and rests on nothing. They have a different cause and a different fix: outdated information is solved by updating your own sources, a fabrication is not.

Time-sensitive facts such as prices, rates and subscriptions go wrong on 6.8% of questions; stable facts such as address, year founded or core activity on 2.5%. And within those time-sensitive errors, 88% is outdated rather than invented. So the error is almost always in what changes, not in what stays put.

That the models know the fixed facts well shows in a single number: across 106 questions about stable facts, Gemini made exactly zero errors. The core of this study fits in one sentence. AI rarely hallucinates; AI mostly serves yesterday's reality.

The model comparison

Which AI model is the most reliable?

Gemini was the most reliable model in this study with 1.1% wrong answers, followed by ChatGPT with 2.7% and Perplexity with 9.2% at question level.

The difference is not only in the volume of errors, but also in their kind. Genuine fabrications occurred only at Perplexity: 9 answers spread across 6 questions. For ChatGPT and Gemini the counter stood at zero. Almost all of Gemini's and ChatGPT's errors are outdated facts, not fantasy.

The error mix sketches the character of each model. At ChatGPT and Gemini nearly every error is a superseded fact: their knowledge is largely correct, but sometimes a step behind. At Perplexity a layer is added on top, because alongside outdated facts there are also a handful of fabrications and a single case of brand confusion, where information about one brand was attributed to another. The model with the most errors also makes the most kinds of error.

Reliability is more than scoring well on average; it is also giving the same answer to the same question. On 1 in 5 questions Perplexity did not reach a single verdict across three identical prompts, against roughly 1 in 12 for ChatGPT and Gemini. A model that shifts is harder for a brand to trust than a model that is consistent, even when that consistent model is occasionally consistently wrong.

For a brand that distinction is very practical. A model that answers the same way every time can be steered by adjusting the underlying source. A model that shifts gives your customer a correct answer one time and not the next, without you being able to see why. Consistency does not make an error less bad, but it does make it easier to repair.

In practice

What does this mean in practice? Four examples

In practice it almost never goes wrong on a fabrication, but on a price, a condition or a status that has not been updated online.

The four examples share the same core. In none of them does AI fantasise out of nothing; every time, the model repeats what is written somewhere online, even when that is no longer true or never was. So anyone who wants a grip on what AI says about their brand steers not on the model, but on the sources the model reads.

The sources

Where do the wrong answers come from?

In wrong answers from ChatGPT and Gemini the source profile shifts sharply towards comparison and affiliate sites (from 4 to 20% and from 23 to 40% of the cited sources); Perplexity, the model with the most errors, structurally leans hardest on third-party sources, with social media and forums as the largest block.

Across all three models combined, 32.8% of the cited sources in wrong answers come from comparison and affiliate sites, against 23.9% in correct answers; the share of the brand's own domain drops from 28.6 to 18.2%. One caveat belongs with that pooled figure: the error volume is unevenly spread across the models. Of all source citations in wrong answers, 64% comes from Perplexity, which already leans hardest on third parties in correct answers too, so the average is coloured by that one model.

Presence and share are not the same thing

Presence and share measure two different things. Presence is the question of whether the brand's own domain appears among the cited sources at all; share is how much of all cited sources comes from that brand domain. An answer can list the brand domain and still lean mostly on third parties.

Presence: share of answers in which the brand's own domain appears among the sources, run level.

Share per category (in % of source citations), correct and wrong, run level.

Which third parties show up in the wrong answers

Alongside the brand's own domain, the sources cited in wrong answers are mostly third parties. The domains below are the most frequently cited third-party domains per model, counted over the wrong runs.

Trap questions

Do the models fall for trap questions?

In 93.7% of cases the three models resisted a false premise about an invented product or service.

When we asked, for instance, about the precise terms of a discount or a service that does not exist, the models usually corrected that assumption instead of going along with it. Gemini did not fall for it a single time. Perplexity was the only model that both confirmed things that do not exist and denied services that do, which again shows that the risk works both ways.

That the models usually resist a false premise matters more than it seems. It means AI does not blindly adopt every suggestion inside a customer question, not even when that question insinuates a discount or a condition that does not exist. So the weakness is not in invented assumptions from the user, but once again in outdated facts from the source.

Beforehand we expected smaller brands to produce more errors than large ones. That pattern turned out to be absent. The opposite direction we saw at Perplexity, slightly more trouble with well-known brands than with small ones, was not statistically significant and therefore remains an exploratory observation, not a conclusion.

Even the researcher lagged behind

How current were the facts we judged AI against?

Twelve of the 185 verified facts turned out to be superseded again during the assessment, even though they had been checked by hand a few days earlier.

Moneybird scrapped its free plan as of 1 July, Knab lost its own Dutch banking licence at the end of November 2025 and HEMA stopped offering insurance. In several cases the searching models flagged these changes before our manual verification did. For a moment, AI was ahead of the researcher rather than the other way around.

For your business

What does this mean for your business?

Put time-sensitive facts such as prices, terms and current offers on your own up-to-date, crawlable page, because outdated third-party information is by far the largest source of error in AI answers about brands.

The three recommendations below follow directly from the data. They are not about blocking or embracing AI, but about something simpler: making sure the most current version of your facts is also the most findable version. If you do not, the outdated version wins, because it has often been sitting somewhere else for years.

Frequently asked questions

Questions about this study

Related

This page is the English edition of a study first published in Dutch. Read the original Dutch version.