Research · October 8, 2026
How consistent are AI brand mentions? A study of 60 buying questions
60 Dutch-language buying questions without a brand name, each asked 8 times on the same day through the APIs of ChatGPT, Gemini, and Perplexity. Plus 20 questions that we asked 20 times each.
Matt Timmermans, Timmermans Media. Measured: September 26, 2026. Preregistration: doi.org/10.5281/zenodo.22974863. Data: doi.org/10.5281/zenodo.23010967.
AI brand mentions only partly repeat: ask a chatbot exactly the same buying question twice and you see partly different brands. Of all brands that appeared across two answers, on average 47% appeared in both at ChatGPT, 52% at Gemini, and 38% at Perplexity. The brand named first was also a different one in about half of the cases.
So one answer, or a screenshot of it, is not enough to tell how often a brand gets mentioned. The same goes for three runs: for about a quarter of brands per question on average, an estimate from three answers deviated strongly from an estimate from twenty. The results apply to the APIs on September 26, 2026; the apps were not studied.
The findings
Brand mentions across repeated questions: four figures
47%, 52%, and 38%:
the average share of brands from two answers to the same buying question that appeared in both, at ChatGPT, Gemini, and Perplexity.
About half:
in 50 to 52% of answer pairs, the same brand was named first.
19 to 26%:
the share of brands per question for which an estimate from three runs was usually more than 20 percentage points off an estimate from twenty.
17 to 27%:
the brand overlap between two different chatbots on the same question, compared with 39 to 44% between runs of the same chatbot.
Repetition
Does ChatGPT mention the same brands every time you ask?
No, only partly. When the same buying question was asked again, the average overlap in brands mentioned was 47% at ChatGPT, 52% at Gemini, and 38% at Perplexity: somewhere between a bit less than half and a bit more than half.
What overlap means here: the number of brands that appear in both answers, divided by the number of distinct brands across the two answers combined. For example, answer 1 namesMoneybird, e-Boekhouden.nl, and Exact (Dutch accounting software brands), and answer 2 names Moneybird, Exact, and Jortt. Two brands appear in both, four brands in total. The overlap is then 2 of 4, or 50%. So an overlap of 50% does not mean that half the brands from the first answer disappear.
| Chatbot | Average overlap | 95% interval | Average brands per answer | Usable buying questions |
|---|---|---|---|---|
| ChatGPT | 47% | 43 to 50% | 2.2 | 52 of 60 |
| Gemini | 52% | 49 to 55% | 7.1 | 58 of 60 |
| Perplexity | 38% | 34 to 42% | 4.0 | 56 of 60 |
60 buying questions, 8 runs per question, all 28 pairs of runs per question. Weighted average across 20 categories. All 20 categories are represented for all three chatbots. A question for which a chatbot named no brand in any run does not count for that chatbot. A pair in which one answer names a brand and the other names none counts as 0% overlap. Leave those pairs out and the overlap is 48% at ChatGPT, 53% at Gemini, and 43% at Perplexity (supplementary, not preregistered).
In this measurement, Gemini has the highest overlap and also names the most brands per answer; Perplexity has the lowest overlap. We did not test the chatbots against each other on this measure. The number of brands per answer also differs sharply (2.2 versus 7.1), and that affects how high the overlap can get.
Mentioned or recommended. This study counts mentions. An answer that recommends a brand and an answer that advises against the same brand count as a match. For recommended brands only, the overlap is nearly the same: 46% at ChatGPT, 51% at Gemini, and 36% at Perplexity. Whether a brand was recommended was determined by a language model.
Because all runs were carried out on the same day and in shuffled order, the differences are not caused by a model update between two measurement days.
Chart 1. Average overlap in brands mentioned between two answers to the same buying question, with 95% interval: ChatGPT 46.7% (43.3 to 50.1%, 52 of 60 questions usable), Gemini 52.0% (49.0 to 54.8%, 58 of 60), and Perplexity 37.7% (34.1 to 41.7%, 56 of 60). The line at 70% is the research threshold we set in advance, not a norm; at 100%, both answers name exactly the same brands. 60 buying questions, 8 runs per chatbot, measured September 26, 2026.
First mention
How often do chatbots name the same brand first?
About half the time. When the same buying question was repeated, ChatGPT, Gemini, and Perplexity named the same brand first in 50 to 52% of answer pairs.
| Chatbot | Same brand named first | 95% interval | Pairs not counted (one of the two without a brand) |
|---|---|---|---|
| ChatGPT | 52% | 46 to 58% | 16% |
| Gemini | 52% | 47 to 57% | 6% |
| Perplexity | 50% | 43 to 56% | 21% |
Same 60 buying questions and 8 runs. Only pairs in which both answers name at least one brand.
This is an average across questions. For some questions the chatbots almost always lead with the same name; for others it changes a lot. Whether the first-named brand is also the recommended brand is not something this measure captures.
Sample size
How accurately can you measure brand visibility with three answers?
Only roughly. For 19 to 26% of brands per question, an estimate from three runs was usually more than 20 percentage points off an estimate from twenty runs. This study does not give a fixed minimum number of runs.
Brand visibility here is the share of runs in which a brand is mentioned. If a brand is mentioned in 12 of 20 answers, its brand visibility is 60%.
Example: Batavus at Perplexity
Perplexity named the Dutch bike brand Batavus in 8 of 20 answers to the question about the best e-bike, a brand visibility of 40%. A measurement with three runs then comes out at 0, 33, 67, or 100%, depending on which three answers you happen to get. In 612 of the 1,140 possible combinations (54%), that measurement was more than 20 percentage points away from the 40%.
These are combinations of the same twenty answers, not 1,140 separately run measurements. Any brand that appears in 8 of 20 answers produces exactly this distribution. So the example shows what happens with a small sample, not that Perplexity is unusually erratic.
Chart 2. In 612 of the 1,140 measurements (54%), the result lands more than 20 percentage points away from the 40%. Any brand that appears in 8 of 20 runs produces exactly this distribution.
How often do three answers deviate strongly?
For 20 buying questions, one per category, we asked each chatbot the same question 20 times. We then calculated brand visibility for all 1,140 possible combinations of three out of those twenty runs, and checked per brand how often an estimate from three runs deviated more than 20 percentage points from the estimate from twenty.
| Chatbot | Brands for which three runs usually deviate more than 20 percentage points | Range across questions |
|---|---|---|
| ChatGPT | 26% | 0 to 67% |
| Gemini | 24% | 0 to 43% |
| Perplexity | 19% | 0 to 45% |
Average per question, across 19 buying questions per chatbot; for one question per chatbot, no brand at all was named in twenty runs. “Usually” means: in more than half of the 1,140 combinations.
The estimate from twenty runs is an estimate too, not a fixed value. It is, however, based on many more answers.
What this means for our other publications. In our studies on Reddit, Google rankings, Google traffic, and llms.txt, we asked each question three times per chatbot. Those publications answer different questions and combine data from many questions or websites. This analysis does not prove that three runs are enough for all of those outcomes; the uncertainty has to be assessed per outcome. Those publications report intervals with their headline figures, and figures per category and individual examples are labeled as exploratory.
Between chatbots
Do ChatGPT, Gemini, and Perplexity name the same brands?
Less than one chatbot agrees with itself. For the same question, the overlap between two different chatbots was smaller than between runs of the same chatbot. That held for all three pairs.
| Pair | Overlap between the chatbots | Overlap within the chatbots, same questions | Difference (95% interval) | Usable questions |
|---|---|---|---|---|
| ChatGPT and Gemini | 17% | 44% | -28 percentage points (-32 to -23) | 64 |
| ChatGPT and Perplexity | 27% | 39% | -12 percentage points (-15 to -9) | 60 |
| Gemini and Perplexity | 24% | 42% | -17 percentage points (-20 to -15) | 68 |
Broad subset of 100 questions (3 buying questions and 2 informational questions per category), 8 runs per chatbot. Only questions for which both chatbots name at least one brand. Buying questions and informational questions weighted equally. Differences were calculated on unrounded values. The overlap within the chatbots is, per question, the average of the overlap between runs of each of the two chatbots, over the same questions as the previous column and with buying and informational questions weighted equally; that is why it differs from 47%, 52%, and 38%, which cover buying questions only. Buying questions only: -30, -19, and -19 percentage points.
ChatGPT and Gemini are the least alike: two answers from those two chatbots to the same question have, on average, 17% of their brands in common.
How many brands are named by only one chatbot?
In the baseline measurement, with three runs per chatbot, 70% of brand and buying question combinations turned up at only one chatbot. 11% were named by all three.
| Found at | Share of all brand and question combinations | Average per question |
|---|---|---|
| 1 chatbot | 70% | 11.3 brands |
| 2 chatbots | 19% | 3.1 brands |
| All 3 chatbots | 11% | 1.7 brands |
All 229 buying questions, three runs per chatbot pooled. Counted per question. Of the combinations that appeared at only one chatbot, 65% came from Gemini, 27% from Perplexity, and 8% from ChatGPT (supplementary, not preregistered).
Gemini names far more brands per answer than ChatGPT (7.1 versus 2.2) and supplies most of the combinations that only one chatbot names (65%). In the subset of 60 buying questions with 8 runs per chatbot, 70% of combinations also appeared at only one chatbot (supplementary, not preregistered). Because both the number of runs and the questions differ there, that is not a clean test of the effect of the number of runs.
For SEO specialists
What does this mean for AI visibility tracking?
That one answer is not enough to track AI visibility or determine how often a brand is mentioned. A brand missing from a screenshot may well be named the next time the question is asked, and vice versa. This study does not give a fixed number of runs, but it does give a basis for measuring and reporting differently.
| Task | How this study applies |
|---|---|
| Reporting | With every percentage, state how many times the question was asked, and preferably give an interval. A percentage from one or three answers can differ sharply from an estimate based on many more answers. |
| Evaluating tools and agencies | Ask how many runs a brand visibility measurement uses, and per chatbot. A report without that number cannot be evaluated. |
| Competitor analysis | Measure per chatbot. ChatGPT, Gemini, and Perplexity named largely different brands for the same question. |
| Client conversations | Treat a single screenshot from a client (“ChatGPT doesn’t mention us”) as one observation, not as a conclusion. |
| Interpreting results | Distinguish between mentioned, mentioned first, and recommended. This study mainly measures the first. |
| Monitoring | Do not draw conclusions about the effect of your work from a single higher or lower result. Repeat the same questions under comparable conditions and report the uncertainty. |
This applies to Dutch-language buying questions through the APIs, measured on a single day. How brand mentions shift over weeks or months is something we did not measure.
Which websites these chatbots use as sources is covered in our study on Reddit as an AI source and our study on Google rankings and AI sources. Whether websites with little Google traffic get cited is covered in our study on the Google traffic of AI sources, and what llms.txt has to do with it in our study on llms.txt.
Deep dive
Deep dive: differences by category and at product level
The parts below are exploratory. They round out the picture, but the main conclusion does not rest on them.
Consistency by category
The overlap varies widely by category. For some questions the chatbots keep naming largely the same brands; for others it changes much more.
| Category | Overlap ChatGPT | Overlap Gemini | Overlap Perplexity |
|---|---|---|---|
| Software and SaaS | 79% | 69% | 54% |
| Pets | 72% | 55% | 74% |
| Food and drink | 54% | 67% | 62% |
| Accounting and bookkeeping | 54% | 63% | 53% |
| Education and courses | 26% | 41% | 28% |
| HR and staffing | 25% | 33% | 9% |
| Legal | 14% | 24% | 21% |
Selection of notable categories. Only 3 buying questions per category, fewer for some chatbots. No intervals per category, so these are rough estimates.
In this selection, the highest values are for software and pets. The lowest values differ per chatbot: legal at ChatGPT and Gemini, HR and staffing at Perplexity. With 3 questions per category, that is an indication, not an established difference between industries.
Overlap at product level
Look at products instead of brands, such as Nike Pegasus instead of Nike, and the overlap is lower: 30% at ChatGPT, 32% at Gemini, and 22% at Perplexity.
Informational questions
For informational questions, the chatbots name few brands. There the overlap is 31% at ChatGPT, 36% at Gemini, and 21% at Perplexity, but on too few questions (13 to 27) for a firm conclusion.
Methodology
Methodology: measuring brand mentions across repeated questions
Preregistration and hypotheses
We recorded the research questions, hypotheses, measurement settings, and analysis rules and deposited them publicly on Zenodo before the measurement started (DOI 10.5281/zenodo.22974863).
Amendments, hypotheses, measurement design, brand recognition, calculations, and limitations
After the deposit, two rules were changed on September 26, 2026, both before any analyses had been run: retrieving a second page of Google results for 88 search terms (amendment 1) and the classification of websites for which Ahrefs gives no rank but does give traffic (amendment 2). This publication uses no Google or Ahrefs data, so the amendments do not affect these figures. Both are listed in the amendment log in the data repository.
| Hypothesis | Content | Type | Outcome or estimate |
|---|---|---|---|
| H4a | Overlap in brands mentioned between runs below 70% (buying questions) | Primary | Confirmed for ChatGPT, Gemini, and Perplexity |
| H4a-bis | Same measure for recommended brands | Secondary | 46% (41 to 50) / 51% (48 to 55) / 36% (32 to 39) |
| H4b | First-named brand identical in fewer than 70% of pairs | Secondary | 52% / 52% / 50% |
| H4c | For at least 25% of brands, three runs deviate more than 20 percentage points from twenty | Secondary | 26% / 24% / 19% (descriptive, no interval) |
| H11 | Overlap between chatbots lower than within chatbots | Secondary | -28 / -12 / -17 percentage points (ChatGPT and Gemini, ChatGPT and Perplexity, Gemini and Perplexity) |
Order of the estimates: ChatGPT, Gemini, Perplexity.
We chose the 70% threshold for H4a in advance: at 70% overlap, 30% of all distinct brands from an answer pair appear in only one of the two answers. That threshold is a choice, not a general norm. All three chatbots fall clearly below it; the hypothesis is confirmed with a corrected p-value of 0.0015 for all three chatbots.
Only the primary hypothesis gets an outcome label. We report secondary hypotheses as estimates, with an interval where available. Exploratory or supplementary: the figures per category, at product level, and for informational questions, and the figures marked as such in the text.
Measurement design
| Component | Details |
|---|---|
| Questions | 400 Dutch-language questions without a brand name in 20 categories. For this publication: a broad subset of 100 questions (3 buying questions and 2 informational questions per category) and a deep subset of 20 buying questions (1 per category) |
| Runs | Broad subset 8 times per chatbot, deep subset 20 times per chatbot, each time in a new session. Baseline measurement: all 400 questions 3 times per chatbot |
| Chatbots | ChatGPT (gpt-6-sol, OpenAI Responses API with web_search, location Netherlands), Gemini (gemini-3.8-flash, Gemini API with Google Search), Perplexity (sonar, Agent API with web_search, location Netherlands) |
| Settings | No system prompt, default settings |
| Order | All questions, chatbots, and runs in one randomly shuffled list, executed on September 26, 2026. Exception: 54 Gemini answers that failed because of an empty credit balance were rerun the same day outside that order. Without those 54, the main conclusion stays the same: the overlap at Gemini is then 52% (49 to 56%), still confirmed |
| Total | 6,120 answers across the full study |
Recognizing brands
The brands in all answers were recognized automatically by the language model Claude Sonnet 5, using a fixed instruction. A brand is the name under which a product or service is offered and chosen, such as Moneybird orCoolblue. Government agencies and quality labels do not count as brands. Product names are reduced to the brand: iPhone counts as Apple.
Before the measurement, we compared that recognition with a second model, Claude Opus 5.5, on 150 test answers. The agreement (F1 score, a measure from 0 to 1) was 0.95 for ChatGPT, 0.94 for Gemini, and 0.95 for Perplexity. That is the agreement between two models, not the share of correctly recognized brands. An automated text check then verified whether the name as it appeared in the answer, so before reduction to the brand (iPhone, not Apple), occurs literally in the answer text; in the main measurement that check removed at most 1.3% of the recognized names.
We also ran the recognition twice on 100 test answers. The two rounds had 98% overlap. So the recognition varies far less than the chatbot answers do. This test does not give a hard upper bound for the impact of recognition errors on the overlap figures.
How we calculate
Per question, we average the overlap across all pairs of runs. A pair in which neither answer names a brand is left out: that applied to 13% of pairs at ChatGPT, 4% at Gemini, and 10% at Perplexity. A pair in which one answer names a brand and the other names none counts as 0% overlap.
We then average per category and give every category the same weight. The intervals come from a bootstrap: we resample the questions within each category 2,000 times and look at how much the outcome fluctuates. Because we test the hypothesis for three chatbots, we correct the p-values with the Holm method.
Limitations of the repeated measurements
- API, not the apps. The apps may behave differently, for example because of memory or personal settings.
- Cause of the differences not measured. We measure how much answers differ, not how much of that comes from the way the model generates text, from changing search results, or from other processes.
- Brand recognition by a language model. The check was done with a second model from the same maker, not by humans. Shared errors cannot be ruled out.
- Role not checked by humans. Whether a brand is recommended or merely mentioned was determined by the model. That is why “mentioned” is the main measure.
- Only buying questions in the main test. Informational questions yield too few brands.
- One measurement day. Variation over weeks or months is not covered here.
- Model versions. The model names carry no date. A silent update by the provider would not be visible in the data.
Open data
Data and citation for the brand mentions study
The preregistration (10.5281/zenodo.22974863) and the data repository (10.5281/zenodo.23010967) are on Zenodo. The answers, brand extractions, and analysis scripts are in the data repository. Three supplementary analyses were calculated for this page: the overlap without pairs that include one empty answer, the share per chatbot of the brands that only one chatbot names, and the same count with 8 runs per chatbot. Those are in the data repository as well.
The data are licensed under CC BY 4.0, the analysis scripts under MIT.
Preregistration and data on Zenodo
Cite this study
Timmermans, M. (2026). How consistent are AI brand mentions? A study of 60 buying questions. Timmermans Media. Preregistration: doi.org/10.5281/zenodo.22974863. Data: doi.org/10.5281/zenodo.23010967. License CC BY 4.0.
This study is available under the CC BY 4.0 license.
Frequently asked questions
Questions about brand mentions in ChatGPT, Gemini, and Perplexity
Why does ChatGPT give different answers to the same question?
ChatGPT, Gemini, and Perplexity are language models that put their answer together from scratch every time, even for exactly the same question, so the brands they mention can differ too. With search turned on, the search results can also differ from one request to the next. This study measures how large the differences are, not how much of them comes from which cause.
How many times should I ask a question to measure brand visibility?
This study did not establish a fixed minimum. It does show that three runs, averaged per question, produced a strongly deviating estimate for about a quarter of brands. The more often you measure, the smaller the uncertainty.
My brand is not in an answer. Does that mean I am not being mentioned?
One answer does not tell you that. Of all brands from two answers to the same question, on average 38 to 52% appeared in both. A brand that is missing from one answer can show up in the next.
Does this also apply when I ask the question in the app?
We did not study that. We measured through the APIs with default settings. In the apps, memory, earlier conversations, and personal settings can play a role.
Are your other studies reliable, then, with three runs?
Those publications answer different questions and combine data from many questions or websites. This study does not prove that three runs are enough for all of those outcomes. Those publications report intervals with their headline figures, and figures per category and individual examples are labeled as exploratory.
Can you measure this for my own brand?
Yes. Using the same measurement design, we can measure for your industry how often and how consistently your brand is mentioned in ChatGPT, Gemini, and Perplexity. See our GEO service.
Related research
Research · same measurement
Does ChatGPT use Reddit as a source? What 3,600 AI answers show
The same measurement, focused on Reddit and on sources found versus sources cited.
Research · same measurement
Pages in Google's top 3 are rarely cited by ChatGPT
The same 400 questions, set against the Google results from the same day.
Research · same measurement
Websites with little Google traffic get cited by AI too
The same 400 questions, with the estimated Google traffic of every cited website.
Research · same measurement
Most websites that ChatGPT and Gemini cite have no llms.txt that we could find
The same measurement: 3,048 cited and non-cited websites checked for an llms.txt file.
This page is the English edition of a study first published in Dutch. Read the original Dutch version.
Glossary
Dutch terms on this page
The study used Dutch questions and Dutch websites. These are the Dutch names and words that come up.
- Batavus
- Dutch bicycle and e-bike brand, one of the best known in the country.
- Coolblue
- Dutch online electronics and appliance retailer, comparable to Best Buy.
- e-Boekhouden.nl
- Dutch online bookkeeping software for small businesses; boekhouden means “bookkeeping.”
- Exact
- Dutch business and accounting software company for small and midsize firms.
- Jortt
- Dutch online accounting software for freelancers and small businesses.
- Moneybird
- Dutch online invoicing and accounting software for small businesses, comparable to QuickBooks.
Does your website show up as a source in AI?
Measure your visibility in AI answers
For your industry and region, we measure whether your website appears as a source in ChatGPT, Gemini, and Perplexity, and which websites the chatbots cite instead.