ChatGPT API vs App: How Far Apart Are the Brands They Recommend — Canlah AI Data Report

The ChatGPT API and the ChatGPT app name brands at noticeably different rates. In a pre-registered Canlah AI test of 30 Singapore buyer questions on October 15 and 16, 2026, how often gpt-5.5 through the API named each frequently named brand differed from the ChatGPT app by 23.2 percentage points on average (root-mean-square, 90% interval 18.7 to 27.1). The test required a gap under 10 points before the API could count as the same as the app, so it could not be confirmed as a stand-in.

ChatGPT Plus and signed-out answers were further apart still, by 32.5 points beyond normal run-to-run noise, although the two also ran different models at different times of day. Asking a question 3 to 5 times in the app predicted what another account saw no worse than the API’s three answers, within a pre-set 5-point margin.

About this research: Canlah AI makes a Chrome extension, now in early access, that measures AI answers inside the user’s own ChatGPT, so it has an interest in this result. The questions, decision rules and analysis code were frozen and hashed before the first test question was sent, and every change after that point was logged.

Key findings

• The API and the app named brands at different rates: for the brands named most often, the API and the app differed by 23.2 points on average (90% interval 18.7 to 27.1). The gap was 16.8 points for GEO agencies, 25.1 for kids’ coding schools and 26.6 for dental clinics.
• ChatGPT Plus and signed-out answers differed most: they were 32.5 points apart beyond the noise inside each account (90% interval 28.9 to 35.8), and their top five brands overlapped on 3.7 names. The two accounts also ran different models at different times of day, so the gap is not the account alone.
• A few app runs were no worse than the API: asking 3 times in the app predicted another account’s brand rates with an error 2.2 points above the API’s for Plus (90% upper bound 4.6) and 0.2 points above for signed-out (upper bound 2.5), inside the pre-set 5-point margin. In an arm added later, gpt-6.1-sol did 5.1 to 9.7 points worse than the app.
• The regular names need 4 to 5 runs: for brands named in at least 20% of answers, 4 to 5 app runs were enough to expect 95% of them to appear on a typical question. Pinning one brand’s rate to a sampling error of 5 points took 35 to 58 runs of the same question.
• The app adds things the API never returns: 90.4% of Plus answers carried a map or business card, and 35.4% of signed-out answers (220 of 622) carried an ad. Ads were cut out and only brands in the answer text were counted.

Study design: 30 Singapore buyer questions asked in three ChatGPT accounts and through the API

The test asked 30 questions: 15 buyer needs, each in English and in Chinese, split evenly across three Singapore industries. Every question named Singapore and asked for a list of recommendations, for example “best Invisalign clinic in Singapore? Please answer directly with a recommended list.”

The three industries and five buyer needs per industry

Industry	The five buyer needs asked about
GEO agencies	best overall; recommended by ChatGPT with a reason; for small and medium businesses; for B2B tech; English and Chinese
Dental and orthodontic clinics	best Invisalign clinic; braces with a reason; for children and teenagers; affordable Invisalign; braces or Invisalign in Tampines
Kids’ coding schools	best overall; classes worth signing up for; for beginners aged 7 to 10; Python for teenagers; schools in Bukit Timah

Answers came from four sources, all from one Singapore connection. Throughout, “the app” means ChatGPT as people use it at chatgpt.com.

The four sources:

• ChatGPT Plus: a paid account in a temporary chat with personalisation off. The app labelled the model gpt-6-thinking. 515 answers were collected; the main comparison uses 353 answers from 12 complete rounds, where a round counts as complete at 27 or more of the 30 questions.
• Signed out: two browser profiles that had never signed in; the test could not read a model name. 622 answers were collected; the main comparison uses 495 answers from 17 complete rounds.
• ChatGPT Free: a free account. It hit a usage limit after 43 questions in one hour and then switched to a smaller model. Of its 84 answers, 37 failed to load, and it finished no complete round, so it is described but not compared.
• The API: the OpenAI Responses API with gpt-5.5, web search forced on and the user location set to Singapore. The plan called for 6 answers per question; three waves were dropped, one under the pre-set cost rule and two when budget moved to a second model, leaving 3 per question (90 in total). That second arm, added after the rules were frozen, asked gpt-6.1-sol 5 times per question (150 answers). API fees for the whole test came to $43.29.

A “brand” is an organisation offered as an option in the answer text, after spelling variants were merged. Ads were cut out, and map and business cards were left out of the brand counts, so the app and the API were compared on the answer text alone.

Results: how far the API, Plus and signed-out answers were from each other

The main measure is the root-mean-square gap in brand rates. For each brand named often, take the share of answers that name it in one source and in the other, then take the root-mean-square of those differences, corrected for the gap that random sampling alone would produce. A gap of 0 means both sources name the same brands equally often.

Brand-rate gaps between ChatGPT sources, 30 Singapore questions, October 15 to 16, 2026

Comparison	Gap in brand rates	90% interval	Pre-registered verdict
API (gpt-5.5) vs the app	23.2 points	18.7 to 27.1	Unclear: above the 10-point ceiling for “the same”
ChatGPT Plus vs signed out	32.4 points	28.7 to 35.8	Different: 32.5 points beyond within-account noise
Signed-out profile A vs profile B	2.2 points	0.0 to 8.6	Unclear: 6.9 points beyond noise, interval −8.5 to 12.4
Each account split in half, average	5.1 points	0.0 to 11.0	The noise floor inside one account

At 23.2 points, the API was too far from the app to pass the test for “the same”. It could not be called “different” either, because the two app accounts were further from each other, 32.4 points, than the API was from the app. The pre-registered verdict was therefore “unclear”.

API vs app gap by industry

Industry	API vs app gap	Signed-out answers with a map, counted afterwards
GEO agencies	16.8 points	10.8% (18 of 167)
Dental and orthodontic clinics	26.6 points	43.5% (70 of 161)
Kids’ coding schools	25.1 points	22.2% (37 of 167)
All 30 questions	23.2 points	25.3% (125 of 495)

ChatGPT Plus showed a map or business card in 90.4% of its answers (319 of 353). The signed-out map counts come from a recount of the saved pages after the main analysis, because the extractor did not detect that map. Dental questions showed the widest API gap and the most maps in the signed-out app, so a clinic that checks only API answers misses both.

Accuracy: what a few app runs get right

The accuracy test asked a practical question: if a business asks a question a few times in its own ChatGPT, how well does that predict what other ChatGPT users see? The “truth” was the brand rates seen by the other account, so neither side was graded against its own answers. The API side had 3 answers per question.

Method	Error vs another account’s brand rates	Difference vs the API
ChatGPT Plus, asked 3 times	26.4 points	+2.2 (90% upper bound 4.6)
ChatGPT Plus, asked 5 times	24.8 points	+0.6 (upper bound 3.1)
Signed out, asked 3 times	28.9 points	+0.2 (upper bound 2.5)
Signed out, asked 5 times	27.6 points	−1.2 (upper bound 1.3)
API gpt-5.5, 3 answers	24.2 to 28.8 points	—
API gpt-6.1-sol, 5 answers, added later	31.5 to 37.3 points	the app was 5.1 to 9.7 points better

Both the 3-run and the 5-run comparisons passed the pre-set “no worse than the API” margin of 5 points after Holm correction (p = 0.023 and p = 0.002). The 5-run result held in all three industries; the 3-run result held in two, and for kids’ coding Plus asked 3 times was 5.6 points worse than the API. The errors on both sides are large, 24 to 37 points, because another account genuinely sees a different list.

How many runs a question needs

For brands named in at least 20% of answers, a typical question needed 5 Plus runs for GEO and 4 for dental and coding before 95% of those brands were expected to appear; signed-out answers needed 4 to 5. Reaching 95% sample coverage of all brands, a measure weighted towards the brands named most often, took 6 to 27 runs on a typical question and up to 58 on the hardest one. Repeating a question 5 times in one chat gave answers 1.35 times as similar as fresh chats on Plus, below the pre-set 1.5 threshold for “repeating itself”, so the result was unclear and fresh chats remain the safer default. On Plus, across 6 buyer needs, rewording a question explained 15.4% of the variation in how often the regular brands appeared (90% interval 5.9 to 24.9).

What the October 2026 ChatGPT account test does and does not show

The test shows that, for Singapore buyer questions in three industries, brand rates from the API differed from brand rates in the ChatGPT app by 23.2 points on average, and that Plus and signed-out answers differed by more. It shows that a handful of app runs per question estimated what another account saw no worse than three API calls, within a 5-point margin.

It does not separate the account from the model or the time of day. Plus ran gpt-6-thinking, the signed-out model could not be read and the API ran gpt-5.5. Most signed-out rounds ran on the morning of October 15 and most Plus rounds that afternoon and evening, with the last Plus round on October 16, so each gap mixes account, model, interface and time. It covers two days, one Singapore connection, three industries and ChatGPT only. The findings are associations measured on this sample, not causes.

Implications for anyone tracking a brand in ChatGPT

1. Ask which layer a number comes from. A visibility score built only from API calls describes the API. In this test the API differed from the app by 23.2 points on average.
2. Name the account type. A signed-in Plus user and a signed-out visitor saw shortlists that overlapped on 3.7 of their top five brands. A tracking report should say whose view it shows.
3. Ask 4 to 5 times, in fresh chats, before reading the regular names. One answer is an anecdote. Pinning a rate down to a sampling error of 5 points takes dozens of runs.
4. Keep the answer text. Every rate should open to the answers behind it, so a buyer can check what ChatGPT said.

Canlah AI’s Chrome extension, Canlah GEO, takes the app-side approach: it asks the user’s buyer questions in their own ChatGPT, in a temporary chat with personalisation off when signed in or as a signed-out visitor when not, and shows the first percentage only after 10 answers are collected.

See what ChatGPT says about your brand: join the Canlah GEO early-access list →

Frequently asked questions

Does the ChatGPT API give the same answers as the ChatGPT app?

Not the same brand rates. In Canlah AI’s October 2026 test of 30 Singapore buyer questions, how often gpt-5.5 through the API named each frequently named brand differed from the ChatGPT app by 23.2 percentage points on average, more than the 10-point ceiling the test set for “the same”. The app also shows map cards, business cards and, for signed-out visitors, ads, which the API does not return.

Do ChatGPT Plus and signed-out users see the same brands?

No. ChatGPT Plus and signed-out answers to the same 30 questions differed by 32.5 points beyond normal run-to-run noise, and their top five brands overlapped on 3.7 names on average. The two accounts also ran different models at different times of day, so the gap reflects account, model and timing together.

How many times should a question be asked to track a brand in ChatGPT?

Four to five times, in fresh chats, to see the brands named regularly: in the test, 4 to 5 app runs were enough to expect 95% of the brands named in at least 20% of answers to appear on a typical question. Pinning one brand’s rate to a sampling error of 5 points took 35 to 58 runs of the same question.

Methodology and limitations

Answers were collected on October 15, 2026 (Singapore time) through one Singapore connection, and the twelfth Plus round was completed on October 16, 2026. The question set, schedules, decision rules and analysis code were frozen at 03:49 Singapore time on October 15, before the first test question was sent; a short pilot ran before the freeze and is excluded. Each file’s SHA-256 hash was recorded, and every later change is in a numbered deviation log. Each question was asked in a new chat unless the module tested repeated questions in one chat. The app accounts ran in separate browser profiles, and the Plus account’s temporary chat and personalisation setting were checked at every send.

Brands were extracted from the answer body by a rule-based extractor; low-confidence answers were reviewed blind by a separate AI reviewer, and spelling variants were merged through an alias table. A blind check on a 10% sample scored the extractor at F1 0.97 for API answers, 0.97 for Plus and 1.00 for signed-out answers, above the 0.95 bar; the Free account scored 0.89. Free was left out of the comparisons because it finished no complete round. Intervals are 90% intervals from 10,000 bootstrap resamples clustered by buyer need, so the English and Chinese versions of a question count as one cluster. The pre-registered consistency test and the accuracy tests at 3 and 5 runs were corrected together with the Holm method; the Plus vs signed-out comparison and the gpt-6.1-sol arm were outside that family.

Limits: two days, one Singapore connection, three industries and 30 questions; ChatGPT only, with no Gemini or Google AI Overviews; each account ran a different model at a different time of day; the Free account produced too few usable answers to compare. Three of six planned API waves were dropped, as were a planned evening re-run and a deliberate rate-limit ramp, which stopped when the Plus account was forcibly signed out. The gaps are measured on the brands named most often in these answers and may differ for other industries or markets.

• Canlah AI ChatGPT three-account matrix archive, October 15 to 16, 2026: pre-registration and hashes, deviation log, raw answers and page captures, API call records, brand extraction and quality checks, analysis output.

Related articles

• The AI visibility measurement gap
• GEO measurement methods
• Which ChatGPT plan to use for optimising a website
• How to check if ChatGPT recommends your brand
• Which industries benefit most from GEO
• Singapore clinic website benchmarks 2026