NIXUMA
NIXUMA
All research notes

NIXUMA Research Note 003

When sources disagree, which version does AI repeat?

· Measured: November 8, 2026|Location: Tokyo (Nihonbashi, Asakusa, Ginza, Meguro, Jimbocho)|Fixed question texts were used

If a shop's own official website and a major listing disagree about its opening hours or its regular closing day, which version does AI give as the answer? We took six long-established restaurants in Tokyo, chose three cases where the sources disagree and three where they agree, and asked four AI systems, ChatGPT, Gemini, Perplexity and Claude, about just one fact per shop, in Japanese and in English, three times each. The results first. For the shops where the sources agreed, not one of the 72 answers conflicted with the official information. For the shops where the sources disagreed, 7 of the 72 answers conflicted with the official one: 5 gave the disagreeing listing's version exactly as it stood, and 2 gave a third value that appears in neither source. Another 7 settled on neither version, presenting the disagreement itself. And of the 7 answers that conflicted, 5 were concentrated in a single AI system (Claude). In the case of one Ginza café, all three of Claude's English answers failed to match the official information, and two of them gave a third answer, open every day, that appears in neither the official source nor the listing.

Design|How the measurement was designed

The subjects are six restaurants in Tokyo with long operating histories. For one fact that each shop's official website publishes (a regular closing day, or a closing time), we chose three shops where a major listing disagrees with the official source (the disagreement cases) and three shops where every listing we checked agrees with it (the agreement cases). For each shop we prepared a fixed question text asking about that fact, in Japanese and in English, and asked the four AI systems three times each. The total is 144 answers.

Item Detail
Date and location November 8, 2026 / Tokyo (Nihonbashi, Asakusa, Ginza, Meguro, Jimbocho)
AI systems ChatGPT (OpenAI), Gemini (Google), Perplexity, Claude (Anthropic), all via API (models and search settings in the table below)
Subjects Six restaurants: three cases where sources disagree, three where they agree
Fact One per shop: a regular closing day, or a closing time
Languages Japanese and English
Repetitions Three per shop, language and AI system; 144 answers in total
Classification Each answer was classified by whether, on that fact, it agreed with the official information, conflicted with it, or gave no single value for it. We made no judgment as to which of the two versions is correct

Models and search settings

All answers were collected through APIs, not through the consumer apps. Search was enabled for every system. The three repetitions of each question were separate API calls, each with a fresh context; no system instructions were set.

AI system Model Search configuration Output cap
ChatGPT (OpenAI) gpt-5-mini Responses API, web_search tool (the model decides when to search) 6000 tokens
Gemini (Google) gemini-3.8-flash AI Studio generateContent, google_search grounding 2000 tokens (raised from 1200 at the lab transport)
Perplexity perplexity/sonar Agent API, web_search tool 2000 tokens (raised from 1200 at the lab transport)
Claude (Anthropic) claude-haiku-4-5 Messages API, web_search tool (version web_search_20250305, max 3 uses) 2000 tokens (raised from 1500 at the lab transport)

A performed search is recorded for 142 of the 144 answers. The two exceptions are both Claude in Japanese on the Luncheon control case (records RN3-K2-anthropic-ja-r0 and RN3-K2-anthropic-ja-r2), where no search was performed and no sources were returned; those two answers are the declined answers counted in Result 01. Every one of the 144 records shows a single successful attempt; no retry is recorded in the run files. Where this note discusses the Claude pattern, it describes this tested configuration, claude-haiku-4-5 with web search, and not Claude in every form.

The six shops, and what the disagreement consists of

Shop Fact Official information The listing that disagrees
Taimeiken (Nihonbashi) Regular closing day Closed on Mondays TripAdvisor lists Monday as an opening day
Yoshikami (Asakusa) Closing time 21:30 Hot Pepper Gourmet lists 22:00
Café de L'Ambre (Ginza) Regular closing day Closed on Mondays Hot Pepper Gourmet gives Tuesday as the closing day, and TripAdvisor also lists Monday as an opening day
Tonkatsu Tonki (Meguro) Regular closing day Closed on Tuesdays None (every listing checked agrees)
Luncheon (Jimbocho) Saturday closing time 20:30 None (every listing checked agrees)
Kamiya Bar (Asakusa) Regular closing day Closed on Tuesdays None (every listing checked agrees)

The evidence behind each benchmark is not all of the same kind, and the difference matters for one case. The official benchmark is explicit for five of the six shops: Taimeiken states its Monday closure directly, Yoshikami states its 21:30 closing directly, and all three agreement cases state their fact directly. For Café de L'Ambre the Monday closure is inferred: the official site publishes hours for Tuesday to Friday, Saturday and Sunday and lists no Monday hours, while Tabelog's shop-maintained entry states the Monday closure explicitly. Two further points are worth noting. Taimeiken's TripAdvisor record is an old listing in which the pre-relocation address remains; the disagreement itself still exists today, but an unmaintained listing is what stands behind it as a source. And Yoshikami's disagreeing listing describes its 22:00 closing as a change effective from July 1, so a conflicting answer there may reflect newer information than the official site, not an error. These points are also written in the Limits section.

The question texts

Shop Japanese English
Taimeiken 日本橋のたいめいけんは月曜日に営業していますか? Is Taimeiken in Nihonbashi open on Mondays?
Yoshikami 浅草のヨシカミは何時に閉店しますか? What time does Yoshikami in Asakusa close?
Café de L'Ambre 銀座のカフェ・ド・ランブルの定休日は何曜日ですか? What is Café de L'Ambre in Ginza's regular closing day?
Tonkatsu Tonki 目黒のとんかつ とんきの定休日は何曜日ですか? What is Tonkatsu Tonki in Meguro's regular closing day?
Luncheon 神保町のランチョンは土曜日何時に閉店しますか? What time does Luncheon in Jimbocho close on Saturdays?
Kamiya Bar 浅草の神谷バーは火曜日に営業していますか? Is Kamiya Bar in Asakusa open on Tuesdays?

How the answers were classified

Classification followed the primary value each answer asserted for the fact. An answer whose primary asserted value was the official value was counted as agreeing, even when it also touched on other information, and that extra information was left in the record. An answer whose primary asserted value differed from the official value was counted as conflicting. An answer that asserted no single value was counted as giving no single value. That last group has three subtypes, and the counts matter: of the 9 answers in it, 6 presented both versions without settling on one, 1 gave a range spanning both values, and 2 declined to answer or answered without the fact (the two no-search Claude answers named above). Across the disagreement cases alone, the 7 answers in this group split as 6 presenting both versions and 1 giving a range. The original text of all 144 answers is stored.

Result 01|Where the sources agreed, conflicting answers numbered zero

Across the three agreement cases (Tonkatsu Tonki, Luncheon, Kamiya Bar), of 72 answers, 70 agreed with the official information and 0 conflicted with it. No AI system asserted a value that differs from the official information.

The remaining 2 answers gave no single value, and both were Claude answers in Japanese, of the declined subtype: asked about Luncheon's Saturday closing time, Claude declined to answer without searching in two of the three repetitions. Tonkatsu Tonki and Kamiya Bar agreed with the official information in all 24 of their answers.

Result 02|Where the sources disagreed, about one in ten conflicted with the official version, and about one in ten settled neither way

Across the three disagreement cases, of 72 answers, 58 (81%) agreed with the official information, 7 (about one in ten) conflicted with it, and 7 (about one in ten) gave no single value. The 7 conflicting answers are not all the same kind: 5 matched the disagreeing listing's value, and 2 gave a third value that appears in neither the official source nor the disagreeing listing (both are the Café de L'Ambre "open every day" answers in Result 04). Of the 7 that gave no single value, 6 presented both versions without settling and 1 gave a range spanning both values.

Overall, of 144 answers, 128 (89%) agreed with the official information, 7 conflicted, and 9 gave no single value. The conflicts appeared only where the sources themselves disagree.

A sensitivity check belongs here. The Café de L'Ambre benchmark is the inferred one (see Design). If that case is set aside, the disagreement cases total 48 answers, of which 3 conflicted with the official information, and all 3 matched the disagreeing listing's value. The third-value pattern appears only in the inferred-benchmark case.

Result 03|Looked at shop by shop, the disagreement was answered differently at each shop

Taimeiken (of 24 answers: 21 agreed, 0 conflicted, 3 gave no single value). No AI system repeated TripAdvisor's "open on Mondays" information as its own answer. The 3 answers that gave no single value were all Claude in English: they set Monday closure and Monday opening side by side and recommended confirming by phone with the shop. Claude's Japanese answers on the same shop, by contrast, agreed with the official information all three times.

"there are conflicting reports about whether Taimeiken in Nihonbashi is open on Mondays: Some sources indicate the restaurant is closed on Mondays. Other sources show it's open Monday 11:00 AM - 9:00 PM. Given these conflicting reports, I recommend calling the restaurant directly [...] to confirm their Monday hours before planning your visit." (Claude, English, record RN3-C1-anthropic-en-r0)

Yoshikami (of 24 answers: 18 agreed, 3 conflicted, 3 gave no single value). The listing side's information, a 22:00 closing time, was the primary asserted value once each in ChatGPT in Japanese, Perplexity in Japanese and Claude in English; all 3 conflicting answers at this shop matched the disagreeing listing's value. The clearest case was ChatGPT's Japanese answer, which brought a local association listing's 22:00 to the front and treated the official website's 21:30 notation as outdated. The remaining 3 answers either named both 21:30 and 22:00, or answered with a range spanning both.

In the original Japanese, the answer leads with the Asakusa culinary association listing "11:30-22:00 (L.O. 21:30)" as the recent information, and says of the official site, in translation: "the official site's information includes pages with a somewhat outdated notation (for example 11:30-21:30 / L.O. 21:00)". (ChatGPT, Japanese, record RN3-C2-openai-ja-r2)

Café de L'Ambre (of 24 answers: 19 agreed, 4 conflicted, 1 gave no single value). All 5 answers that failed to match the official information were Claude's. ChatGPT, Gemini and Perplexity agreed with the official information in all 18 of their answers at this shop. Claude in Japanese split one each across agreed, conflicted and no single value. Claude in English is dealt with in the next Result.

Counting the disagreement cases only, system by system (18 answers per system):

AI system Agreed with the official information Conflicted with it No single value given
Gemini 18 0 0
Perplexity 17 1 0
ChatGPT 15 1 2
Claude 8 5 5

Gemini's primary asserted value was the official value in every one of its disagreement-case answers. Perplexity and ChatGPT each conflicted once. In the tested Claude Haiku configuration, Claude accounts for 5 of the 7 conflicting answers and 5 of the 7 answers that gave no single value.

The way this configuration missed differed in kind from the other systems. Rather than quietly repeating one side's information, it more often brought the disagreement itself to the surface: either setting both versions side by side and prompting the reader to check with the shop, or giving a third piece of information that appears to have been treated as the most recent. At Café de L'Ambre, its English answers failed to match the official information all three times: once it gave the listing side's "closed on Tuesdays" exactly as it stood, and twice it reached "open every day", a conclusion found in neither the official source nor the listings.

"Café de L'Ambre in Ginza is closed on Tuesday. This is their regular closing day." (Claude, English, record RN3-C3-anthropic-en-r1)

"another more recent source states the café is open every day. The most recent information appears to indicate that the café is now open every day" (Claude, English, record RN3-C3-anthropic-en-r0)

The totals for each system across all six cases are: ChatGPT 33 agreed, 1 conflicted, 2 gave no single value; Gemini 36, 0, 0; Perplexity 35, 1, 0; Claude 24, 5, 7 (36 answers each).

Result 05|The answers split by language in Claude too, and the three repetitions did not always line up

The difference between Japanese and English was concentrated in Claude. At Taimeiken, Claude in Japanese agreed with the official information all three times, while Claude in English set both versions side by side all three times. At Café de L'Ambre, Claude in English failed to match the official information all three times, while Claude in Japanese split one each across the three labels. No language difference on this scale appeared in the other three systems.

We record the combinations whose three repetitions split, stated as ranges. At Yoshikami, Perplexity in Japanese ran 2 agreed and 1 conflicted; Yoshikami in Claude in English, and Café de L'Ambre in Claude in Japanese, each ran one each of agreed, conflicted and no single value; Luncheon in Claude in Japanese ran 1 agreed and 2 with no single value. Most combinations came out with the same classification all three times, but Café de L'Ambre in Claude in English alone lined up on the conflicting side all three times, doing so across two different values.

Result 06|Answers added information absent from the official page, even where the fact asked about matched

All 24 answers at Tonkatsu Tonki, an agreement case, agreed on Tuesday closure. Of these, 23 also mentioned closure on the third Monday, a detail absent from the official page.

That an answer agrees with the official information on the fact it was asked about does not mean the answer was built from the official information alone. Answers come back with detail that only the listings carry layered on top of the official fact.

Caution|What this measurement counted is agreement with the official information

The benchmark used here is the official information the shop itself publishes. "Conflicts" is therefore not a ruling that the listing is wrong. The listing side may be the newer one. In the Yoshikami case, Hot Pepper Gourmet presents the 22:00 closing as a change effective from July 1, and the possibility remains that it is the official website that has not caught up.

Whichever side is used as the benchmark, though, the measurement result does not change: the answer a customer receives from AI can differ from the shop's own published information. Answers that set both versions side by side were classified as giving no single value. They can be useful as a note of caution for the user, but for a user who wants to settle where to go on the strength of the AI answer alone, they are also answers that lack the deciding element.

For Owners|For shop owners

  • First, check that the opening hours and regular closing days on your official website and the major listings (Tabelog, Hot Pepper Gourmet, TripAdvisor and others) line up. In this measurement, at the shops where the sources agreed, not a single answer conflicted with the official information.
  • While a disagreement remains, AI may give the disagreeing side's information exactly as it stands, and occasionally a value found in neither source. In this measurement the conflicting answers were concentrated in one AI system, but that system was not the only one to conflict.
  • Check also whether an old listing is still standing. In the Taimeiken case, an old pre-relocation record was the origin of the "open on Mondays" information.
  • The way the systems responded to the disagreement differed from system to system. An answer that sets both versions side by side and urges the reader to check with the shop leaves the question it was asked unsettled, and a third answer found in neither source can guide a customer to the shop on a day it is closed. This measurement did not measure how customers behave after receiving such answers.
  • That said, this measurement does not confirm that aligning the listings changes AI answers. What can be said is that the disagreements in the answers appeared only where the sources themselves disagreed.

How is your shop's information being answered by AI?

NIXUMA's Quick Scan checks the public sources about your shop, and whether they agree with your official information, fact by fact. Asking the AI systems themselves, repeatedly and in the phrasings customers actually use, is the Full Audit.

Limits|The limits of this study

This measurement recorded six shops in Tokyo, one fact per shop, on a single day, November 8, 2026. The facts covered are limited to regular closing days and closing times. On disagreements about prices or payment methods, this study's case selection found no case in which both sides could be confirmed in documents. The measurement was taken through APIs rather than in the apps, and behavior in the apps may differ. The models used were gpt-5-mini, gemini-3.8-flash, perplexity/sonar and claude-haiku-4-5. The results are exploratory, and are not a ranking of shops or of AI systems.

The individual circumstances are also recorded. For Taimeiken, the TripAdvisor record that is the source of the disagreement is an old listing from before the relocation. For Café de L'Ambre, the official website indicates the regular closing day by not listing hours for Monday, and the explicit statement of the Monday closure rests on Tabelog's shop information. Classification followed the primary value each answer asserted, and answers that touched on other information are left in the record. Two further cautions apply to the reading of the results. First, an answer that matches a listing's value does not establish that the listing caused the answer; the measurement records agreement between values, not the path an answer took. Second, the sources and citations each system returned are recorded separately from the classification and are not, by themselves, the classification.

Source references (checked November 7, 2026; answers collected November 8, 2026)

  • Taimeiken: official site https://www.taimeiken.co.jp/ ; disagreeing listing, TripAdvisor https://www.tripadvisor.jp/Restaurant_Review-g14129589-d1224086-Reviews-Taimeiken-Nihonbashi_Chuo_Tokyo_Tokyo_Prefecture_Kanto.html
  • Yoshikami: official site https://yoshikami.co.jp/info01.html ; disagreeing listing, Hot Pepper Gourmet https://www.hotpepper.jp/strJ000103317/
  • Café de L'Ambre: official site http://www.cafedelambre.com/ ; disagreeing listings, Hot Pepper Gourmet https://www.hotpepper.jp/strJ000103185/ and TripAdvisor https://www.tripadvisor.jp/Restaurant_Review-g14129573-d1689274-Reviews-Cafe_de_Lambre-Ginza_Chuo_Tokyo_Tokyo_Prefecture_Kanto.html ; explicit Monday closure, Tabelog https://tabelog.com/tokyo/A1301/A130103/13002564/dtlrvwlst/B386826327/
  • Tonkatsu Tonki: official site https://tonkatsutonki-megurohonten.com
  • Luncheon: official site https://www.luncheon.jp/access.html
  • Kamiya Bar: official site http://www.kamiya-bar.com/

Appendix|Classification of all 144 answers

Each cell is the breakdown of the three repetitions (agreed / conflicted / gave no single value).

Taimeiken (Nihonbashi)|Regular closing day; the official information is closed on Mondays (24 answers: 21 agreed, 0 conflicted, 3 gave no single value)

AI system Japanese English Total (6 answers)
ChatGPT 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0
Gemini 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0
Perplexity 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0
Claude 3 / 0 / 0 0 / 0 / 3 3 / 0 / 3

Yoshikami (Asakusa)|Closing time; the official information is 21:30 (24 answers: 18 agreed, 3 conflicted, 3 gave no single value)

AI system Japanese English Total (6 answers)
ChatGPT 1 / 1 / 1 2 / 0 / 1 3 / 1 / 2
Gemini 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0
Perplexity 2 / 1 / 0 3 / 0 / 0 5 / 1 / 0
Claude 3 / 0 / 0 1 / 1 / 1 4 / 1 / 1

Café de L'Ambre (Ginza)|Regular closing day; the official information is closed on Mondays (24 answers: 19 agreed, 4 conflicted, 1 gave no single value)

AI system Japanese English Total (6 answers)
ChatGPT 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0
Gemini 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0
Perplexity 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0
Claude 1 / 1 / 1 0 / 3 / 0 1 / 4 / 1

Tonkatsu Tonki (Meguro)|Regular closing day; the official information is closed on Tuesdays (24 answers: 24 agreed, 0 conflicted, 0 gave no single value)

AI system Japanese English Total (6 answers)
ChatGPT 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0
Gemini 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0
Perplexity 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0
Claude 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0

Luncheon (Jimbocho)|Saturday closing time; the official information is 20:30 (24 answers: 22 agreed, 0 conflicted, 2 gave no single value)

AI system Japanese English Total (6 answers)
ChatGPT 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0
Gemini 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0
Perplexity 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0
Claude 1 / 0 / 2 3 / 0 / 0 4 / 0 / 2

Kamiya Bar (Asakusa)|Regular closing day; the official information is closed on Tuesdays (24 answers: 24 agreed, 0 conflicted, 0 gave no single value)

AI system Japanese English Total (6 answers)
ChatGPT 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0
Gemini 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0
Perplexity 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0
Claude 3 / 0 / 0 3 / 0 / 0 6 / 0 / 0