Research Note 001: NIXUMA Research Note
We asked AI about Ginza.
Change one condition, and the recommendations changed.
On November 1, 2026, we used the APIs for OpenAI (ChatGPT), Google Gemini and Perplexity to ask repeatedly for restaurant recommendations in Ginza. We counted how often 5 preselected restaurants (whose names were not in the questions) appeared in the answers. Here is the result first. One of the five tracked restaurants appeared in 12 of 30 answers to the vegetarian question, but in only 1 of 29 answers to the general “dinner in Ginza” question. The same restaurant can appear or not appear depending on how the question is asked.
Test date: November 1, 2026 / Published: November 2, 2026 / Location: Ginza, Tokyo / This article withholds the names of the tracked restaurants.
Design
How the test was set up
We split how guests search into three scenarios: “dinner in Ginza,” “anniversary dinner,” and “dinner for someone who does not eat meat or fish.” We wrote each scenario in English and Japanese, and asked each of the 3 AIs 5 times. We planned 90 tests, received 89 answers, and 85 of those completed in full (1 failed, 4 were cut off).
| Item | Detail |
|---|---|
| Date and location | November 1, 2026 / Ginza, Tokyo |
| AIs tested | OpenAI (ChatGPT), Google Gemini, Perplexity, all via API |
| Scenarios | 3: dinner / anniversary / vegetarian (someone who does not eat meat or fish) |
| Languages | English and Japanese |
| Repetitions | 5 per scenario × language × AI (90 planned, 89 answers received) |
| Tracked restaurants | 5 preselected restaurants in Ginza. Their names were not in the question text. |
| What we counted | How often the 5 restaurants appeared in the recommendations. We did not judge which restaurant is “best.” |
Result 01
Change the scenario, and the result changes
For one of the five tracked restaurants, appearances by scenario were as follows.
| How the guest asked | Appearances | Share of answers |
|---|---|---|
| Dinner in Ginza | 1 of 29 | 3.4% |
| Anniversary dinner | 3 of 30 | 10.0% |
| Vegetarian dinner | 12 of 30 | 40.0% |
This restaurant appeared in 40% of the vegetarian answers. Some answers recommended it while mentioning its vegetarian and vegan options. But this test cannot separate why this result happened: whether the restaurant's official site, third-party articles, knowledge the AI already held, or search results drove it. We changed none of the restaurant's information during the test, so we cannot name one source as the cause.
What we can say is this much. Asking one general question, “recommend a restaurant?”, cannot tell you whether that restaurant appears in AI answers. If you do not measure by guest scenario, you cannot see the actual position.
Result 02
Change the language, and the result changes
Across the 3 AIs, that same restaurant appeared in 10 of 45 Japanese answers and 6 of 44 English answers.
The gap does not always point the same way, though. In Gemini's vegetarian answers, this restaurant appeared 5 of 5 times in Japanese and 1 of 5 times in English. With the other AIs the gap was smaller, and across the whole test we cannot say “Japanese is always easier to appear in.” Five repetitions per cell are not enough to conclude which language has the advantage.
The practical conclusion is still clear. Measure each language separately. Do not treat English results as a stand-in for the answers Japanese-speaking guests see. Local guests and visitors to Japan have to be measured separately, or the gap stays hidden.
Result 03
The counting side makes errors too
Our first automated count missed 8 real recommendations. They were in the AI answers, but our counting method did not catch them. We also recorded one restaurant's ranking position incorrectly, in 1 case.
We audited the collected data separately, found this error, and corrected it before publishing. After the correction, one restaurant's total rose from 10 appearances to 16 appearances. With the first numbers, we would have shown that restaurant's presence as smaller than it was, and distorted comparisons across languages and AIs.
The misses came from the many ways a restaurant name can be written: kanji, roman letters, or a name with a description attached. Numbered headings, bullet lists, and formats that follow no fixed pattern. And a name can appear without being a recommendation, in a caution or a comparison, for example. Automated counting alone is not enough. Human checking is part of the measurement.
Caution
Appearing in recommendations and being described correctly are different questions
This test did not check whether descriptions of the restaurants were correct. An AI can recommend a restaurant for vegetarians and still get the menu details wrong. It can give booking methods that have changed, or confuse the restaurant with another with a similar name.
One more point. The amount of search trace left in the record differed by AI. For some answers we could confirm a search had run. For others, the record did not let us judge whether a search had run. And even when a source is shown next to an answer, that source does not necessarily support everything in the answer. Counting “does it appear” and checking “is it described correctly” have to stay separate jobs.
For Owners
What this means for your restaurant
Guests do not ask in one way. Anniversaries, business dining, dietary restrictions, whether online booking is possible. Change how they ask, and the answer the AI assembles changes too.
So the measurement has to match. Ask repeatedly, by scenario and by language. Do not judge “appears / does not appear” from one answer to a general question. Our restaurant checks use 29 questions and ask several AIs repeatedly because of the variation this test showed.
The base on the restaurant side is to write scenario-specific information accurately: whether you handle vegetarian requests, whether the restaurant suits anniversaries, how guests book, opening hours and closing days. If that is written accurately in the public information AIs read, at least the material is in place. This test did not check whether fixing information increases appearances, or whether more appearances increase bookings. Both need separate tests.
Limits
Limits of this study, and what comes next
This study ran for one day only, covered 5 restaurants and 3 AIs, and measured via API, not via the apps. Behaviour in the apps may differ. The results are exploratory. They do not rank Ginza restaurants.
Two things we want to test next. First, whether the information AIs state about a restaurant (opening hours, prices, booking methods) is correct. Second, whether appearances in recommendations change over time when a restaurant's verified information is fixed. We will also improve how we count restaurant names in answers that mix Japanese and English.
Has your restaurant been measured by scenario?
In NIXUMA's AI Visibility Check, we ask several AIs repeatedly, using wording guests actually use, and report whether your restaurant was named, which restaurants were named instead, and whether anything stated did not match the facts. The Full Audit ($59), which measures in more depth, uses 29 questions and measures in both Japanese and English.