Part four

Search and AI: the questions owners ask, answered from primary sources

Answers to the questions owners actually type, with the answer in the first two lines and the evidence underneath it. Where the industry quotes a number, this section prints what that number is a percentage of.

  • 68.01%

    Of US Google searches ended without a click, January to April 2026. The UK is 69.5 percent

    SparkToro and Similarweb 1

  • 48 / 20 / 18

    Three published figures for how often Google shows an AI answer. Three different denominators

    BrightEdge, SparkToro, Pew 2

  • 0.34%

    Of US searches went to AI Mode, against a billion monthly users. Both are true

    SparkToro, Google 1 , 3

  • 17%

    Of AI Overview citations also rank in the organic top ten, on one vendor's sample. Another says 38

    BrightEdge, Ahrefs 4

  • 15 Sep

    The date a Cloudflare setting many owners flipped starts blocking Googlebot

    Cloudflare, 1 July 2026 5

  • Google says it does not read llms.txt

    In its own documentation, not an off-hand remark: Google Search "doesn't use them." 6

  • The GEO 40 percent is from a machine the authors built

    GPT-3.5-turbo over the top five Google results, on a 10,000 query benchmark. Not a live search engine. 7

  • Search Console shows impressions and nothing else

    No clicks, no click-through rate, and AI Overviews and AI Mode cannot be separated. 8

  • Google is calling businesses, in three named categories

    Home repair, beauty and pet care, in the US. Restaurants are not named anywhere in the announcement. 9

  1. SparkToro with Similarweb panel data, 9 and 16 June 2026. 2nd
  2. BrightEdge 12 Feb 2026; SparkToro 9 Jun 2026; Pew Research Center 22 Jul 2025. 2nd/1st
  3. Google, 3 June 2026 and Alphabet Q2 2026 call, 22 July 2026. 1st
  4. BrightEdge, 12 February 2026; Ahrefs, relayed May 2026. 2nd
  5. Cloudflare, Your site, your rules , 1 July 2026. 1st
  6. Google Search Central, AI optimization guide, updated 10 July 2026. 1st
  7. Aggarwal et al., GEO: Generative Engine Optimization , KDD 2024. 1st
  8. Google, Search Console Help. 1st
  9. Google, I/O 2026, 19 May 2026. 1st

The question

How often does Google actually show an AI answer?

Between 18 and 20 percent of the searches a real person performs, and around 48 percent of the keywords an SEO tool tracks. Both figures are correct. They count different things, and almost nobody says which.

Source Figure Date Sample What the percentage is a share OF
BrightEdge 2nd 48% 12 Feb 2026 Size not disclosed anywhere. Nine verticals, measured daily Keywords in BrightEdge's own tracked set, which is what its enterprise clients pay it to track. Composition also undisclosed
Semrush 2nd 15.69% 15 Dec 2025 10 million-plus keywords in Semrush's database Keyword strings in a database, each counting once regardless of how often it is searched. Not weighted by volume
SparkToro and Similarweb 2nd 20%+ 9 Jun 2026 Similarweb US desktop and mobile panel, Jan to Apr 2026. Panel size not published Searches actually performed by real people. High-volume navigational searches dominate, and those rarely get an AI answer
Pew Research Center 1st 18% 22 Jul 2025 68,879 searches by 900 US adults, browser panel, data collected March 2025 All Google searches by the panel. The only figure here from a research institute rather than a vendor

Why the gap is not a contradiction

A keyword panel samples keyword strings . Each one counts once whether it is searched ten times a month or ten million, so a database leans heavily into the long tail, which is exactly where informational questions live and exactly where AI answers appear. A clickstream panel samples search events . Every time somebody types the name of their bank, or a restaurant, or "weather," that counts, and those searches almost never get an AI answer.

Which one to use, and when

If you want to know what your customers are seeing, use the clickstream or Pew figure, because those count human beings. If you want to know how many of the phrases you might target are affected, use the keyword-panel figure. Never quote either as "how often Google shows an AI answer" without saying which. The 48 percent is the number the industry repeats, and it is the one whose sample size has never been published.

  1. BrightEdge, AI Overviews at the One-Year Mark , 12 February 2026. Produced with BrightEdge's own tooling. 2nd
  2. Semrush, AI Overviews Study , 15 December 2025. 2nd
  3. SparkToro, In 2026, Less than One Third of Google Searches Still Send a Click , 9 June 2026. Data provider Similarweb. 2nd
  4. Pew Research Center, 22 July 2025. Data collected March 2025, so 17 months old at the time of writing. 1st

The question

Am I losing clicks to AI answers?

Every independent measurement says yes. Google says no. Google is the only party with the data and it publishes no numbers, so the disagreement cannot be settled from outside. Here are both sides, unreconciled.

What the outside measurements say

Finding Figure Base
US searches ending with no click, Jan to Apr 2026 68.01% Similarweb US panel
The same figure in 2024 60.45% Same panel
UK searches ending with no click 69.5% Same panel, UK cut
France 65.3% Same panel
Germany 62.1% Same panel
Clicked a result with an AI summary present 8% Of visits, Pew panel
Clicked a result with no AI summary 15% Of visits, Pew panel
Clicked a link inside the AI summary 1% Of visits, Pew panel

Ahrefs models the position-one effect at a 58 percent reduction in click-through rate when an AI Overview is present, on 300,000 keywords. That is a modelled counterfactual, not an experiment: it forecasts what the rate would have been and subtracts. The forecast is the load-bearing assumption and nobody can check it. 14 , 15 , 16

What Google says

"Overall, total organic click volume from Google Search to websites has been relatively stable year-over-year." And: "average click quality has increased and we're actually sending slightly more quality clicks to websites than a year ago."

Liz Reid, VP and Head of Google Search, 6 August 2025 17

No number appears anywhere in that post. Not a percentage, not a base, not a comparison figure. "Relatively stable" and "slightly more" are the whole of it. On the 22 July 2026 earnings call Google added that AI features send "billions of clicks to websites every week," again with no denominator. 18

  1. Pew Research Center, 22 July 2025. 68,879 searches by 900 US adults, 12,593 with an AI summary. Data collected March 2025. 1st
  2. SparkToro, 9 and 16 June 2026, Similarweb desktop and mobile panel, January to April 2026. Panel size not published. 2nd
  3. Ahrefs, AI Overviews Reduce Clicks , updated 4 February 2026. Supersedes their own 34.5 percent figure from April 2025, which is still circulating. 2nd
  4. Google, 6 August 2025. 1st
  5. Alphabet Q2 2026 call, 22 July 2026. 1st

The question

How does an AI answer decide who to quote?

Google has published a full explanation and it is the only company that has. OpenAI publishes one sentence. Perplexity and Anthropic publish nothing about selection at all. Every guide claiming to know how to get cited by those three is inference.

Google, in its own documentation

  1. 01

    Grounding

    "A technique (also known as grounding) used to improve the quality, accuracy, and freshness of AI responses by relying on our core Search ranking systems to retrieve relevant, up-to-date web pages from our Search index." 19

  2. 02

    Query fan-out

    "A set of concurrent, related queries generated by the model to request more information and fetch additional relevant search results to address the user's query." 19

  3. 03

    Then it reads the pages

    The system will "review the specific information from those retrieved pages to generate a more reliable and helpful response, showing prominent, clickable links to relevant web pages that support the information." 19

Google's own conclusion from that: "The best practices for SEO continue to be relevant because our generative AI features on Google Search are rooted in our core Search ranking and quality systems." To be quoted, a page "must be indexed and eligible to be shown in Google Search with a snippet." 19

Everybody else

Company What it publishes about selection
OpenAI "Ranking in ChatGPT Search is based on a number of factors designed to help users find reliable, relevant information." No list, no detail. It does state that a site blocking OAI-SearchBot "will not be shown in ChatGPT search answers" 20
Perplexity It gathers from "authoritative sources" and compiles "the most relevant insights." Neither term is defined anywhere. No ranking factors, no weighting, no method 21
Anthropic Documents when Claude searches and that citations are mandatory. Nothing at all about how sources are chosen or ranked 22
  1. Google Search Central, Guide to Optimizing for Generative AI Features on Google Search , last updated 10 July 2026. 1st
  2. OpenAI Help Center, ChatGPT Search , and Overview of OpenAI Crawlers . Neither carries an absolute date. 1st
  3. Perplexity Help Center, updated 1 May 2026. 1st
  4. Anthropic, Claude Platform Docs, web search tool. Undated. 1st
  5. arXiv preprint 2604.25707, 28 April 2026. 602 prompts, 21,143 citations, 18,151 pages. Not peer reviewed. 1st
  6. Search Engine Land, sponsored, 29 July 2026. 2nd

The question

Do I need an llms.txt file, or special AI schema?

Not for Google. Google states in its own documentation that Search does not use them. That is a written statement in a maintained document, not an off-hand remark, and it is the clearest answer anyone has published on the subject.

"You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them."

Google Search Central, AI optimization guide, updated 10 July 2026 25

"Structured data isn't required for generative AI search, and there's no special schema.org markup you need to add."

Google Search Central, same document 25

A Google search advocate put it less formally on a public forum in June 2026: llms.txt is "purely speculative for now (the file has existed for years, yet none of the AI systems use it)." We could not reach the original thread, so the documentation above is the citation that matters. 26

Read what that does and does not say

  • It covers Google Search only. It says nothing about ChatGPT, Perplexity or Claude. None of those three has published a position on llms.txt either way
  • "Not required" is not "useless." Structured data still drives ordinary rich results in search, which is a separate and real benefit
  • There is no special AI schema. Anyone quoting you for AI schema markup is selling a thing Google says does not exist
  • An llms.txt file costs nothing to keep. It should just never be sold as a ranking or AI-visibility measure

What Google does say gets you into AI answers

The same page names two things a local business can act on: a page "must be indexed and eligible to be shown in Google Search with a snippet," and Google recommends Business Profiles and Merchant Center to make "products and services visible in both AI responses and other Google Search results." That is the first-party link between a claimed Business Profile and AI answers, and it is the only one we found. 25

  1. Google Search Central, Google's Guide to Optimizing for Generative AI Features on Google Search , last updated 10 July 2026. The older page, AI features and your website , updated 10 December 2025, says the same thing. 1st
  2. John Mueller, Google, quoted by Search Engine Journal, 2 June 2026. The article does not link the original thread, so this is reported speech and is colour rather than the load-bearing citation. 2nd

The question

SEO, AEO, GEO, E-E-A-T. Which of these is real?

One is a defined discipline. One comes from an academic paper whose headline number was measured on a machine the authors built. One has no traceable origin at all. And E-E-A-T is not a score, a factor or anything you can be given.

Term Where it comes from Does the platform use the term What is actually true
SEO No primary, dated source for the coinage exists. Every account is a retelling Yes. Google defines it Google: "SEO, short for search engine optimization, is about helping search engines understand your content, and helping users find your site." 27
AEO No primary source found at all. No founding paper, no platform document, no standards body Once, to dismiss it. Google defines the acronym in its own guide and then says: "From Google Search's perspective, optimizing for generative AI search is optimizing for the search experience, and thus still SEO." 28
GEO A real, peer-reviewed academic paper: Aggarwal and others, KDD 2024, from Princeton and IIT Delhi No. Google names it only in the same dismissal The famous "up to 40 percent" was measured on a system the authors built: they fetched "top 5 sources from Google search engine for every query" and generated the answer with "gpt3.5-turbo," across a 10,000 query benchmark. Not Google, not AI Mode, not ChatGPT. The preprint is 33 months old and the model is obsolete 29
E-E-A-T Google's Search Quality Rater Guidelines, current edition 11 September 2025 Yes, and it defines each letter Experience, Expertise, Authoritativeness, Trust. Google's own line about the people who apply it: " No single rating can directly impact how a particular webpage, website, or result appears in Google Search , nor can it cause specific webpages, websites, or results to move up or down on the search results page." 30

The blunt version

GEO's documented tactics are "Cite Sources, Quotation Addition, Statistics Addition." Adding citations, quotations and statistics to your content is not a new discipline, it is writing something well evidenced. AEO has no origin and no platform recognition. E-E-A-T is a description of what Google is trying to build its systems to reward, not a lever you can pull. Anyone selling "E-E-A-T optimisation" is selling a description as a mechanism.

And what genuinely is different

Two things, and neither of them is an acronym. The unit of retrieval moved from the page to the passage , which is supported by Google's own grounding and fan-out documentation, by Anthropic's capped quote field and by independent measurement. And rank tracking no longer describes visibility , because most AI Overview citations are not page-one organic results. Two vendors put that overlap at 17 percent and 38 percent and disagree. Whichever is right, it is well under half. 31

  1. Google Search Central, SEO Starter Guide , updated 10 December 2025. 1st
  2. Google Search Central, AI optimization guide, 10 July 2026. 1st
  3. Aggarwal, Murahari, Rajpurohit, Kalyan, Narasimhan, Deshpande, GEO: Generative Engine Optimization , KDD 2024. Preprint 16 November 2023, revised 28 June 2024. GEO-bench is 10,000 queries. 1st
  4. Google, Search Quality Rater Guidelines , edition dated 11 September 2025. 1st
  5. BrightEdge, 12 February 2026, and Ahrefs, relayed May 2026. Both vendor samples, unresolved. 2nd

The question

What actually changed in Google Search in the last twelve months?

Seven confirmed updates, three of them core. AI Mode went from a US feature to over 200 regions and 35-plus languages. And in July 2026 Google merged AI Overviews and AI Mode into one experience, which it announced on an earnings call rather than in any documentation we could find.

Confirmed ranking updates, from Google's own status dashboard

Update Started Ran for
August 2025 spam update 26 Aug 2025 26 days
December 2025 core update 11 Dec 2025 18 days
February 2026 Discover update 5 Feb 2026 21 days
March 2026 spam update 24 Mar 2026 19 hours
March 2026 core update 27 Mar 2026 12 days
May 2026 core update 21 May 2026 11 days
June 2026 spam update 24 Jun 2026 2 days

No core update has been confirmed since 21 May 2026, which is eleven weeks at the time of writing. Every date and duration here is from Google's own Search Status Dashboard, which is the only authoritative record of what ran and when. 32

AI Mode, from launch to everywhere

  • 21 Aug 2025. "We're excited to expand AI Mode to over 180 new countries and territories in English" 33
  • 8 Sep 2025. Five more languages: Hindi, Indonesian, Japanese, Korean, Brazilian Portuguese 33
  • 7 Oct 2025. "More than 35 new languages and over 40 new countries and territories," taking it past 200 regions 33
  • 19 May 2026. Over a billion monthly users, with "queries more than doubling every quarter since launch." No base published for the doubling 34
  • 22 Jul 2026. "We recently brought together AI Overviews and AI Mode into one seamless Search experience" 35
  1. Google Search Status Dashboard, ranking history, retrieved 8 August 2026. Times US Pacific. 1st
  2. Google, The Keyword, 21 August, 8 September and 7 October 2025. 1st
  3. Google, I/O 2026 search updates and How AI Mode is changing the way people search , both 19 May 2026. 1st
  4. Sundar Pichai, Alphabet Q2 2026 earnings call and CEO remarks, 22 July 2026. 1st

The question

Can I see any of this in Search Console?

Partly. There is a Generative AI performance report, it shows impressions and nothing else, it cannot separate AI Overviews from AI Mode, and it is only rolled out to some site owners. So you can see whether you were shown. You cannot see whether it earned you anything.

What the report contains, in Google's words

  • "The generative AI performance report includes impressions for the following generative AI capabilities on Google Search: AI Overviews and AI Mode" 36
  • "Impressions are how many times links to your site were shown to a user in a generative AI feature on Google Search" 36
  • "We're rolling out this report to a subset of website owners, allowing for thorough testing before rolling it further" 36

The help documentation lists no clicks, no click-through rate and no average position for this report. The two AI features are reported together and cannot be split. The data is not additive either: it is a view of numbers already inside the Web search type of the main Performance report. 36

And there is now an off switch

Alongside the report, Google gave site owners a control to "decide if they want their site to appear in and help ground responses in our generative AI Search features." Sites that opt out receive no traffic from those features, and Google states the control does not affect ordinary rankings. The initial rollout was to a subset of website owners in the United Kingdom , so UK owners may see both the report and the control before US owners do. 37

  1. Google, Search Console Help, Generative AI performance report (Search) . The help page carries no publication date. Retrieved 8 August 2026. 1st
  2. Google, New opportunities, control and insights for website owners , 3 June 2026. 1st
  3. Google, 3 June 2026, and SparkToro with Similarweb, 9 June 2026. 1st/2nd

The question

How does Google decide which local business to show?

Google names three things: relevance, distance and prominence. It publishes them on a help page that carries no date anywhere, gives no weighting, and states plainly that you cannot pay for a better position.

The three factors, verbatim

  1. 01

    Relevance

    "Relevance is how well a Business Profile matches what someone is searching for." 39

  2. 02

    Distance

    "Distance refers to how far each business is from the customer who's searching." 39

  3. 03

    Prominence

    "Prominence means how well-known a business is. Prominent places are more likely to show up in search results." Google folds links, articles and directories into this, plus: "More reviews and positive ratings can help your business's local ranking." 39

And the line worth having ready when somebody offers to sell you a position: "There's no way to request or pay for a better local ranking on Google." 39

The rules that actually get a listing suspended

Google's guidelines for representing a business are the ones with teeth. The core instruction: "Represent your business as it's consistently represented and recognized in the real world across signage, stationery, and other branding." 40

  • Business name. No marketing taglines, store codes, trademark symbols, excessive capitalisation, hours, phone numbers or special characters, unless they appear on your signage
  • Address. No P.O. boxes. A virtual office must be staffed during the hours you state
  • Categories. "Use as few categories as possible to describe your overall core business"

Google says violations can lead to suspension. It publishes no severity ladder and no probability, so treat all of these as binary. 40

  1. Google Business Profile Help, Improve your local ranking on Google . No date on the page. Retrieved 8 August 2026. 1st
  2. Google Business Profile Help, Guidelines for representing your business on Google . No date on the page. Retrieved 8 August 2026. 1st

The question

What has anyone actually tested about local ranking?

A handful of things, by one firm, with samples ranging from three listings down to one. Here they are with their real sample sizes attached, because the sample is usually the first thing dropped when these get quoted.

What was tested Finding The actual sample What it does not tell you
Do Business Profile posts affect ranking? No effect 441 keywords per location, across 3 listings . One post a week for 9 weeks. Original test 2021, page maintained to Feb 2026 The firm that ran it explicitly reassigns posts to conversion, not ranking. It did not test click-through, profile conversion or AI answer visibility. "441 keywords" is per location, not 441 businesses 41
Do services listed on the profile affect ranking? Yes, they say Sample size not published. "Various clients in various industries," effect visible in 24 to 72 hours This reverses the same firm's own 2019 finding. Both stand. With no stated sample this is a practitioner observation, not a measurement 42
Does a keyword in the business name affect ranking? Dramatically One listing. A restaurant with no salad bar, no website and no other online mentions. Keyword added, removed, re-added A clean mechanism demonstration and a useless population estimate. It is also a direct breach of Google's business name guidelines and a suspension risk. Never do it 43
Does the service area you set affect ranking? No One listing plus an unquantified "several." Rankings follow the address used to verify the listing The only documented service-area effect is visual: a map overlay on desktop and mobile Maps 44
  1. Sterling Sky, Do Google Posts Impact Ranking? , published 26 July 2021, last modified 25 February 2026. 2nd
  2. Sterling Sky, Do Services in Google Business Profiles Impact Ranking? , 27 February 2026. 2nd
  3. Sterling Sky, Do Keywords in Business Names on Google Maps Impact Ranking? , 9 August 2023, last modified 17 November 2025. 2nd
  4. Sterling Sky, Does the Service Area in Your Google Business Profile Impact Ranking? , 31 March 2025. 2nd
  5. Whitespark, Local Search Ranking Factors , 2026 edition, published 6 November 2025. 47 practitioners scoring 187 factors. 2nd, opinion survey

The question

What are the rules on reviews now, and who enforces them?

Google's policy is one layer. Since 2024 and 2025 there is a second layer with real penalties behind it: the Federal Trade Commission in the US and the Competition and Markets Authority in the UK. Asking for reviews is fine. Paying for them, or filtering who you ask, is where it stops being fine.

What Google prohibits, verbatim

  • Merchants must not "Offer incentives, such as payment, discounts, free goods and/or services, in exchange for posting any review" 46
  • Prohibited: content "that is not based on a real experience or does not accurately represent the location or product in question" 46
  • Prohibited: content "posted from multiple accounts by or at the request of one person" 46
  • Prohibited: content "based on a conflict of interest," which "may include current or former employment, a contractual or consultory relationship, or other professional or personal affiliations" 46
  • Permitted: merchants may "Solicit or encourage the posting of content that does represent a genuine experience, without offering incentives" 46

We could not find a Google sentence prohibiting review gating by name, meaning asking only the customers you expect to be happy. The rule against that is regulatory, not Google's.

The two regulators

United States. The FTC's Rule on the Use of Consumer Reviews and Testimonials, 16 CFR Part 465, published in the Federal Register on 22 August 2024. Its sections are named plainly: fake or false reviews and testimonials, buying positive or negative reviews, insider reviews, company-controlled review websites, review suppression , and misuse of fake indicators of social media influence. Review suppression is the section that reaches gating. 47

United Kingdom. The CMA published dedicated fake reviews guidance, CMA208, on 4 April 2025, alongside its unfair commercial practices guidance CMA207, under the Digital Markets, Competition and Consumers Act 2024. The subject matter is fake reviews, concealed incentivised reviews, and false or misleading consumer review information. 48

  1. Google, Maps user contributed content policy. No date on the page. Retrieved 8 August 2026. 1st
  2. Electronic Code of Federal Regulations, Title 16 Part 465, Federal Register citation 89 FR 68077, 22 August 2024. Content current to 1 August 2026. 1st
  3. Competition and Markets Authority, CMA208 and CMA207, both published 4 April 2025. 1st
  4. BrightLocal, Local Consumer Review Survey 2026 , 11 February 2026. 1,002 US adults. 2nd

The question

Is Google about to start phoning my business?

In three named categories, in the United States, yes. Home repair, beauty and pet care. Restaurants are not named anywhere in the announcement, and the trade coverage saying otherwise is going beyond what Google published.

"And for select categories like home repair, beauty or pet care, you can ask Google to call businesses on your behalf."

Elizabeth Reid, VP Search, Google I/O 2026, 19 May 2026 50

"Just share your specific criteria, like finding a private karaoke room for six on a Friday night that serves food late, and Search brings together the latest pricing and availability with direct links to finish booking through the provider of your choice."

Same announcement, on booking local experiences and services 50

The booking expansion covers "local experiences and services," and Google's own illustrative example is hospitality-adjacent without being a restaurant reservation. Country: the US. Timing: stated as "this summer," which is the season now in progress. 50

The question nobody has answered

When ChatGPT, Perplexity or Gemini recommends a restaurant, where does that recommendation come from? Google Maps? A reviews site? Somebody's directory? None of those companies documents it. We looked for a first-party statement from each and found nothing. That is a gap in the public record, and we are going to leave it as a gap rather than fill it with a guess.

What Google does say helps

Google's own AI optimisation guide recommends Business Profiles and Merchant Center to make "products and services visible in both AI responses and other Google Search results." That is the only documented first-party link between a Business Profile and an AI answer we found. Google's consumer-facing help page for AI Mode does not say where its local data comes from. 51

The other two map surfaces

Apple Business Connect is free and lets a business control its Apple Maps place card, including actions for ordering food and booking a table. Apple publishes no Apple Maps usage figure anywhere we could find, so whether it is worth your time is genuinely unproven. Microsoft reports Copilot citations inside Bing Webmaster Tools, which strongly suggests Bing's index feeds Copilot, but Microsoft does not say so outright and we are not going to infer it as fact. 52 , 53

  1. Google, The Keyword, Google Search's I/O 2026 updates: AI agents and more , 19 May 2026. 1st
  2. Google Search Central, AI optimization guide, 10 July 2026; Google Search Help, AI Mode in Google Search , undated. 1st
  3. Apple, Apple Business Connect . No date on the page. Retrieved 8 August 2026. 1st
  4. Microsoft, Bing Webmaster Blog, Introducing AI Performance in Bing Webmaster Tools , 10 February 2026. 1st

The question

If I block AI from training on my site, do I disappear from AI answers?

It depends entirely on the company. Four of them run separate crawlers for training and for search, so blocking one does not cost you the other. Three do not, and those three are Google, Apple and Meta, which are exactly the ones a local business cannot afford to lose.

Company Separate crawlers? The agents, and what each does in the owner's words What blocking training actually costs you
OpenAI Separate GPTBot crawls "content that may be used in training." OAI-SearchBot is "used to surface websites in search results in ChatGPT's search features." ChatGPT-User is user-initiated Nothing. Block GPTBot and you can still appear in ChatGPT search. But block OAI-SearchBot and you "will not be shown in ChatGPT search answers" 54
Anthropic Separate ClaudeBot collects content "that could potentially contribute to their training." Claude-SearchBot "analyzes online content specifically to enhance the relevance and accuracy of search responses." Claude-User is user-initiated Nothing. Same three-way split as OpenAI 55
Perplexity Neither trains Both PerplexityBot and Perplexity-User are stated as "not used to crawl content for AI foundation models" Blocking either costs you visibility and buys you no training protection, because there is none to buy 56
Amazon Separate Amazonbot "may be used to train Amazon AI models." Amzn-SearchBot and Amzn-User both state they do not crawl for training Nothing. Note there are now three agents, not one, so old robots.txt rules are out of date 57
Google One control, both jobs Googlebot crawls. Google-Extended is a control, not a crawler, covering both training and "grounding (providing content from the Google Search index to the model at prompt time)" Your content stops being pulled into Gemini answers at prompt time. Google states there is no Search ranking consequence, and no ranking penalty is not no consequence 58
Apple One agent, both jobs Applebot powers "Spotlight, Siri, and Safari," may train Apple foundation models, and may "provide additional context and up-to-date content when AI models are used to generate output" Block Applebot and you lose Apple's search surfaces too. Applebot-Extended is the training-only opt-out and Apple says search inclusion survives it 59
Meta One agent, both jobs Meta-ExternalAgent crawls "for use cases such as training foundation AI models or improving products by indexing content directly." Meta-WebIndexer is the separate Meta AI search crawler. FacebookExternalHit fetches your link previews Dual purpose by Meta's own description, so blocking it is not a clean training-only decision. Blocking FacebookExternalHit breaks the preview image when somebody shares your menu 60

Common Crawl's CCBot is a separate case: a non-profit that publishes an open corpus which many labs train on. Blocking it is a pure training decision with no search or grounding cost. We could not retrieve first-party documentation for Bytespider or Bingbot, so we are not stating their owners' purposes for them. 61

  1. OpenAI, Overview of OpenAI Crawlers . Undated page. 1st
  2. Anthropic support documentation, updated 7 April 2026. 1st
  3. Perplexity, bots documentation. Undated page. 1st
  4. Amazon, Amazonbot developer documentation. Undated. 1st
  5. Google Search Central, Google crawlers , last updated 14 July 2026. 1st
  6. Apple, About Applebot , last updated 8 June 2026. 1st
  7. Meta, Web crawlers developer documentation. Undated page. 1st
  8. Common Crawl, CCBot . Undated. Bytespider and Bingbot not retrieved on 8 August 2026. 1st

The question

What happens on 15 September 2026?

If anyone has ever switched on Cloudflare's "Block AI bots" setting for your site, Googlebot gets blocked from that date. Not blocked from AI answers. Blocked from Google Search. This is the single most actionable thing in this document.

Cloudflare's own words

"Since the defaults will be enforced by the most restrictive applicable rules, multi-purpose crawlers such as Googlebot, Applebot, and BingBot will be blocked by customers who have selected to block Training (either through the new options to manage AI traffic, or through the legacy Block AI bots service)."

Cloudflare, "Your site, your rules," 1 July 2026 62

Read the part in brackets twice. It covers the legacy one-click toggle , not just the new controls. Anybody who flipped that switch in the last two years, believing they were keeping their content out of model training, is on course to block the crawler that builds Google's search index. Why: Cloudflare now classifies crawlers by all of their behaviours, and Googlebot both feeds Search and contributes to training. When a crawler does two things and you have blocked one of them, the most restrictive rule wins. 62

Why an ordinary business ends up here

Nobody sets out to block Google. The route in is a well-meant default: a one-click switch labelled as protection from AI, a managed robots.txt that "blocks common AI crawlers used for training," a developer tidying up during a build. None of those look like a decision about search, and all of them are. 64

The same logic applies to Apple and Meta, named in the same Cloudflare sentence and in their own documentation on the previous page. One crawler, two jobs, and a switch that does not distinguish between them.

What is actually changing, precisely

  • Three categories: Search ("collects or indexes your content, so it can answer questions about it later"), Agent ("acting, usually in real time, on a person's behalf") and Training 62
  • The new defaults apply to new domains onboarding to Cloudflare , and block Training and Agent only on "pages that display ads" 62
  • Search remains allowed by default. "Cloudflare blocks AI crawlers by default" is an over-read of what was announced 62
  • Existing customers can "opt out of these new default configurations" in Security settings before 15 September 62
  • The controls are on every plan, including Free 63
  1. Cloudflare, Your site, your rules: new AI traffic options for all customers , 1 July 2026. 1st
  2. Cloudflare developer changelog, New options to manage AI traffic , 1 July 2026. 1st
  3. Cloudflare AI Crawl Control documentation, last updated 23 April and 28 July 2026; Cloudflare Radar and Originality.ai dashboards, no figures retrievable 8 August 2026. 1st

Eight claims checked against the source.

Search and AI is where the most money is currently being spent on advice, and it is where the evidence is thinnest. These are the eight most repeated claims in the category.

The claim Verdict What the source actually says
AI answers appear on about half of searches Half of tracked keywords 48 percent is a share of one vendor's tracked keyword set, whose size has never been published. Real searches by real people run at 18 to 20 percent across two separate panels. 65
You need an llms.txt file Not for Google Google's own documentation: "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them." 66
You need special schema for AI search It does not exist Same document: "Structured data isn't required for generative AI search, and there's no special schema.org markup you need to add." Ordinary structured data still drives ordinary rich results. 66
GEO improves visibility by up to 40 percent On a machine the authors built The paper's own method: fetch "top 5 sources from Google search engine," generate the answer with "gpt3.5-turbo," measure on a 10,000 query benchmark. Not Google, not AI Mode, not ChatGPT. The preprint is 33 months old. 67
E-E-A-T is a ranking factor It is a description From the guidelines it comes from: "No single rating can directly impact how a particular webpage, website, or result appears in Google Search, nor can it cause specific webpages, websites, or results to move up or down." 68
Posting on your Business Profile helps you rank Not on the one test that exists 441 keywords per location across three listings, weekly posts for nine weeks, no measurable local pack effect. The firm that ran it reassigns posts to conversion. Nobody has tested their effect on AI answers. 69
Blocking AI training has no downside Depends on the company, and on Cloudflare OpenAI, Anthropic, Amazon and Perplexity separate training from search, so blocking training is free. Google, Apple and Meta do not. And from 15 September 2026 Cloudflare blocks Googlebot for anyone who has selected to block Training, including through the legacy one-click toggle. 70
Google will start booking your restaurant tables Not what was announced "For select categories like home repair, beauty or pet care, you can ask Google to call businesses on your behalf." US only. Restaurants are not named anywhere in the announcement. 71
  1. BrightEdge 12 Feb 2026; SparkToro and Similarweb 9 Jun 2026; Pew Research Center 22 Jul 2025. 2nd/1st
  2. Google Search Central, AI optimization guide, 10 July 2026. 1st
  3. Aggarwal et al., KDD 2024. 1st
  4. Google, Search Quality Rater Guidelines, 11 September 2025. 1st
  5. Sterling Sky, 2021, page maintained to February 2026. 2nd
  6. Crawler documentation per company; Cloudflare, 1 July 2026. 1st
  7. Google, I/O 2026, 19 May 2026. 1st

Questions people ask about this chapter

Every answer below is the chapter's own wording, with a link to the page it sits on.

How often does Google actually show an AI answer?

If you want to know what your customers are seeing, use the clickstream or Pew figure, because those count human beings.

Read the full answer
Am I losing clicks to AI answers?

What is not in dispute is the direction for an individual page: when an AI answer appears above your listing, fewer people go past it.

Read the full answer
Do I need an llms.txt file or special AI schema?

The company running the largest AI answer surface on earth has written down that it does not read them.

Read the full answer
How does Google decide which local business to show?

It publishes them on a help page that carries no date anywhere, gives no weighting, and states plainly that you cannot pay for a better position.

Read the full answer
If I block AI from training on my site, do I disappear from search?

Four of them run separate crawlers for training and for search, so blocking one does not cost you the other.

Read the full answer
What happens on 15 September 2026?

Find out whether your site is behind Cloudflare, and if it is, open the security settings and look at what is set for AI bots.

Read the full answer

Sources

  1. SparkToro with Similarweb panel data, 9 and 16 June 2026. 2nd
  2. BrightEdge 12 Feb 2026; SparkToro 9 Jun 2026; Pew Research Center 22 Jul 2025. 2nd/1st
  3. Google, 3 June 2026 and Alphabet Q2 2026 call, 22 July 2026. 1st
  4. BrightEdge, 12 February 2026; Ahrefs, relayed May 2026. 2nd
  5. Cloudflare, Your site, your rules , 1 July 2026. 1st
  6. Google Search Central, AI optimization guide, updated 10 July 2026. 1st
  7. Aggarwal et al., GEO: Generative Engine Optimization , KDD 2024. 1st
  8. Google, Search Console Help. 1st
  9. Google, I/O 2026, 19 May 2026. 1st
  10. BrightEdge, AI Overviews at the One-Year Mark , 12 February 2026. Produced with BrightEdge's own tooling. 2nd
  11. Semrush, AI Overviews Study , 15 December 2025. 2nd
  12. SparkToro, In 2026, Less than One Third of Google Searches Still Send a Click , 9 June 2026. Data provider Similarweb. 2nd
  13. Pew Research Center, 22 July 2025. Data collected March 2025, so 17 months old at the time of writing. 1st
  14. Pew Research Center, 22 July 2025. 68,879 searches by 900 US adults, 12,593 with an AI summary. Data collected March 2025. 1st
  15. SparkToro, 9 and 16 June 2026, Similarweb desktop and mobile panel, January to April 2026. Panel size not published. 2nd
  16. Ahrefs, AI Overviews Reduce Clicks , updated 4 February 2026. Supersedes their own 34.5 percent figure from April 2025, which is still circulating. 2nd
  17. Google, 6 August 2025. 1st
  18. Alphabet Q2 2026 call, 22 July 2026. 1st
  19. Google Search Central, Guide to Optimizing for Generative AI Features on Google Search , last updated 10 July 2026. 1st
  20. OpenAI Help Center, ChatGPT Search , and Overview of OpenAI Crawlers . Neither carries an absolute date. 1st
  21. Perplexity Help Center, updated 1 May 2026. 1st
  22. Anthropic, Claude Platform Docs, web search tool. Undated. 1st
  23. arXiv preprint 2604.25707, 28 April 2026. 602 prompts, 21,143 citations, 18,151 pages. Not peer reviewed. 1st
  24. Search Engine Land, sponsored, 29 July 2026. 2nd
  25. Google Search Central, Google's Guide to Optimizing for Generative AI Features on Google Search , last updated 10 July 2026. The older page, AI features and your website , updated 10 December 2025, says the same thing. 1st
  26. John Mueller, Google, quoted by Search Engine Journal, 2 June 2026. The article does not link the original thread, so this is reported speech and is colour rather than the load-bearing citation. 2nd
  27. Google Search Central, SEO Starter Guide , updated 10 December 2025. 1st
  28. Google Search Central, AI optimization guide, 10 July 2026. 1st
  29. Aggarwal, Murahari, Rajpurohit, Kalyan, Narasimhan, Deshpande, GEO: Generative Engine Optimization , KDD 2024. Preprint 16 November 2023, revised 28 June 2024. GEO-bench is 10,000 queries. 1st
  30. Google, Search Quality Rater Guidelines , edition dated 11 September 2025. 1st
  31. BrightEdge, 12 February 2026, and Ahrefs, relayed May 2026. Both vendor samples, unresolved. 2nd
  32. Google Search Status Dashboard, ranking history, retrieved 8 August 2026. Times US Pacific. 1st
  33. Google, The Keyword, 21 August, 8 September and 7 October 2025. 1st
  34. Google, I/O 2026 search updates and How AI Mode is changing the way people search , both 19 May 2026. 1st
  35. Sundar Pichai, Alphabet Q2 2026 earnings call and CEO remarks, 22 July 2026. 1st
  36. Google, Search Console Help, Generative AI performance report (Search) . The help page carries no publication date. Retrieved 8 August 2026. 1st
  37. Google, New opportunities, control and insights for website owners , 3 June 2026. 1st
  38. Google, 3 June 2026, and SparkToro with Similarweb, 9 June 2026. 1st/2nd
  39. Google Business Profile Help, Improve your local ranking on Google . No date on the page. Retrieved 8 August 2026. 1st
  40. Google Business Profile Help, Guidelines for representing your business on Google . No date on the page. Retrieved 8 August 2026. 1st
  41. Sterling Sky, Do Google Posts Impact Ranking? , published 26 July 2021, last modified 25 February 2026. 2nd
  42. Sterling Sky, Do Services in Google Business Profiles Impact Ranking? , 27 February 2026. 2nd
  43. Sterling Sky, Do Keywords in Business Names on Google Maps Impact Ranking? , 9 August 2023, last modified 17 November 2025. 2nd
  44. Sterling Sky, Does the Service Area in Your Google Business Profile Impact Ranking? , 31 March 2025. 2nd
  45. Whitespark, Local Search Ranking Factors , 2026 edition, published 6 November 2025. 47 practitioners scoring 187 factors. 2nd, opinion survey
  46. Google, Maps user contributed content policy. No date on the page. Retrieved 8 August 2026. 1st
  47. Electronic Code of Federal Regulations, Title 16 Part 465, Federal Register citation 89 FR 68077, 22 August 2024. Content current to 1 August 2026. 1st
  48. Competition and Markets Authority, CMA208 and CMA207, both published 4 April 2025. 1st
  49. BrightLocal, Local Consumer Review Survey 2026 , 11 February 2026. 1,002 US adults. 2nd
  50. Google, The Keyword, Google Search's I/O 2026 updates: AI agents and more , 19 May 2026. 1st
  51. Google Search Central, AI optimization guide, 10 July 2026; Google Search Help, AI Mode in Google Search , undated. 1st
  52. Apple, Apple Business Connect . No date on the page. Retrieved 8 August 2026. 1st
  53. Microsoft, Bing Webmaster Blog, Introducing AI Performance in Bing Webmaster Tools , 10 February 2026. 1st
  54. OpenAI, Overview of OpenAI Crawlers . Undated page. 1st
  55. Anthropic support documentation, updated 7 April 2026. 1st
  56. Perplexity, bots documentation. Undated page. 1st
  57. Amazon, Amazonbot developer documentation. Undated. 1st
  58. Google Search Central, Google crawlers , last updated 14 July 2026. 1st
  59. Apple, About Applebot , last updated 8 June 2026. 1st
  60. Meta, Web crawlers developer documentation. Undated page. 1st
  61. Common Crawl, CCBot . Undated. Bytespider and Bingbot not retrieved on 8 August 2026. 1st
  62. Cloudflare, Your site, your rules: new AI traffic options for all customers , 1 July 2026. 1st
  63. Cloudflare developer changelog, New options to manage AI traffic , 1 July 2026. 1st
  64. Cloudflare AI Crawl Control documentation, last updated 23 April and 28 July 2026; Cloudflare Radar and Originality.ai dashboards, no figures retrievable 8 August 2026. 1st
  65. BrightEdge 12 Feb 2026; SparkToro and Similarweb 9 Jun 2026; Pew Research Center 22 Jul 2025. 2nd/1st
  66. Google Search Central, AI optimization guide, 10 July 2026. 1st
  67. Aggarwal et al., KDD 2024. 1st
  68. Google, Search Quality Rater Guidelines, 11 September 2025. 1st
  69. Sterling Sky, 2021, page maintained to February 2026. 2nd
  70. Crawler documentation per company; Cloudflare, 1 July 2026. 1st
  71. Google, I/O 2026, 19 May 2026. 1st