Your Website Is Probably Invisible to ChatGPT. Here's Exactly Why, and What You Can Do About It.

Step 1, Retrieval. When you ask a question, the AI first goes looking for relevant information. ChatGPT search uses its own crawler, OAI-SearchBot, alongside outside search providers. Gemini and Google's AI Mode use Google's own index. Perplexity runs its own crawler, PerplexityBot. If your website isn't reachable by these systems, the AI never even considers you. You're eliminated before the game starts.

Step 2, Extraction. The AI takes the pages it finds and breaks them into passages. It converts each passage into a kind of mathematical fingerprint that captures its meaning. Then it compares those fingerprints against the question being asked, looking for the closest match. This is why how you write matters just as much as what you write. A passage that makes no sense without the paragraph before it produces a confused fingerprint. A passage that is self-contained, clear, and specific produces a precise one.

Step 3, Synthesis. The AI weaves the best matching passages into a coherent answer, citing the sources it found most trustworthy, most relevant, and most clearly structured. The sources that get cited are the ones that survived Steps 1 and 2 cleanly.

That's it. That's the whole process. And once you understand it, the reasons a website gets left out become immediately obvious.

The problem is that AI crawlers are new, and they have their own bot names: OAI-SearchBot decides whether you can appear in ChatGPT search, GPTBot collects content to train OpenAI's models, ClaudeBot works for Claude and PerplexityBot for Perplexity. If your robots.txt was set up before these bots existed, there's a good chance you're blocking them without knowing it.

In a study published in May 2025, Ahrefs checked the robots.txt files of about 140 million websites and found that 5.9% blocked GPTBot and 5.5% blocked OAI-SearchBot, yet only 0.5% named GPTBot directly, so most of that blocking came from blanket rules rather than a decision about AI. OpenAI is clear about what it costs: a site that blocks OAI-SearchBot will not be shown in ChatGPT search answers.

It isn't only robots.txt. On 15 September 2026 Cloudflare, one of the biggest hosting and security networks, replaced its single "Block AI bots" switch with separate controls for Search, Training and Agent crawlers. Sites already on Cloudflare had their old settings moved across so they work the way they did before, and new domains are now offered a preset when they join: search, training and agents all allowed for a site without ads, and for a site that carries ads, training disallowed and agents blocked on the pages that show ads. If your site sits behind Cloudflare, check those three settings and make sure Training says Disallow AI Training or Allow rather than Block. If it sits behind a different firewall, check its AI crawler settings too. We go through every place a block can hide in Something Might Be Blocking AI From Your Website.

There is a second way to shut them out without meaning to. Vercel watched AI crawlers on its own network and found in December 2024 that none of the crawlers behind ChatGPT, Claude or Perplexity ran JavaScript, even the ones that downloaded the files. If your text, prices or menu only appear after JavaScript runs, those crawlers see an empty page.

What to do: Go to yourwebsite.com/robots.txt right now. If you see a line that says Disallow: / under User-agent: *, you may be blocking everything. Make sure your file explicitly allows OAI-SearchBot, GPTBot, ClaudeBot and PerplexityBot, and check your hosting firewall as well. Then open your homepage, choose View Source, and search for a sentence you can see on the page; if it isn't there, the AI crawlers can't read it either. If you're not sure how to do any of this, ask your web developer. It's a quick fix, and it comes first because nothing else on this list matters if they can't get in.

2. The AI Has Never Heard of You From Anyone But You

Here's the uncomfortable part.

AI tools don't just read your website. They are trained on, and retrieve from, the wider internet. News articles, reviews, directories, forums, social media discussions, industry publications. When an AI decides whether to recommend a business, it's not just evaluating your website in isolation. It's asking: does the rest of the internet agree that this business is credible?

Yext looked at 6.8 million citations in ChatGPT, Gemini and Perplexity answers to local questions in July and August 2025. Businesses' own websites made up 44% of those citations, and listings, the directory and map profiles that describe a business, made up another 42%, with reviews and social media at 8%. Your website matters, and so do the places that list and describe you.

If the only place your business exists online is your own website, the AI has one source of evidence. And one source, no matter how good your website is, is not enough to build confidence.

What to do: Build your external presence. Get listed on credible directories in your industry. Ask your clients for Google reviews. Publish your results on LinkedIn. Get mentioned by name in articles or roundups. Every third-party reference to your business is a vote of confidence that AI systems can verify independently of your own claims.

3. Your Content Doesn't Pass the Island Test

This is probably the most practically useful thing in this entire article.

Remember how AI tools break your content into passages? Each passage needs to be understood completely on its own, without any reference to the paragraphs before or after it. We call this the Island Test. Can this paragraph stand alone, like an island, and still make complete sense?

Copy written to flow from one paragraph into the next often fails this test. Look on your own pages for sentences like:

"It offers three key benefits for our clients."

What is "it"? The AI has no idea. The antecedent, the thing being referred to, was in the previous paragraph, which may now sit in a different passage with a different fingerprint.

What to do: Go through your most important pages and read each paragraph in isolation. If a paragraph starts with "It", "This", "They", or "These", replace the pronoun with the specific noun it refers to. A paragraph that stands alone is far easier for a machine to lift and quote than one that leans on the paragraph before it.

4. Your Website Has No Structured Data

Structured data, sometimes called schema markup, is a piece of code you add to your website that tells search engines and AI tools exactly what your content is about. Not through words they have to interpret, but through a standardised, machine-readable format that removes all ambiguity.

Think of it this way. Your About page might say "We're a small web design studio in Varna, building sites in Bulgarian and English." A human reader understands that instantly. But an AI reading raw HTML has to infer the business type, the location, the languages, and the identity from the text. Structured data removes that guesswork.

Be clear about what it does and doesn't do, though. Ahrefs tracked 1,885 pages that added schema between August 2025 and March 2026 and found no major lift in how often ChatGPT or Google's AI features cited them. Where it clearly helps is exact facts: in Kevin Indig's July 2026 test of AI agents, adding Product and Offer markup with the price moved a page's score from 73 to 93. It's labelling, and good labelling is worth having.

What to do: At minimum, add Organisation schema to your homepage, FAQ schema to any page with questions and answers, Article schema with a dateModified field to every blog post, and Offer markup with the price on any page that shows one. If you use WordPress, plugins like Yoast or Rank Math handle most of this automatically. If you have a custom-built site, ask your developer. It's a one-time addition that keeps working as long as it matches what the page actually says.

5. Your Content Is Written to Impress, Not to Answer

Website copy written to sound impressive uses elevated language, talks about passion and commitment and unique approaches, and carefully avoids saying anything too specific in case it sounds limiting.

AI tools have no interest in this kind of content whatsoever.

AI tools are built to answer questions. They are looking for specific, factual, declarative statements they can extract and present as reliable information. "We build custom restaurant websites that load in under two seconds and include integrated booking systems" is a sentence an AI can use. "We deliver exceptional digital experiences for the hospitality sector" is a sentence an AI ignores entirely.

What to do: Go through your service pages and count how many sentences make a specific, verifiable claim. If most of your copy is adjectives and generalities, rewrite it with specifics. Real numbers. Real outcomes. Real named services with real described benefits.

6. You Have No FAQ Content

FAQ-style content makes complete sense once you understand how AI tools work.

AI tools are primarily question-answering machines. When someone types a question, the retrieval system goes looking for content that matches question patterns. Pages that contain explicit questions followed by clear answers are inherently easier for the retrieval system to match against user queries.

What to do: Add a genuine FAQ section to your most important pages. Not marketing questions like "Why should I choose you?", but the real questions your clients ask before hiring you. "How long does a website redesign take?" "What do I need to prepare before our first meeting?" "Do you work with clients outside Bulgaria?" Write clear, direct answers to each one.

7. Your Content Is Stale and Never Updated

Freshness matters more in AI search than you might expect.

Ahrefs looked at 17 million citations across seven AI search platforms in July 2025 and found that the pages AI assistants cite are 25.7% fresher on average than the pages in ordinary Google results. AI tools are trying to give accurate, current answers, so they lean towards content that shows signs of being maintained and up to date.

A page you published in 2023 and never touched since sends a weak freshness signal. A page you updated last month, even with a small but real addition of new information, sends a much better one.

What to do: Set a quarterly reminder to review your most important pages and update them. You don't need to rewrite everything, add a new statistic, update a result, add a new FAQ, refresh an example. Then update the dateModified field in your schema. Only change the date when the content has really changed, though: Google lists "changing the date of pages to make them seem fresh when the content has not substantially changed" as a sign of content made for search engines rather than people.