Skip to content
geo· 7 min read
Published: Updated:

How does your website show up in ChatGPT? A GEO guide

Search is no longer just Google. What Generative Engine Optimization (GEO) is, how AI bots read your site, and what to put in robots.txt and llms.txt — with our own files as examples.

Short answer

To be cited in ChatGPT, Claude, Perplexity or Google's AI answers, your site first has to be open to those services' search bots and its content has to be readable in the raw HTML. After that it is classic SEO: content that answers up front, cites its sources and stays current. Google states plainly that AI Overviews need no extra file or special markup.

What is GEO, and how is it different from SEO?

SEO is the work of climbing a search engine's ranked list. GEO (Generative Engine Optimization) is the work of appearing inside an AI assistant's answer as a source. The term comes from "GEO: Generative Engine Optimization" by Aggarwal et al., published at KDD 2024.

The paper measures that content changes can raise a source's visibility in generative search answers by up to 40%. It also stresses that which method works varies from domain to domain. The measurement was made on the researchers' own benchmark; it is not a guarantee of the same lift on your site.

First: can the bots get in?

This is the first and most-skipped step. OpenAI, Anthropic and Perplexity list their bots on official pages. The distinction matters: some collect content for model training, some index pages for search answers, and some open a page the moment a user asks something.

BotCompanyWhat it doesrobots.txt
GPTBotOpenAICollects content that may be used to train modelsRespected
OAI-SearchBotOpenAISurfaces websites in ChatGPT's search featuresRespected
ChatGPT-UserOpenAIOpens a page at a user's requestRules may not apply
ClaudeBotAnthropicCollects content that could contribute to model trainingRespected
Claude-SearchBotAnthropicCrawls to improve the quality of Claude's search resultsRespected
Claude-UserAnthropicOpens a page when a user asks Claude a questionRespected
PerplexityBotPerplexitySurfaces websites in Perplexity search results; not used for model trainingRespected
Perplexity-UserPerplexityOpens a page at a user's requestGenerally ignored
Google-ExtendedGoogleNot a crawler but a control token: manages use of content for Gemini trainingRespected

In practice: if you don't want your content used for training but do want to appear in search answers, you can block GPTBot and ClaudeBot and leave OAI-SearchBot, Claude-SearchBot and PerplexityBot open. Google states that blocking Google-Extended does not affect your inclusion or ranking in Google Search.

On our own site we allow all of them. An excerpt from our robots.txt:

text
User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: ClaudeBot
Allow: /

Is the content in the raw HTML?

The bot can get in, but what does it see? On a site built with React or Vue, content is often created in the browser after JavaScript runs. The official pages don't say whether these bots execute JavaScript; the safest assumption is that the content must already be in the raw HTML.

You can check this yourself: right-click the page and choose "View page source". If your text isn't there, you need server-side rendering (SSR) or prerendering. This site is built with React too; before each release we generate every page as static HTML.

llms.txt: our own file

llms.txt is a format proposed by Jeremy Howard: a single Markdown file summarising what a site is and listing its key pages. It is a proposal, not an official standard. Google states plainly that AI Overviews and AI Mode need no such file or special markup.

Still, it is easy to write, and a one-glance summary of your site helps any tool that reads pages. The top of our file (custudio.com.tr/llms.txt):

text
# CU Studio Yazılım (CU Studio Software)

> CU Studio Yazılım is an independent software studio in Turkey. We design,
> build and launch websites, web apps, mobile apps (iOS + Android, React
> Native) and e-commerce stores end-to-end — fixed price, written scope,
> source code handed over — and we build and run our own apps. Turkish pages
> live at the root; English pages under /en.

## Pages (Turkish)

- [Home](https://custudio.com.tr/): What the studio does, our own products, selected work, services and pricing summary
- [Services](https://custudio.com.tr/services): Websites, mobile apps, e-commerce, MVP development and maintenance

The structure follows the proposal: an H1 title, a short summary in a blockquote, then annotated link lists under H2 sections. The pricing section of the file is regenerated from the site's pricing file on every release, so llms.txt and the pages never contradict each other.

Which content gets cited?

The platforms don't publish their citation rules, so there is no exact formula. The two most solid sources we have:

  • Google: there are no additional requirements or special optimizations to appear in AI Overviews or AI Mode. The page must be indexed and eligible to be shown with a snippet. Structured data must match the visible text on the page.
  • The GEO paper: content changes can raise visibility, but the effective method depends on the domain. There is no single "writing for AI" template.

The practical rule that follows is simple: ask the question in the heading, answer it in the first sentence, back your claims with sources and keep the content current. Traffic from Google's AI features shows up in Search Console's Performance report under the "Web" search type; there is no separate report.

The to-do list

  1. Allow the search bots (OAI-SearchBot, Claude-SearchBot, PerplexityBot) in robots.txt; decide on training bots separately.
  2. Open the page source: if the content isn't in the raw HTML, set up prerendering or SSR.
  3. Add llms.txt: who you are, what you do, your key pages.
  4. Restructure content as Q&A and lists; move the answer into the first sentence.
  5. Add Schema.org markup (Organization, FAQPage, Article) and keep it consistent with the visible text.
  6. Update your posts; outdated information does as much harm as wrong information.

If you want a site built on this foundation, what's included is on our Next.js web development page.

Sources

#geo#seo#ai

Have a project in mind?

A short description is enough; if it isn't clear yet, we can think it through together. We'll write back within one business day.

With your consent, we use analytics cookies to improve the site and measure visits. Cookie policy