Google-Extended
Google-Extended is a robots.txt token that controls whether a site's content may be used to train Google's Gemini models, without affecting Google Search indexing.
It is a training control, not a visibility control. Disallowing it does not remove a site from Google Search, and does not remove it from AI Overviews, which are generated from the ordinary Search index.
This makes it one of the few clean choices available: a site can decline training use while remaining fully visible in Google's AI-generated answers.
Why it matters
It is widely misunderstood as an AI Overviews opt-out. Treating it as one leads either to a false sense of control or to unnecessary anxiety about visibility.
An example
A news site adds "User-agent: Google-Extended" with "Disallow: /" to robots.txt. Its articles are no longer used to train Gemini models. They still appear in Google Search and can still be summarised in AI Overviews, because both are built from the ordinary Search index that Googlebot crawls.
Common mistakes
- Treating it as an AI Overviews opt-out.
- Assuming it is a separate crawler. It is a control token; crawling is still done by Google's existing agents.
Questions
- Does blocking Google-Extended affect my rankings?
- No. It governs whether content may be used for Gemini training and related uses. It does not change Search indexing or ranking.
- Is Google-Extended a crawler?
- Not in the usual sense. It is a token read from robots.txt that tells Google whether content its existing crawlers collect may be used for those purposes. There is no separate Google-Extended bot fetching pages.