What is Google-Extended?
Google-Extended is a name you can address in robots.txt to tell Google whether your content may be used for its Gemini models: for training and for grounding answers in Gemini apps and the Vertex AI API. No bot with that name visits your site, and Google Search is unaffected.
It answers a narrow question: "Google may index me, but may it also use me for Gemini?"
Syntax
To opt out:
User-agent: Google-Extended
Disallow: /
To opt out for part of a site:
User-agent: Google-Extended
Disallow: /research/
Allow: /
With no Google-Extended group in the file, the use is allowed. Note that a blanket User-agent: * group with Disallow: / covers this token as well, like every other crawler that lacks a group of its own.
Why you will not find it in your logs
Most crawler names are both a robots.txt token and part of the user agent string in each request. Google-Extended is only the first. The crawling is done by Google's existing crawlers with their usual user agent strings, and the token just tells Google what it may do with the result. Searching access logs for "Google-Extended" finds nothing, and a firewall rule matching that string blocks nothing.
What it does and does not change
| Affected by blocking Google-Extended? | |
|---|---|
| Crawling and indexing by Googlebot | No |
| Rankings in Google Search | No. Google states it is not a ranking signal |
| AI Overviews and AI Mode in Search | No. These are Search features, governed by Googlebot access and snippet controls |
| Training of future Gemini models | Yes |
| Grounding in Gemini apps and Vertex AI | Yes. Your pages are not used to support those answers |
The third row is the one most often misunderstood. If your aim is to stay out of AI Overviews, this token does nothing. The controls there are nosnippet, data-nosnippet and max-snippet, which also shrink or remove your ordinary snippet; see nosnippet.
Should you block it?
Consider the trade on both sides.
Reasons to block. You publish content whose value lies in its text, such as original reporting, paid research or a licensed catalogue, and you do not want it used for model training without an agreement. Your position on AI training is a matter of policy, and you want it stated in the standard place.
Reasons to allow. Gemini is a place where people ask questions about products and services like yours. If your pages cannot be used for grounding, Gemini answers those questions from other sources. For a business that wants to be discovered, that is a loss with little offsetting gain.
A middle path is to disallow only the sections that hold your paid or licensed material and leave marketing and documentation pages open.
Like all robots.txt rules, it works from the time Google next reads the file and is not retroactive.
Setting it on WordPress
WordPress serves a virtual robots.txt unless a physical file exists in the site root. Either edit the physical file, or use an editor that manages the virtual one. In Hydrogen SEO that is Robots.txt Editor under Tools, described in the robots.txt editor doc. Add the group on its own lines, separated from other groups by a blank line.
Then verify three things. Open yoursite.com/robots.txt and confirm the group is present. Run the token through our robots.txt tester to see the rule that applies to a sample URL. And make sure you did not also add a broad rule that blocks Googlebot by accident, which would affect Search. Hydrogen SEO's AEO analysis reports the Google-Extended status of a page separately from the search crawlers, so you can see at a glance which decision is in force.
For the wider set of bots and how they differ, see AI crawler and our guide to blocking AI crawlers in robots.txt.
Common questions
Does blocking Google-Extended hurt my Google rankings?
No. Google states that Google-Extended does not affect a site's inclusion in Google Search and is not used as a ranking signal.
Does Google-Extended stop my content appearing in AI Overviews?
No. AI Overviews and AI Mode are part of Google Search and rely on Googlebot's crawl. To limit use there, Google points to the snippet controls: nosnippet, data-nosnippet and max-snippet.
Is Google-Extended a crawler?
No. It is a robots.txt token with no user agent string of its own. Google's regular crawlers fetch the pages, and the token controls how the content may be used.