Google-Extended

A robots.txt switch that decides whether Google may use your pages to train Gemini models and to ground their answers.

robots.txt switch, never crawls

User agent token: Google-Extended
Operator: Google
Verification: Not needed, it sends no requests
Source: Google documentation

What an AI training crawler does

A training crawler gathers public pages that may end up in the data used to train AI models. It sends no visitors back, so blocking it is the usual choice for sites that don't want their content used that way.

Blocking it does not remove you from Google Search.

Block or allow Google-Extended in robots.txt

Add one of these to the robots.txt file at the root of your domain. Crawlers read it before they fetch anything else.

Block Google-Extended

User-agent: Google-Extended
Disallow: /

Allow Google-Extended

User-agent: Google-Extended
Allow: /

More from Google

GooglebotCrawls pages for Google Search, Discover, Images, Video and News. GoogleOtherA general crawler Google product teams use to fetch public content outside Search. Google-CloudVertexBotCrawls a site when its owner asks Vertex AI to build an agent from it. Storebot-GoogleCrawls product pages for Google Shopping. Google-InspectionToolFetches a page when someone runs the Rich Results Test or URL Inspection in Search Console. AdsBot-GoogleChecks the quality of landing pages used in Google Ads. Googlebot-ImageCrawls images for Google Images, Discover and the image features in Search. Googlebot-VideoCrawls videos for the video features in Google Search. Googlebot-NewsA robots.txt name for Google News: it crawls with the regular Googlebot user agents, and these rules decide what appears in News. GoogleOther-ImageThe image version of GoogleOther, fetching public image URLs for Google product teams. GoogleOther-VideoThe video version of GoogleOther, fetching public video URLs for Google product teams. AdsBot-Google-MobileChecks the quality of mobile landing pages used in Google Ads. Mediapartners-GoogleReads pages that show AdSense or Ad Manager ads so the ads match the content. APIs-GoogleDelivers push notification messages sent through Google APIs. Google-SafetyCrawls for abuse checks, such as finding malware. Google-AgentBrowses the web and takes actions on pages when a person asks a Google AI agent to. Google-GeminiNotebookFetches a page that a person added as a source in a Gemini Notebook project. FeedFetcher-GoogleFetches RSS and Atom feeds for Google News and WebSub. GoogleProducerProcesses the feeds publishers supply to Google News through Publisher Center. Google-Read-AloudFetches a page so Google can read it out loud with text to speech. GoogleMessagesBuilds the link preview when someone sends your page in Google Messages. Google-PinpointFetches a page that a Pinpoint user added as a source to their research collection. Google-Site-VerificationFetches the token that proves you own a site in Search Console. Google-CWSRequests the URLs a developer lists in a Chrome Web Store extension or theme.

Browse all 66 crawlersAI and search bots from 16 companies, with the rules to block or allow each one.

Questions, answered.

Google-Extended is not a crawler. A robots.txt switch that decides whether Google may use your pages to train Gemini models and to ground their answers.

Google-Extended never visits your site. It is a robots.txt name that Google reads to decide how pages its other crawlers fetch may be used. Blocking it does not remove you from Google Search.

Add "User-agent: Google-Extended" and "Disallow: /" to your robots.txt.

There is nothing to verify: Google-Extended sends no requests.

No. Google-Extended never visits your site; it only decides whether Google may use pages its other crawlers fetch for AI training.

Block it if you don't want Google using your pages for AI training. Its search crawler keeps working, so your search rankings stay the same.

In its own documentation at https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers. Every fact on this page comes from there.

Still have questions? We're happy to help.

Read the docs Contact support

See which bots really read your site.

NoirTrack counts Google-Extended and every other bot apart from real visitors, and its firewall can block them before they reach your pages.

Start free trial

Free 14-day trial, no credit card. The server SDK also sees bots that never run JavaScript.