AI assistants
Google-Agent
Browses the web and takes actions on pages when a person asks a Google AI agent to.
May skip robots.txt- User agent token
- Google-Agent
- Operator
- Verification
- Published IP list
- Source
- Google documentation
What an AI assistant fetcher does
An assistant fetcher visits only when someone asks an AI assistant a question that needs your page. Each visit stands for a real person, often one who is about to click through, so blocking it usually costs you readers.
Google says user-triggered fetchers generally ignore robots.txt, because a person asked for the fetch.
Block or allow Google-Agent in robots.txt
Add one of these to the robots.txt file at the root of your domain. Crawlers read it before they fetch anything else.
Block Google-Agent
User-agent: Google-Agent
Disallow: /
Allow Google-Agent
User-agent: Google-Agent
Allow: /
More from Google
Questions, answered.
Google-Agent is an AI assistant fetcher run by Google. Browses the web and takes actions on pages when a person asks a Google AI agent to.
Not always. Google-Agent fetches pages for a person, and Google says robots.txt may not apply. Google says user-triggered fetchers generally ignore robots.txt, because a person asked for the fetch.
Add "User-agent: Google-Agent" and "Disallow: /" to your robots.txt. Since robots.txt may not stop it, block it at your firewall or server too.
Check the request's IP address against the list Google publishes. A user agent alone proves nothing, since any script can copy it.
Indirectly. Google-Agent opens your page because a person asked about it, and the answer can link back to you.
Usually not. Each visit stands for a person asking about your page, so blocking Google-Agent turns readers away.
In its own documentation at https://developers.google.com/search/docs/crawling-indexing/google-user-triggered-fetchers. Every fact on this page comes from there.
Still have questions? We're happy to help.
See which bots really read your site.
NoirTrack counts Google-Agent and every other bot apart from real visitors, and its firewall can block them before they reach your pages.
Start free trialFree 14-day trial, no credit card. The server SDK also sees bots that never run JavaScript.