GPTBot
GPTBot is OpenAI's web crawler that collects publicly available content for training its models. Site owners control it via robots.txt: blocking GPTBot keeps pages out of future training data, but does not remove existing knowledge and does not affect ChatGPT's separate live-search fetching, which uses other user agents such as OAI-SearchBot.
GPTBot became the reference case for AI crawler policy because it was among the first clearly documented, opt-out-respecting AI user agents. The decision it forces is strategic: content excluded from training is content the base model never internalizes, which for a brand means future ChatGPT versions know you only through what others publish.
A frequent misconfiguration is blocking GPTBot while expecting ChatGPT to describe the company accurately from its website. Retrieval fetching (for ChatGPT search and browsing) uses different user agents, but teams often block those too in the same rule, silently removing their canonical pages from cited answers.
The pragmatic posture for most B2B brands: allow both training and retrieval agents on marketing, docs, and pricing pages; restrict only genuinely sensitive paths; and re-check the user-agent list periodically, since it evolves.
Related terms
See what AI currently says about your brand — free snapshot.