Skip to content
Answer Rankings Answer Rankings Answer Rankings

OAI-SearchBot: The Crawler Behind ChatGPT Search Citations

OAI-SearchBot is OpenAI’s search crawler. It builds and refreshes the index that ChatGPT Search pulls from when it answers with live web results and citation links. Allowing it is the single most direct robots.txt lever for appearing in ChatGPT’s cited answers.

User agent

Section titled “User agent”
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot

Robots.txt token: OAI-SearchBot.

Why it matters more than GPTBot for citations

Section titled “Why it matters more than GPTBot for citations”

GPTBot feeds training — slow-moving, uncredited influence. OAI-SearchBot feeds retrieval: when ChatGPT searches the web, sources come from this index and get linked in the answer. For traffic and recommendations, this is the bot that pays the bills. Sites that blocked “all AI bots” in 2023 usually caught this one in the same rule and now wonder why competitors get the ChatGPT citations.

Allow or block?

Section titled “Allow or block?”
# Recommended
User-agent: OAI-SearchBot
Allow: /

There’s rarely a good reason to block it: it doesn’t feed training data, it drives referral traffic, and OpenAI honours the directive. If you block nothing else AI-related, don’t block this.

Check whether your current robots.txt lets it through — plus five other AI crawlers — with the free AI visibility check.

Section titled “Related”