Skip to content
Answer Rankings Answer Rankings Answer Rankings

Bingbot: The Crawler Behind Copilot (Don't Block This One)

Bingbot is Microsoft’s search crawler — and in the AI era it does double duty: it feeds classic Bing Search and the retrieval layer behind Microsoft Copilot. One Disallow: / against Bingbot removes you from both, which makes it the most expensive possible line in a robots.txt.

User agent

Section titled “User agent”
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/116.0.1938.76 Safari/537.36

Robots.txt token: Bingbot. Verify claimed hits via Bing’s published IP verification tool — Bingbot is among the most-spoofed user agents.

Why it matters beyond Bing

Section titled “Why it matters beyond Bing”
  • Copilot — built into Windows, Edge and Microsoft 365 — grounds its web answers in the Bing index. Enterprise users asking Copilot to shortlist suppliers see only Bing-indexed sites.
  • Historically, several other AI engines have bootstrapped from Bing’s index — being absent from it has knock-on effects beyond Microsoft.
  • Roughly a third of desktop search outside Google touches the Bing index one way or another (Bing, DuckDuckGo, Yahoo, Ecosia).

The audit that takes 30 seconds

Section titled “The audit that takes 30 seconds”

Blanket “block all bots except Google” configurations — common on stores after a bot-traffic scare — frequently catch Bingbot. Check your robots.txt for it, or run the free AI visibility check, which tests Bingbot alongside five AI-specific crawlers.

# Recommended
User-agent: Bingbot
Allow: /

Also confirm your CDN or bot-management layer isn’t challenging Bingbot with a CAPTCHA — a challenge page indexes as an empty page. This is a recurring silent failure on Cloudflare-fronted ecommerce sites; the fix is an allow rule for verified search bots.

Section titled “Related”