What is Bingbot?
Bingbot is the crawler Microsoft uses to fetch web pages for Bing. It follows links and sitemaps, downloads pages, renders them, and passes the content to Bing's index. That index answers searches on Bing and grounds answers in Microsoft Copilot, so being crawlable by Bingbot matters beyond Bing's own share of search.
It behaves much like Googlebot, with a few differences in the controls it accepts.
How it identifies itself
The robots.txt token is bingbot. The full header value looks like this:
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/W.X.Y.Z Safari/537.36
W.X.Y.Z is replaced by the browser version Bing currently renders with. There is also a mobile variant with a phone device string.
To confirm a request is genuine, do a reverse DNS lookup on its IP address: real Bingbot hosts end in search.msn.com, and a forward lookup of that hostname returns the same IP. Bing Webmaster Tools includes a Verify Bingbot tool that does this for you, and Microsoft publishes the crawler's IP ranges.
Controls Bingbot respects
- robots.txt, including a
bingbotgroup of its own:
User-agent: bingbot
Disallow: /internal-search/
Crawl-delay: 2
- Crawl-delay. Unlike Google, Bing honours it. Bing Webmaster Tools also offers a crawl control chart for shifting crawl activity to quieter hours.
- Meta robots and X-Robots-Tag, including
noindex,nofollow,nosnippetandnoarchive. A tag addressed to<meta name="bingbot" ...>applies to Bing only. - XML sitemaps, submitted in Bing Webmaster Tools or listed in robots.txt.
- IndexNow. Bing is one of the engines behind this protocol. A ping tells it a URL was added, changed or deleted, without waiting for a crawl.
Differences from Googlebot worth knowing
| Googlebot | Bingbot | |
|---|---|---|
| Crawl-delay in robots.txt | Ignored | Honoured |
| IndexNow | Not supported | Supported |
| Console | Search Console | Bing Webmaster Tools |
| Verification hostname | googlebot.com, google.com | search.msn.com |
Both render JavaScript with an up-to-date browser engine, and both work best when the main content is present in the HTML the server sends.
When Bingbot is not crawling your site
- Check robots.txt for a rule that blocks
bingbotor everything. Our robots.txt tester shows the matching rule. - Check the firewall. Bot protection that lets Googlebot through sometimes blocks other crawlers. Look for
403or429responses to Bingbot in your logs. - Verify the site in Bing Webmaster Tools and submit the sitemap. The URL inspection there shows crawl and index status for a page.
- Remove an oversized crawl-delay. A value like 30 limits Bing to a few thousand pages a day.
On WordPress
Nothing in WordPress core treats Bingbot differently from other crawlers. Two plugin features help it along: Hydrogen SEO adds the Bing verification meta tag from its Webmaster Tools screen, described under search engine verification, and sends IndexNow pings when content is published or updated, covered in instant indexing. For the wider picture, see our page on how to rank in Bing.
Common questions
Should I block Bingbot?
Almost never. Blocking it removes your pages from Bing and from the products built on Bing's index, including Copilot. If it causes server load, lower its rate with crawl-delay or the crawl control in Bing Webmaster Tools.
How do I know if a visit is really Bingbot?
Reverse-resolve the IP address. A genuine Bingbot hostname ends in search.msn.com and resolves back to the same IP. The Verify Bingbot tool in Bing Webmaster Tools performs the same check.
Does Bingbot follow the same robots.txt rules as Googlebot?
It follows the same standard and uses its own group when one names bingbot. The main difference is that it also obeys the crawl-delay line.