What is the search engine visibility setting?

What the checkbox changes

With the box ticked, three things happen:

1. Every page gets this tag in its head:

<meta name='robots' content='noindex, nofollow' />

2. The built-in sitemap at /wp-sitemap.xml is disabled.
3. The Sitemap: line is removed from the virtual robots.txt.

The robots.txt rules themselves stay as they are. Older WordPress versions wrote Disallow: / when the box was ticked. Current versions do not, and for a good reason: a crawler that is blocked from fetching pages cannot read the noindex tag on them, and blocked URLs can still be indexed from links. Allowing the crawl and serving noindex is the method that reliably keeps pages out.

In the database, the setting is the blog_public option: 1 for visible, 0 for discouraged.

What it does not do

Under the checkbox WordPress prints: "It is up to search engines to honor this request." The setting is a request, and it is not privacy.

• Pages stay publicly reachable by anyone with the address.
• Crawlers that ignore robots directives, including many scrapers, read everything.
• Files in the uploads folder are served by the web server and carry no meta tag.
• Pages already indexed stay in results until they are recrawled.

For a staging site or anything confidential, use password protection at the server or host level, which stops crawlers and people alike. The checkbox is an extra layer, not the lock.

Checking a live site

If a site is missing from search, check this before anything else.

• In the admin: Settings → Reading. The box should be empty. The At a Glance widget on the dashboard also shows a "Search engines discouraged" notice when it is on.
• From outside: view the source of the homepage and search for noindex, or run:

$ curl -s https://example.com/ | grep -i "name='robots'"

• In Search Console: affected pages are listed under Excluded by 'noindex' tag in the Page indexing report, and URL Inspection shows "Indexing allowed? No".

Hydrogen SEO's site checks flag the setting on its dashboard. While the box is ticked the plugin also leaves out its own snippet and preview directives, since the whole site is already noindex, nofollow; see advanced robots meta.

After you untick it

Unticking takes effect on the next page load, but search engines only notice when they recrawl. To speed that up:

1. Clear every cache layer: page cache, CDN, host cache. A cached page keeps serving the old tag.
2. Recheck the page source for noindex. A second tag may come from an SEO plugin setting, a theme, or an X-Robots-Tag header added by the host on temporary domains.
3. Submit the sitemap in Search Console and request indexing for the homepage and a few key pages.

Recovery time depends on how often Google crawls the site. A new site has little crawl history, so allow days to a few weeks. Our guide to fixing "Excluded by noindex tag" covers the other sources of a stray noindex, and the SEO setup for a new WordPress site puts this check in its launch list.

The staging trap

The mistake runs in both directions. A staging copy with the box unticked gets indexed and competes with the live site. A live site cloned from staging inherits the tick, because the option travels in the database. Add "check Settings → Reading" to every launch and every migration, right next to "check robots.txt".