What Is the Meta Robots Tag? SEO Guide

What is the meta robots tag?

The meta robots tag is a <meta> element in a page's <head> that tells search engine crawlers how to treat that specific page: whether it may be indexed, whether its links should be followed, and how it may be shown in results. It looks like <meta name="robots" content="noindex, follow">.

Where robots.txt works at the path level and controls crawling, the meta robots tag works per page and controls indexing and display. The crawler has to fetch the page to read it.

Updated September 30, 2026By the Hydrogen SEO team

How it works

With no tag at all, search engines assume index, follow. You only add a tag to change that. The directives Google supports include:

noindex: keep the page out of search results.
nofollow: do not follow the links on this page.
noarchive: historically blocked the cached copy. Google retired its cache links in 2024, so this matters mainly for other engines.
nosnippet: show no text snippet or video preview.
max-snippet:[n], max-image-preview:[none|standard|large], max-video-preview:[n]: limit how much of the page is shown.
notranslate, noimageindex, and unavailable_after:[date].
none: shorthand for noindex, nofollow.

name="robots" addresses all crawlers. You can target one crawler with its name instead, such as name="googlebot". When directives conflict, Google applies the most restrictive one.

nosnippet and max-snippet also control whether page text can be used in Google's AI features such as AI Overviews, which makes them the current opt-out for those surfaces.

Example

An internal search results page that should not rank, but whose links should still be crawled:

html

<head>
  <meta name="robots" content="noindex, follow" />
</head>

A product page allowed to rank with large image previews:

html

<meta name="robots" content="max-image-preview:large" />

In WordPress

WordPress core prints <meta name='robots' content='max-image-preview:large'> by default and switches to noindex, nofollow when Discourage search engines is checked. SEO plugins take over the rest. Hydrogen SEO offers noindex, nofollow, and noarchive site-wide per post type and taxonomy on its Robots screen and per post in the SEO Settings sheet, leaving other directives at search engine defaults; see robots meta settings.

Common mistakes

Two robots tags on one page from a theme and a plugin. Google combines them and applies the most restrictive.
Adding the tag with JavaScript. Google can read it after rendering, but many other crawlers cannot. Put it in the server HTML.
Using nofollow on a page when you meant noindex. They do different jobs.

Common questions

Where does the meta robots tag go?

Inside the head element of the HTML page. Tags placed in the body are not reliably honored.

What is the difference between meta robots and X-Robots-Tag?

They accept the same directives. The meta tag lives in HTML; the X-Robots-Tag is an HTTP header, so it also works for PDFs, images, and other non-HTML files.