Indexed, though blocked by robots.txt: which of the two fixes you need
"Indexed, though blocked by robots.txt" means Google put a URL in its index without ever reading the page, because links pointed at it while a Disallow rule kept Googlebot out. If the page belongs in search, remove the rule. If it does not, allow crawling and add noindex.
The status appears under indexed pages in Search Console with a warning, not under errors. That matches what it is: an instruction that did not do what its author expected. robots.txt controls fetching. It has never controlled whether a URL can be listed.
How a blocked URL ends up in the index
Google learns that a URL exists from links: internal links, links on other sites, and sometimes a sitemap entry. A Disallow rule stops the fetch, so Google has no content, no title tag and no robots meta tag to work from. When enough links point at the address it can still be indexed on the strength of those links alone. In results it shows as a bare URL or an anchor-text title with no description.
The same mechanism defeats the most common attempt to fix it. Adding noindex to a page that is still disallowed changes nothing, because the tag sits in HTML that Googlebot is not permitted to download.
Confirm which rule is responsible
- Open the URL in Search Console's URL Inspection tool. Under Page indexing, Crawl allowed? reads "No: blocked by robots.txt".
- Open
https://example.com/robots.txtin a browser and read the live file, not the copy you remember editing. - Paste the file and the affected URL into our robots.txt tester. It reports allowed or blocked for the crawler you choose and names the line that decided it.
Rules match from the start of the path, so short patterns catch more than intended. Disallow: /p blocks /pricing/ and /products/ as well as /p/. Wildcards widen the reach further: Disallow: /*? blocks every URL with a query string, including paginated views on sites with plain permalinks.
Case 1: the page should rank
Delete or narrow the rule. In WordPress the file is usually virtual, generated on request, so there may be nothing to edit over SFTP. With Hydrogen SEO, open Hydrogen SEO → Robots.txt Editor, remove the line, save, and use View Robots.txt to read what is now served. The steps are in the robots.txt editor documentation.
Two WordPress-specific traps:
- A physical robots.txt file wins. If a real file sits in the site root, the web server returns it and WordPress never runs. The editor is locked in that case; edit or delete the file itself.
- Discourage search engines. With Settings → Reading → Discourage search engines from indexing this site ticked, Hydrogen SEO serves
Disallow: /for the whole site and addsnoindex, nofollowto every page. Untick it on a live site.
Then request indexing for the URL so Google fetches it soon.
Case 2: the page should stay out of search
Do the opposite of what feels natural: unblock it.
- Remove the
Disallowrule that covers the URL. - Add
noindexto the page. For a single post or page, open the SEO Settings sheet, open Robots Meta and switch on No Index. For a whole post type or taxonomy, use Hydrogen SEO → Robots. - Wait for Google to recrawl. The URL moves to "Excluded by 'noindex' tag", which is the status you want.
- Remove internal links to the page if it has no reason to be linked.
For files that cannot carry a meta tag, such as PDFs, send the directive as an X-Robots-Tag: noindex HTTP header from the server. For content that must stay private, use a login. A robots.txt rule is public and voluntary, and it lists the very paths you would prefer nobody noticed.
When leaving it alone is fine
Some blocked URLs are worth nothing in search and are linked from nowhere important: cart and checkout steps, internal search results, faceted filter combinations on a large store. If the block exists to stop crawlers spending requests on thousands of such URLs, a handful of them appearing in this report is an acceptable price. Blocking saves crawling; noindex requires crawling. On very large parameter spaces you sometimes have to pick the first.
Checking that it worked
Run a live test in URL Inspection. For case 1 it should report that crawling is allowed and indexing is allowed. For case 2 it should report crawling allowed and indexing not allowed because of noindex. Then open the status in the Page indexing report and click Validate fix. The count falls as Google recrawls each URL, which takes days for well-linked pages and longer for obscure ones.
Common questions
Does this status hurt the rest of my site?
No. It describes individual URLs. The practical harm is a blank, description-less result showing for an address you may not have wanted listed at all.
Why is it a warning and not an error?
Because the URL is indexed, which is normally the goal. Search Console flags it because blocking crawling and wanting a page indexed, or blocking crawling and expecting de-indexing, are both contradictions.
Can I add the noindex rule to robots.txt instead?
No. Google does not support a noindex directive inside robots.txt. Use a robots meta tag or the X-Robots-Tag header on a URL that Googlebot is allowed to fetch.
How long until the warning clears?
Until Google next crawls each affected URL. Requesting indexing for the important ones and validating the fix in the report speeds up the count, but there is no fixed schedule.