IndexNow for WordPress: Instant Indexing in Hydrogen SEO

Instant Indexing (Hydrogen SEO, under Settings) tells search engines that use the IndexNow protocol about a URL soon after you publish, update or remove it, instead of waiting for their crawlers to come back. It is off until you switch it on. Once on, Hydrogen SEO creates a key, serves the key file at https://yoursite.com/<key>.txt, and sends changed URLs to the shared IndexNow endpoint in the background a short time after each change. The screen arrived in version 0.0.27, is free, and needs no Hydrogen SEO account.

IndexNow is not used by Google; as the screen puts it, Google keeps finding changes through your sitemap. For Google, your XML sitemap and Search Console remain the way in; Instant Indexing adds to them and replaces neither.

What IndexNow is, in one paragraph

IndexNow is an open protocol: a site proves it controls a domain by hosting a key in a text file, then sends lists of changed URLs together with that key. The engines that take part have agreed to share every submission with each other, so one request reaches all of them. According to indexnow.org, IndexNow is supported by Microsoft Bing, Naver, Seznam.cz, Yandex and Yep; the protocol's published endpoint list also includes the Internet Archive and Amazon's crawler. A submission is a hint, not an order. The engine still decides whether and when to crawl the URL and whether to index it.

Switch it on

In wp-admin, open Hydrogen SEO → Instant Indexing (under Settings).
In the IndexNow card, turn the switch on. The change is saved at once, and the first time you switch it on Hydrogen SEO generates a key (32 random hexadecimal characters) and starts serving the key file.
Under Content types to announce, every public content type except media attachments is ticked to begin with. Untick any whose changes you do not want announced; each click is saved as you make it.
In the Key card, click Check key file. Hydrogen SEO requests the file from your own site and tells you whether it came back with the key. You can also open the link shown there in a private window: it should be a plain-text page containing only the key.

It is opt-in because it sends your URLs to an outside service. Nothing is sent while it is off, and the key file is only served while it is on.

The key file is served with Plain permalinks too, but on some servers a .txt address is answered by the web server itself and never reaches WordPress. With Plain permalinks the Key card warns you about this: click Check key file, and if it fails, choose any other structure under Settings → Permalinks.

What gets submitted, and when

A URL is queued when a post of a selected type is:

published, for the first time or again after being unpublished,
updated while it is published, or
unpublished, trashed or deleted, so engines recrawl it and learn it is gone instead of keeping a dead result.

Only posts that are indexable, or were until the change you just made, are announced. A post set to noindex (on its own or through its content type's robots settings), password-protected, or with a canonical URL pointing at another page sends nothing when it is published, edited, trashed or deleted: telling engines to crawl a page you have asked them not to index would only waste their visit. A post that the current change makes noindex is sent once, so engines recrawl it, see the noindex and drop it.

If an update changes the slug, both addresses are sent: the new one to be crawled, the old one so engines find the redirect or the 404 there. For an unpublished or trashed post, the address it had while it was published is sent, not the temporary address WordPress gives a post in the trash.

Queued URLs are not sent while you click Update. About 30 seconds after the first change, WordPress's scheduler (WP-Cron) sends everything in the queue as one request, so editing stays fast and a burst of edits becomes one submission. The IndexNow card shows how many changes are waiting to be sent.

Failed sends are retried

Each request waits at most 3 seconds for IndexNow to answer. If an automatic send cannot reach IndexNow, times out, or gets a 429 or a 5xx answer, its URLs are not lost: they go back in the queue and are sent again about 10 minutes later, on their own. Each URL gets at most three attempts; after the third failure it is dropped. A 400, 403 or 422 answer is final, because sending the same request again would get the same answer; fix the cause instead (for 403, start with Check key file). URLs submitted by hand in the Submit URLs box are sent once; if one of those fails, submit them again.

Only live sites send

Submissions go out only when WordPress reports the site as a production environment (wp_get_environment_type() returns production, which is the default when nothing sets it). On a site marked local, development or staging, the IndexNow card names the environment type and explains that nothing is sent from it. That keeps a staging copy from announcing its own URLs, or your live URLs with a key the live site does not serve. Many managed hosts set the environment type on staging copies for you; if yours does not, set it in wp-config.php:

php

define( 'WP_ENVIRONMENT_TYPE', 'staging' );

Developers can override the decision with the hydrogen_seo_indexnow_can_send filter, which receives true on production.

Submit URLs by hand

The Submit URLs card takes full URLs, one per line, up to 10,000 at a time. Use it after a bulk change that did not go through the editor, such as an import, a template change or a price update made by another tool. Every URL must be on this site; the button stays disabled and the box lists the offending lines until they are, because the protocol only accepts URLs on the host the key belongs to.

Submit what actually changed. Re-sending your whole site does not make engines crawl it faster.

Read the submission history

Recent submissions keeps the last 50 requests. Each shows the HTTP status (or Not sent), the number of URLs, whether it was Automatic or Manual, the time, up to three of the URLs it carried and what the response means:

Response	What it means	What to do
`200 OK`	The URLs were received.	Nothing.
`202 Accepted`	The URLs were received, and the key is still being validated. Normal for the first submissions with a new key.	Nothing. It turns into `200` once validation completes.
`400 Bad Request`	The request was not in a valid format. Not retried.	Check the URLs you pasted into the manual box.
`403 Forbidden`	The key is not valid: the key file was not found, or it does not contain the key. Not retried.	Click Check key file. A security plugin, firewall rule or cache serving an old response is the usual cause.
`422 Unprocessable Entity`	A URL does not belong to this site's host, or the key does not match the protocol's format. Not retried.	Check for URLs on another domain or on `www` when the site is not on `www`, and the reverse.
`429 Too Many Requests`	Too many requests, treated as potential spam. Automatic sends are retried about 10 minutes later.	Nothing, unless it keeps happening; then look for another tool on the site also sending IndexNow, and avoid resubmitting unchanged URLs by hand.
No status, or `5xx`	IndexNow could not be reached, did not answer within 3 seconds, or had a server error. Automatic sends are retried about 10 minutes later.	Nothing, unless every attempt fails; then check that your host allows outgoing HTTPS requests.

A 200 is proof of receipt only. It does not mean anything was crawled or indexed.

Check that engines received your URLs

Bing Webmaster Tools has an IndexNow section that reports the URLs Bing received through IndexNow, including recent submissions and whether they have been crawled and indexed. Verify your site there first; the Webmaster Tools screen makes that a paste-and-save job. Expect submitted URLs to show up there with some delay.

Regenerate the key

If the key has been exposed somewhere you did not intend, or a submission keeps failing with 403, click New key in the Key card and confirm. The new key file is served at once, the old one stops being served, and the first submissions with the new key may return 202 again while it is validated.

When Instant Indexing helps, and when it does not

It helps most where content changes often and the change matters quickly: news, events, job listings, stores with stock and price changes, and deletions you want reflected soon. On a site that publishes a few posts a month, the engines' normal crawling usually finds new posts in reasonable time anyway; switching it on does no harm, but the difference is small.

It does nothing for Google, nothing for a page that is blocked in robots.txt or set to noindex, and nothing for a page an engine decides is not worth indexing.

Troubleshooting

Nothing is being submitted. Check that Instant Indexing is on, the content type is ticked, and the IndexNow card does not say the site is not a production environment. A count of changes waiting that never goes down points at the scheduler. Then check WP-Cron: on a site with DISABLE_WP_CRON set to true and no real cron job calling wp-cron.php, background tasks never run. On a site with very little traffic, scheduled tasks wait for the next visit.

Every submission returns 403. Click Check key file, and open the file in a private window. If it returns an error page, a login screen or a cached 404, something in front of WordPress is blocking it. Allow the .txt URL in your security plugin or firewall and purge the cache.

Every submission returns 422. The URLs are not on the host the key was issued for. This happens when the WordPress Address and Site Address settings disagree with the address visitors actually use.

FAQ

Does Instant Indexing work with Google?

No. Google does not take part in IndexNow. Keep your sitemap submitted in Search Console, and use URL Inspection there for the occasional page that needs a nudge.

Do I need a Bing account for IndexNow to work?

No. The protocol only needs the key file on your site. A verified Bing Webmaster Tools property is useful for seeing what Bing received, not for sending.

Will it submit my whole site when I switch it on?

No. It submits URLs as they change from then on. IndexNow is meant for changes; your sitemap already lists everything else.

Does it send anything about my visitors?

No. A submission contains your site's host, the key, the key file location and the list of changed URLs. Nothing about visitors is included.