Submit Updated or Un-indexed Pages to Google Search For Indexing

There are trillions of pages on the web, but Google doesn’t index them all. Some pages are duplicates, some have multiple URLs that lead to the same page, some pages are broken while others are simply forgotten. Sometimes, those forgotten pages happen to be from your website.

Logistically, it’s impossible for Google to index the entire web. A large number of pages are and will always be missing from the search engine. If you are to go through your website’s index through Google Webmaster Tools, you might notice that some pages on your site are not indexed. This is not uncommon and happens to a lot of website.

Previously, it was difficult to make Google index one particular page, since you can only submit sitemaps and not individual pages to Google for crawling. One way to do is is to add the URL to the un-indexed page to your website’s homepage and hope the Google bot finds it.

Google has now added a new feature to Google Webmaster Tools that allows website owners to submit individual URLs for inclusion into the search engine’s index. This might be previously un-crawled pages or recently updated ones.

The Fetch as Googlebot feature in Webmaster Tools now provides a way to submit new and updated URLs to Google for indexing. After you fetch a URL as Googlebot, if the fetch is successful, you’ll now see the option to submit that URL to our index. When you submit a URL in this way Googlebot will crawl the URL, usually within a day. We’ll then consider it for inclusion in our index. Note that we don’t guarantee that every URL submitted in this way will be indexed; we’ll still use our regular processes—the same ones we use on URLs discovered in any other way—to evaluate whether a URL belongs in our index.

Google provides a few examples where this new functionality may be helpful:

• If you’ve just launched a new site, or added some key new pages, you can ask Googlebot to find and crawl them immediately rather than waiting for it to discover them naturally.
• If you’ve updated some key content on your pages such as information about an event you’re hosting this weekend and want to make sure visitors see it in time, you can submit those URLs.
• If you’ve accidentally published information that you didn’t mean to, and want to update Google’s cached version after you’ve removed the information from your site.

When you submit URLs to index you can choose whether you want to submit only the one URL, or that URL and all its linked pages. Website owners have a maximum limit of 50 submissions per week. When submitting URLs with all linked pages, the limit is 10 submissions per month. You can see how many submissions you have left on the Fetch as Googlebot page.

Note, that you can only submit pages that would be suitable for Google Web Search, not images or videos.

The Fetch as Googlebot option can be found under Diagnostics on your Google Webmaster Tools Dashboard.

[via Google Webmaster Blog]