Generally in SEO we want our pages in the Google indices, because a page that is not indexed cannot be found via search. The NoIndex code snip does exactly what it implies; it tells Google to NOT index the page, so that it cannot be found in search.
What does the NoIndex code snip look like?
If you view a web page code and search for “noindex”, you might find this in the head section:
<META NAME=”ROBOTS” CONTENT=”NOINDEX, FOLLOW”>
This code snip, in English, basically tells the spidering programs of search engines (aka: ROBOTS) to not record the information on the page (NOINDEX) but to still relate this page to the pages that link out of it (the FOLLOW part).
Why would we want to use a NoIndex?
We should use NoIndex on any page that we don’t want a human to find without us directly telling them about it. For example:
- Email Promotion: A special promotion page that we communicate to segmented customers via direct email promotions
- Employee only pages: Pages that might be on our websites which would be told to an employee by a link from an internal resource.
- Goal completion pages: If analytic goals are set up to count how many people reach the “thank you” page. This will help eliminate false goal completion.
Also, any page that is highly duplicate content of other pages on the website should have a NoIndex if 1) the page is not a direct duplicate copy of another page and 2) a canonical tag cannot be used. Our mountain resort pages do not have many instances of this, but Blogs (like buzz.snow.com) or personal Word Press sites have many examples of this.
Is there a “Do Index” code snip?
Yes, it is the same as above except it says “INDEX,FOLLOW”. I suggest not using it because it is pointless and only adds more code to your page (generally a negative). Telling a search engine to index your page is like taking me an all you can eat Chinese Buffett and telling me to grab some food; trust me, I already know what to do in this situation.
What is the “Follow” part in the code above; is there a “NoFollow”
FOLLOW tells the spiders/robots to either follow the links on the pages, or don’t. Generally, we want to have them follow the links, because this helps our overall crawl rate of our sites, and to spread PageRank more evenly. NoFollow should only be used at the individual link level, and generally not the entire page. NoFollow does not prevent the linked to page from not getting indexed.
This seems potentially dangerous…is it?
Yes, very. Imagine if your home page suddenly got a NoIndex tag applied to it. With the speed that Google indexes pages now, your site visits (and probably revenue) would drop off within hours. Because of this, I suggest using a monitoring system to track changes to the page and alert you of changes to the Head section of your HTML, and always start your diagnostics process by looking for a NoIndex.