« Google Webmaster Tools Beefs Up Anchor Text Report | Main | Perfect for 2007 March Madness for Some of the Games! »
Google's Supplemental Index: Questions and Inconsistencies
March 19, 2007
What I want to discuss in this post is Google's unreliable method of showing which pages are and are not in the Supplemental index.
What I don't want to get into with this post, beyond a very superficial level:
- Whether pages being in the Supplemental index is bad (or merely not good)
- How pages end up in the Supplemental index
Let's just say that all things being equal, I'd rather have 10,000 pages in Google's main index as opposed to its Supplemental index.
Using the method (that is fairly commonly accepted, in my opinion) of determining which pages from a site are in Google's Supplemental index (site:yoursite.com *** -view -- first read at SEOBook -- see References), I decided to pick a random URL -- in this case, our blog's article archives in the "Crawling and Indexing" category:
So clearly -- at least according to Google (who should be the authority) -- this page does sit in Google's Supplemental index. Right?
But let's refine the query to be a mere listing of the site contents -- site:yourdomain.com. To find the URL I discussed before, I needed to scroll to page 37 of the results:
This time, it doesn't show the "Supplemental" label. But is this a contradiction? Maybe in a straight-up site: query, the Supplemental label doesn't appear.
No, that's not it either, because if you click over to one more page of results -- page 38 in this case -- Supplemental results DO start to show up with the Supplemental label. Here (some on page 38, and all on page 39 and beyond), most of our pages are labeled as Supplemental:
So there's an irrefutable contradiction. Some Google results call this page Supplemental, and some don't. Why is that? Is it a data-center thing? Have some machines in the server farm not yet received the proper memo?
But let's cut to performance. For the query [crawling and indexing], the page ranks #1 at Google (even with no account sign-in). It doesn't bring a ton of traffic, but it does bring some:
So some would say, "There's your answer. It doesn't matter, because it shows up in SERPs and brings traffic. Quit overthinking it."
That's true -- IF the page is truly a Supplemental page. But because we have mixed signals, it's not so clear. Is this a Supplemental page that performs well despite its Supplemental status, or a Main index page that performs well and happens to be sometimes mislabeled as "Supplemental"? That's an important question I can't answer right now.
To follow up this post, I'll try to hit another angle: When a URL that is consistently labeled as Supplemental does not show up in search results, despite the fact that it's the most relevant post for that query.
Resources, Notes:
- Barry Schwartz first (to my knowledge) cracks the issue with this post last September. The technique worked briefly and sporadically.
- Adam Lasnik clarifies some Supplemental concepts at Google Webmaster Help (Google Groups).
- SE Roundtable brings up more questions.
- Aaron Wall brings it up again, with the added refinement of subtracting a nonsense string, which is more or less its current form.
- In pre-emptive response to one of my favorite audience niches (the beside-the-point nitpickers), I have adapted our robots.txt file in response to the third screen shot above. It now excludes search results and and stray comment previews.
All posts by Erik Dafforn
posted by Erik Dafforn at March 19, 2007 03:20 PM
Intrapromote: [ Case studies | SEO services | Bios ]
Trackback Pings
To TrackBack this entry, use the following URL:
http://seoblog.intrapromote.com/mt-tb.cgi/375
Comments
That is really some deep thinking. My guess is that google hasn't even figured that out yet.
Posted by: tanning lotion at March 8, 2008 01:13 AM
Do you think that googles algorythm are that complex?
Posted by: gbg liquid vitamins at April 6, 2008 03:32 PM