« The Hunt for the Perfect SEO: Testing Knowledge vs. Testing Patience | Main | A Universal Truth from Yahoo Site Explorer »
Diagnosing Crawling and Indexing Issues with Yahoo Site Explorer
October 12, 2005
Yahoo recently released the Yahoo Site Explorer, a very helpful way to diagnose issues that may be hampering your performance in Yahoo natural search.
Enter a URL in the Search box, and Yahoo returns two values: First, the number of indexed pages for a given site, and as a second option, the number of incoming links pointing at that site. The following image shows the location of the key data points.
Drilling down, you can select specific URLs from the results page, and "explore" those pages in depth - finding, for example, the number of pages from a specific section of your site that have been indexed, or the incoming links pointing to a specific page of your site. To find an index count for a specific site section, enter or click a URL such as http://www.site.com/press/. This returns indexing and linking results for this specific URL, as well as any pages in the /press/ directory.
Unlike Google, which purposely returns only a percentage of a site's incoming links, Yahoo Site Explorer claims to show all incoming links that it knows about. This can come in very handy when performing a competitive link analysis for sites in your industry.
One of Site Explorer's largest drawbacks is the ability to download only the first 50 results into TSV format, for import into programs like Excel. It would be wonderful to have an entire site's worth of data to sort and play with in a spreadsheet program, but it's unlikely that Yahoo is too eager to spend processing time creating TSV files with tens of thousands of rows. A resourceful programmer named John Mueller has used the Yahoo Search API to create a custom version of Yahoo Site Explorer that overcomes some of these common obstacles; it's worth a look.
If you find large blocks of URLs from your site that Yahoo has not indexed, it offers a submission system similar to that of Google Sitemaps. You can simply fill a text file (such as pages.txt) with the URLs you want Yahoo to index, separated by a hard return. Upload the text file to your web server, then submit the entire URL of the text file (such as http://www.site.com/pages.txt) to the Yahoo Free Submit page.
We don't yet have significant data on the crawl rate for URLs submitted via this method, but we'll be sure to publish any information we collect.
All posts by Erik Dafforn
posted by Erik Dafforn at October 12, 2005 6:39 PM
Intrapromote: [ Case studies | SEO services | Bios ]
Trackback Pings
To TrackBack this entry, use the following URL:
http://seoblog.intrapromote.com/mt-tb.cgi/80
Comments
Good post, I like your website, thanks
Posted by: Bernard Harold at July 31, 2010 2:38 AM
Excellent job.
Posted by: SEO India at July 31, 2010 3:12 AM
A topic close to my heart thanks. Please consider a follow up post.
Posted by: Coloring Pages at August 4, 2010 1:09 PM
Excellent job.
Posted by: SEO Training at August 7, 2010 6:31 AM
Very nice article! I would like to thank you for the efforts you have made in writing this article. I am hoping the same best work from you in the tomorrow as well. In fact your creative writing ability has inspired me to start my own BlogEngine blog now. Really the blogging is spreading its wings rapidly. Your write up is a fine example of it.This is to be honest a Great knowledge gaining post and all thanks to bing search engine get me on here. I loved reading your writing and added to the favorites.Thanks again!
Posted by: what is the survival rate for liver cancer at August 11, 2010 1:57 PM
Cool entry cheers, definitely consider a follow up post.
Posted by: Free Avatars at August 12, 2010 2:18 PM
thanks!You made some good points there. I did a search on the topic and found most people will agree with your blog
Posted by: Sheree Swedberg at August 20, 2010 2:30 AM
Super site you have got here.
Posted by: las vegas golf at August 28, 2010 8:28 AM
Hows It Going found your site on Google and read a few of your other entires. Nice Stuff. Im looking forward to reading more from you. Great site post.
Posted by: Tattoo Aftercare at August 29, 2010 9:46 PM
This is my third visit to this blog. We are starting a new initiative in the same category as this blog. Your blog provided us with important information to work with. You have done a great job.
Posted by: gratis spel at August 31, 2010 5:51 PM