« The Pied (Jaffray) Piper of LOCAL Search? | Main | Search Engine Friendly Pull-Down Navigation Menu »
Optimizing Sitemaps Feeds for Yahoo
October 31, 2006
If you're submitting sitemap feeds to Yahoo, consider using the exact same file you use for your Google feeds (often sitemap.xml or sitemap.xml.gz by default).
Until recently, I'd been using another of Yahoo's recommended formats, urllist.txt (due to its minimal file size), but I hadn't been watching the output code as closely as I should have. I'd been exporting the sitemap.xml file directly to urllist.txt.
As it turns out, this can create bloat even in a text file, because (depending on the program you use to create it), your Google sitemap.xml file contains many URLs you might not actually want to be crawled.
To clarify, I create many Google sitemap.xml files and tell Google to "check" but not "crawl" the incidental graphics files (used in design, nav, and so on). But upon export to urllist.txt, my program was simply listing these graphics files in the list to be crawled, just like all html files. That more or less tripled the size of the file, with two-thirds of the content being URLs I didn't even care about.
As a result, I deleted the reference to urllist.txt in Yahoo Site Explorer, and instead told it to fetch sitemap.xml, and within a week, the index count at Yahoo tripled. (Note that we've been working on a few other things for this site too, so I'm not necessarily claiming a 1:1 relationship here. But I know my change didn't hurt.)
Also follow this thread at YSE forums, where later, "Mr. Slurp" offers a user some keen insight into how Yahoo interprets typical "home" pages such as default.htm, etc. I guess the moral of the story is, canonicalization is in the eye of the beholder - never exclude when you can redirect.
All posts by Erik Dafforn
posted by Erik Dafforn at October 31, 2006 11:59 PM
Intrapromote: [ Case studies | SEO services | Bios ]
Trackback Pings
To TrackBack this entry, use the following URL:
http://seoblog.intrapromote.com/mt-tb.cgi/319
Comments
I found this tool which has helped me a lot in integrating a constistent site map into both Yahoo and Google: http://www.rorweb.com/rormap.htm.
Posted by: Craig Killick at November 1, 2006 01:32 PM
Great article... I too was using the older format and noticing around 0 traffic generated by the urllist.txt. Consistent format is convenient but lets hope it works too :)
Posted by: JV at June 5, 2007 12:04 PM
>>If you're submitting sitemap feeds to Yahoo, consider using the exact same file you use for your Google feeds (often sitemap.xml or sitemap.xml.gz by default).>>
Thanks.That was what I was searching for . I had submitted a sitemap.xml.gz file to google and was wondering whether to use .Xml or .xml.gz for the yahoo feed
Posted by: pillai at January 25, 2008 09:26 AM