Monday, October 08, 2007

Google Search indexer and Google Groups

About a month ago, I posted about my release of a new Windows Media Player skin. In that post, I noted that at the time of that posting, a standard Google web search for the term 'WMPImage_AlbumArtSmall' (a keyword used for adding album art to a Windows Media Player skin) had only a single result.

I just repeated that search, curious to see whether the Google search indexer had picked up my use of that keyword and added my post to the search results -- indeed it did. Interestingly, though, I noticed a 3rd result now present as well: A Google Groups page with more information on the WMPImage_AlbumArtSmall keyword that I had linked to from my original post.

Apparently, prior to that post, the Google search indexer hadn't indexed Google's own archived newsgroup thread; but when it indexed my blog post, it looks like the Google indexer followed the link to the Google Groups thread and indexed the thread at that point.

So it seems that (at least in this case) Google's search indexer doesn't crawl Google's own Groups discussion threads, but does index such threads when there is a link to the thread from elsewhere on the Web. I wonder why the Google indexer doesn't crawl those threads by default? Perhaps the quality of such discussion threads is presumed to be too low to merit inclusion in standard web search results as far as search results go; but if someone links to a thread's page, that thread's context is then assumed to be good enough to merit indexing? Or there might be a concern that Groups pages would appear too often in standard search results if all Groups pages were indexed by default -- and that's one reason that "Groups" is a separate search option from Google's main web search?

1 comment:

  1. I've also noticed that Google Groups isn't standardly indexed. I've always suspected it was both the low overall quality and the fact that some frequent posters (or post topics) would overwhelm the system.

    ReplyDelete