Think you can type a search term into Google and it will find most of the relevant pages featuring that word or phrase? Think again.
Three weeks ago, Google Operating System reported on an odd bug in Google's search results. If you searched for a highly common term, Google would claim to find millions of results, but would stop showing them after delivering only 100 or so, claiming that everything else was a duplicate result.
At the time, I figured it was a temporary bug and Google would fix it in a few days. Google constantly tweaks its search algorithms and sometimes that leads to problems. But when I checked it out again last night, exactly the same issue was still evident.
A search on Google's Australian site for "Yahoo" suggested that there were more than a billion pages featuring that phrase. Not so odd. But if you scroll through the results, you only get to page 8 -- just 80 results -- before Google tells you this:
In order to show you the most relevant results, we have omitted some entries very similar to the 80 already displayed.
I don't doubt there's a massive amount of duplication in Google's billion-odd results including the word "Yahoo". But I also don't doubt that there are more than 80 dinstinct pages with the word Yahoo on them online, or indeed that there are more than the 78 pages of results Google promises in this message. Hell, there are more than 80 posts on Lifehacker referring to Yahoo. Let's not even think about how many pages are on Yahoo!'s own domains, even excluding the Japanese domains that dominate the last page of results we do see.
This is a massive screw-up on Google's part. It looks like its desire to present "relevant" or "current" results is crashing out well before it has shown you everything in its index.
Google offers you the option to see a fuller set of results with the duplicates included. But for the Yahoo query, even that timed out after 49 pages (or 490 results), and didn't even offer me the ability to see more searches. After page 49 of searching for Yahoo, it just gave up. That means of the claimed billion results, Google doesn't want to show you more than 500 at best.
Google's stated aim is to offer you the result you want on the first page, but that's not always the reason we use it. Sometimes we want to see everything linking to a particular term. Right now, you can't trust what Google tells you in that context at all. Bear that in mind if you're involved in an SEO project or are using Google for research.