Everyone in SEO talks about searches that are for something to buy, but what percentage of the time is this true? If 5% of the all searches were for product to purchase, I would be very surprised. There are numerous searches that are either clearly informational, or are in that lovely grey area such as "online recipe", on which commercial sites appear, usually selling a related product.
The real key has nothing to do with whether a user's search for "cheap ink cartridges", is filled, because AdWords will always have ads for such products, but whether her search for "how ink cartridges work
" gets the information the user is after.
Google has to balance the needs of those looking to buy things with the needs of those looking for information about those things. This is not just because of some moral imperative that doesn't exist, but rather because of Gogole's business model, which is based around market share. If people don't find what they need for every search, of which the vast majority of searches are not commercial, Google will either lose part of this market share, or at least be vulnerable to such a lose. Unless Google change their business model, keeping the how, what and why searchers happy is vital.
As far as the free SERPs are concerned, this places a need to slant them heavily in favour of informational content, which is so very hard to define. This is not just for revenue raising purposes (although, financially, it does make sense to disadvantage commercial sites), but because there will always be AdWords selling stuff, meaning such users will always have access to what the need, but AdWords for people giving away stuff for free is far less likely, and only likley to be filled via the free component. I think Google know this, and I think they work hard @ making sure these searches are filled.
IMHO Google also know that their USP is not so much how good their algorithmic search is, but rather how good it is perceived
to be. The former is very difficult to define and murky teritory, but the later is much easier to define and, inevitably, far easier to manipulate. That is why keeping SEOs happy, particularly those with blogs (Peter_d), is important to Google from a marketing perspective. In many ways, SEOs / SEMs are kinda like the NRA in American politics (sans ageing former star): a small group of vocal people who exert far more influence than perhaps they should. Because of the power of the web and Blogs, rightly or wrongly, Google has a vested interest in appeasing this group, to at least some degree, and by defining spam, Google is creating a playing field, which it controls, and funneling peoples attacks in a direction they can quite easily respond to. Punishing spam then becomes an excercise in PR (public relations, that is), in which Google can be seen to be living up to its end of the bargain.