8 replies to this topic

[Jozian]

Anyone have a clue how I could determine a percentage of searches that were resolved sucessfully using G or it's competitors?

I'm looking for data that indicates how many searchers do not find their final answer through a SE, and what these searchers look like; demographics, vertical et al.

I'm guessing that the best data I can find will be from counts of users of advice forums and ask an expert type services.

Anyone have any better data or ideas?

You can see my blog post for more scoping info, or this previous thread.

Jeff

[Respree]

I can't see how Google or anyone else would know when a 'successful search' occured, although they certainly know what the denominator was in the formula below.

Pctg. = 'successful searches' (i.e. founded what they were looking for divided by total searches

Would you say a 'successful' search occurs when there are no subsequent searches containing one or more keywords used in the original search AND there was one or more link clicked on in the SERPS?

That was my first thought, but then it occured to me the searcher could have just given up, having not found what they were looking for.

Impossible to answer, I'd say... :-(

[JohnMu]

Take a peek into the AOL search database. AOL isn't Google - the user base will be a bit different, but they use the same engine (or at least did at the time the data for the database was taken).

Find users who search through multiple pages of results without clicking on any entry (or who continue to search despite having clicked). Oh, and keep track of query refinements while you're at it .

You didn't think we would just give you the number, did you?

John

Added: subscribed to your RSS

[eKstreme]

Would you say a 'successful' search occurs when there are no subsequent searches containing one or more keywords used in the original search AND there was one or more link clicked on in the SERPS?

You mean... personalized search... it's... based on a false premise?

This very question hits at the heart of personalized search as promoted by Google: if you search for X, click on link Y, and then search for X again, it assumes you're looking for Y, and so moves it up the SERPs. In reality, you're probably not satisfied with Y and so want a different answer. The only exception, as I mentioned previously, is navigational queries, when the basic assumption holds true.

Jeff: Take a look at hakia and join the Powerset mailing list (also read their blog). Both are semantic search engines and believe me, once you figure out to write queries properly, G/Y!/M/A and the rest are toast. The learning curve is to move away from keywords and write more naturally.

And search operators like intitle and inurl? They are such weaklings in hindsight!

Pierre

[Jozian]

I can't see how Google or anyone else would know when a 'successful search' occured...

Agreed, Respree, that it's not an easy thing to figure out.

But, this is such a basic indicator of value, one might expect someone like G is at least doing some in-lab testing or customer surveys to gather a feel for customer satisfaction based on specific transactions. Not that I expect them to share it If Google isnt looking at this, then you might want to start selling their stock short.

I'll check the AOL data - maybe some inferences can be drawn, John. And thanks for the signup - new blog and all

Pierre: will check out your recommendations.

-Jeff

[A.N.Onym]

If the user returns to the SERPs fast enough, then he didnt find what he wanted on the page.

I don't think searching for the same can be compared to failing, since someone might be researching and comparing the same stuff on other websites, and thus, the landed site could have given the necessary information.

[Black_Knight]

The measure of a successful search is surely when the person uses the service again to search for something unrelated.

Repeat custom is a valuable part of the metric from the search engine's perspective.

However, since this is obviously rather poor, you can see why Google were so eager to make Analytics a loss-leading freebie in order to have a spy in as many sites as possible, looking at the actual conversion data.

So Jozian, the answer Google found was Urchin and other tracking addons that are made as irresistable as possible to webmasters and marketers.

[Jozian]

Ammon, you make a great point about analytics and Urchin.

From a merchant's point of view it's not hard to see, or measure, the Google value equation. Even if the success or conversion rate is only say ~5-15% during a session. Plus later conversions based the initial hit. Plus branding value.

And from a generic searchers point of view, I would agree that Google has some value in nearly every search, and that returning to search again on something else implies some level of satisfaction with the process.

But, as to your statement "the measure of a successful search is surely when the person uses the service again to search for something unrelated", I can not agree. I would see that as a vote of confidence in the the service, and important indeed in measuring perceived value, but not an indicator of the success of the actual search.

Maybe the real question I am trying to express and measure here is better phrased like this:

Google is often the starting point. How often is it the finishing point?

What percentage of SERP users/queries start at a search engine but end up being resolved elsewhere: expert/advice service, library, custom subject vertical, paid access to non-public data, phone a friend, give up?

I would re-iterate that the SERP could still be adding value along the way, even when the search is unresolved, and even where it is resolved in another connected/unconnected venue.

-Jeff

[Black_Knight]

Maybe the real question I am trying to express and measure here is better phrased like this:

Google is often the starting point. How often is it the finishing point?

Almost never (unless your query is simply "who ranks for x?"). Google's 'job' is to deliver you to a destination, not to be one (mostly - calculator and definitions muddy this a little).

Google's job is to be a search engine, providing satisfactory results to a query. It doesn't actually have to provide the perfect answer, or the best answer possible. Just one that is satisfying enough to keep you coming back to use their service again. Their only real concern is that you should feel you get better results with them than with a direct competitor.

Google spent a long time placing directories and other 'hub' type sites very highly in search, as they still do with Wikipedia today. Wikipedia entries often contain many links that either send you off following cross-references, or lead you to work through citations and further information... but is this not often the 'site seeing' after attaining the answer sought?

Google's success as a search referrer, from their own perspective, cannot be about how many sites you end up visiting after a click-through. It is purely about being top of mind as a great starting point. Search is after all supposed to be a start point, not a destination in itself.

People often continue reading and surfing after finding what they initially sought. People often find a resource that lead to a telephone conversation, to consultation with friends, colleagues, or spouses before a deal is closed. Most importantly, just as is so often the case here at Cre8asite, sometimes the question you asked (of Google in this case) leads to further questions you'd not known enough to ask before, meaning that answering some queries is a long journey of many calls.

What most people will remember at the end of that journey is that Google was a successful starting point. That is the proper measure of success.