Yahoo! Adds Trends to Concept Analysis Options

[bragadocchio]

Another Yahoo! Patent Application was published on May12, 2005.

This one is Systems and methods for search query processing using trend analysis , and it looks at analyzing certain trends shown in search engine use to accompany Yahoo!'s recent patent applications that incorporate searchers queries into the delivery of search results.

Amongst other things, the patent application begins to explain how MyYahoo! information might be used to help the search engine create search results.

<edit - fixed URL to patent>

[randfish]

Bill,

Thanks for the update. Much appreciated! I think there's a lot of similiarities conceptually between Yahoo!'s quest for user information and Google's. I would imagine that the divide between search engines' sources for data will continue to narrow, as everyone will want every possible piece of data.

[bragadocchio]

Hi Rand,

There does seem to be more and more similarities in both search engines attempts to make their results more relevant, and to figure out how to incorporate user information into their results.

There's a lot in this patent application, but I think that you need to read the whole series of four that seem to tie together to get the bigger picture.

Here's where this patent application starts to add something new:

In accordance with an embodiment of the present invention, query processing engine 304 also includes a trend analysis module 320 that looks for trends in the queries along various dimensions, such as time, geography, user demographics, user history or context (referred to herein as a \"vertical\" dimension), etc. Trend analysis module 320 advantageously leverages the functionality of unit analysis module 310 and clustering module 312 to perform concept analysis and/or concept discovery on different subsets of the queries, then compiles the results. The resulting trend information is advantageously added to unit dictionary 306.

Keep in mind that this concept network looks at user queries, possibly a human edited dictionary, and other relevance methods to classify and return results. The trend analysis adds some elements.

Time -

Queries can be divided into periods of time to analyze trends, such as by week or month to look at user interests over time. Time of day, time of week (weekday vs. weekend or holiday days), Time of year such as seasons.

A user's local time may be used for this trend information.


Geography -

IP addresses, or user profiles could be used to identify the possibly country of origin of the query and where the user resides.

Demographic -

may be collected from the query log, on information such as age, gender, etc. Combinations of demographic information may be combined to understand and show trends

Examples include

female users who are over 25 and own their homes,
male users who are over 35 and rent,
users under 25 who own cars,
and so on.

Vertical or user history -

The query log may provide information about the users activity before their query.

Example,

Information could be collected at a portal site such as Yahoo! including a

shopping area,
a general web directory,
a music property,
and so on,

If each of those different portal areas had a a search interface where the user could enter queries, that "vertical" information about the user could be saved. The type of information that could be collected with a continuous access to the different related portals could be included in a page index, so that the search engine can understand more about the searchers who perform queries, and the context in which the search takes place - in other words, what else they've been searching for as they travel around the portal(s).


Examples of how trend analysis might influence search results

One example of this in action might be to recognize that Avril Lavigne is a more popular query for users under 21, and Celine Dion for users over 35.

Another example might be to recognize that when someone searches for java on weekdays, they are more likely to be searching for the programming language, and on weekends, the drink.

Other possible trends are illustrated, including one for party (political or social event), poison (a toxic chemical or a hair band), and more.

Can understanding trends help to "personalize" search results? I'm not sure.

[randfish]

female users who are over 25 and own their homes,
male users who are over 35 and rent,
users under 25 who own cars,
and so on.

Where could a search engine get information like this and map it to an IP address? I could see Yahoo! doing it a little since they have Yahoo! Finance, Autos, etc. where people sign up and volunteer this info, but other than that...

I think that trending can be applied here more in the sense of improving overall SERPs rathen than just to personalize.

[bragadocchio]

There is some indication that Yahoo! could use this type of information gathering to personalize results:

Any inferences made about a particular user can be used to tailor responses to further queries entered by that user, again using the trend data as one indication of likely user intent. Thus, trend data may be used to customize the response of a search server to the particular user who enters a query.

and

As a sixth example, in some embodiments, search server system 160 might suggest related searches based on trend information. For example, if a user who is known to be under 21 enters a query containing a singer's name, search server system 160 might, in addition to displaying a list of sites relevant to that singer, suggest searches related to one or more other singers identified by trend analysis as being popular among users under 21 (e.g., \"Avril Lavigne\"). For a 35-year-old user, different singers (e.g., \"Celine Dion\") might be suggested.

But, you're right, I think, to question where Yahoo! would ideally collect this type of information. MyYahoo may be one place where information like this could be collected. But the patent application gives us a clue as to where they might think the ideal source of this information would be.

Yahoo has filed a series of patent applications that share some ideas, including the idea that concepts taken from user queries could help to create relevance in searches.

Another Yahoo! patent application referred to in this Trend Analysis application could be the perfect vehicle for collecting this type of information. From the Description of the Trend Analysis Patent:

CROSS-REFERENCES TO RELATED APPLICATIONS 

[0001] The present disclosure is related to the following commonly-assigned co-pending U.S. Patent applications: Provisional Application No. 60/460,222, filed Apr. 4, 2003, entitled \"Universal Search Interface Systems and Methods\"; and Provisional Application No. 60/510,220, filed Oct. 9, 2003, entitled \"Systems and Methods for Search Processing Using Clustering of Units.\" The respective disclosures of these applications are incorporated herein by reference for all purposes.

If we look at the first patent application listed, it describes a method for Yahoo! to collect that information:

Universal search interface systems and methods

Here's the abstract:

Systems and methods for enhancing information retrieval and communication functionality through the use of a universal interface that is configurable to interface with multiple applications resident on a user computer, and which provides a persistent two-way communication channel for communicating with search intelligence on a remote system. Sharable, actionable labels and codebooks of labels may be defined by a user. Each label may be defined in a natural language format and may include a mapping to a specific application or set of applications executable on a user system. Transfer of labels and codebooks between user systems allows for enhanced information exchange and retrieval among users as well as information exchange tracking and analysis by a server system.