It's all about the signals, What the Search Engines Look For Options

[bragadocchio]

Hi all,

One of the curses of having an English degree in literature (who would think that would come in useful) is that you pay too much attention to language patterns, and the way people speak. I noticed for instance, that it was pretty common, at the recent SES in San Jose, for search engineers, giving presentations, or in conversations tend to start many sentences with the word "So," as in:

"So, if you think about..."

"So, if we were to...."

"So, you have this..."

I guess that's part of the cost of presenting hypothetical situations and theories to others.

Another word that kept on showing up in the presentations and conversations from these search engine representatives was the word "signals." Mike Grehan and I were both in the audience during a presentation titled "Link Building Q&A" and Mike wrote a column for Clickz which emphasizes the word:

Sending Signals to Search Engines

I liked this quote from Mike's article a lot:

I feel certain, though, that search engines can glean a lot more temporal analysis related to linkage data when it's combined with the other signals they talk about, such as burstiness, end-user behavior data, social search, and personalization.

I do think that search engines still place a lot of reliance on links, but am convinced that understanding user queries - like seen in our thread on the AOL data, is playing a larger and more important role in determining relevance. There have been a number of mentions in recent white papers and patents which mention search engines collecting information from ISPs about user behavior. I wrote about a patent that was granted last week for Overture/Yahoo which describes some ways of collecting and using that information in:

How a Search Engine Might Use Information from an ISP While Capturing Traffic Flows

I think one of the most important aspects of that is the idea that it is a "traffic flow" that a search engine is looking at, and they can capture information over time, which shows how some sites are more popular than others and are attracting traffic in response to searches and other methods of getting people to a site. What role might that have in ranking or reranking a page?

We've talked about Google's patent involving Information Retrieval based upon Historic Data in a number of threads here. Here's a paragraph from that which discusses some signals based upon this type of information:

[0107] In addition to history of positions (or rankings) of documents for a given query, search engine 125 may monitor (on a page, host, document, and/or domain basis) one or more other factors, such as the number of queries for which, and the rate at which (increasing/decreasing), a document is selected as a search result over time; seasonality, burstiness, and other patterns over time that a document is selected as a search result; and/or changes in scores over time for a URL-query pair.

This is a paraphrase of an answer that I received when asking a question during that Link Building Q&A session:

There are a lot of "signals" that search engines look at when determining relevancy and rankings.

Thinking about an information stream like this, I think that we need to expand our thoughts of rankings and relevancy beyond what we commonly call "on page" factors and "off page" factors to another additional category which looks at how our sites fit within that information stream. How well do our sites perform when we look at them from the perspective of users clicking on pages dealing with topical subjects?

This paper talks about burstiness some more:

Bursty and Hierarchical Structure in Streams

One of the reasons why I like blogging so much is that it gives me a chance to write about subjects that are topical, and that will attract viewers based upon what is popular and interesting today. By keeping an eye on conversations going on upon the web within my targeted niche, and taking part in them, I become part of that information stream. For example, Google recently acquired Neven Vision, a company that makes face and object recognition software. I conducted research at the US Patent Office and wrote about the patents which the company had been granted, which helped my site become part of the topical information stream.

This is something that can be done in many different industries, by keeping an eye on what people are thinking about and talking about. It's one of the potential benefits of blogging.

What signals do you think are becoming more important, and how are you taking advantage of being within an information stream?

[earlpearl]

I saw that Barry referenced this at SER and thought I better get over and read it PRONTO!!

Haven't delved into this topic...but the volume of searches on certain url's was interesting to me, especially as it related to competitors and with regard to original references to traffic from the early patent on google's ranking algo.

I've always been bothered by that reference in so far as how can we webmasters know what kind of volume of traffic different url's are gathering. (excepting a peek at a big mistake like the AOL phenomena).

Moreover how does Google know if traffic is coming from other sources, such as links from different websites and traffic from other SE's let alone bookmarked traffic.

Google Analytics gives them an addditonal view into the "guts" or insides of web sites to view this....far better than any competitor.

Actually only scanned the article and need to follow the links....but thanks for addressing this issue, Bill. I've wondered about the volume traffic aspects of serps since I saw this first mentioned.

Dave

[bragadocchio]

Hi Dave,

I've been wondering about how "popular" sites get into search results, too. Recent news articles on a topic sometimes make their way into the top ten rankings for terms without many links to them in a search engine's index. And then they sometimes disappear.

Is that tied to this notion of burstiness, and of looking at traffic and queries and clicked results in data streams?

[Chris Boggs]

Great post as usual Bill!

...think that we need to expand our thoughts of rankings and relevancy beyond what we commonly call "on page" factors and "off page" factors to another additional category which looks at how our sites fit within that information stream. How well do our sites perform when we look at them from the perspective of users clicking on pages dealing with topical subjects?

Wonderful comment. This is the "trend spotting" that is often promised by SEO's and SEM's, but is rarely performed properly. Employing automated search for burstiness could lead to so much more success on both the paid and organic side of things, particularly with a regularly crawled website that has existing content than can be updated or linked to new content with hot terms and ideas, and their semantic equivalents or neighbors.

Your questions require some thought and self-evaluation, and some others may reach the same conclusion as me: "Am I swimming in the stream or just dipping my feet into it?"

In my opinion, two signals that are becoming more important to send are Brand information and Name information. Branded searches should be monitored for (hopeful) increases, especially after high periods of burstiness bought on by multiple forms of online and search marketing, some event or the press. I would think that branded terms are most likely to gain in burstiness at a more regular interval or pace than non-branded terms, with the exception of a fad. If the added bonus is that the fad is named after your brand, kudos to you and prepare for nice rankings, probably. I feel that the algorithms will detect this as you suggest, especially if it leads to many links to the site that "owns" the brand. Monitoring for spikes in activity and putting out more brand-intense content around these times can help to protect the brand from competitors in the rankings, probably.

Name searches can be important as well. Every Corporate website should have at least the names of the senior staff somewhere visible on the site, if not a full bio. That way when someone speaks and/or writes at a popular conference for example, and their name gets mentioned in various online publications, any subsequent increase in searches for that name will yield the proper site. This has to be monitored, and of course a link from the bio page to a recap of the presentation might be a good idea. Another thing is that in some cases the person that is being quoted or covered has the opportunity to "name his or her own link." With this idea of ebbs and flows in link popularity and link gaining, the more links that can be directed to the bio or the recap at the organization's site, the better chance that site may have to rank on the topic that the person covered. I am rambling, but I think you may agree with this?

Are these "signals" the most obvious that can be considered important to send: brand and name information?

[bragadocchio]

Got a kick out of this blog post:

Extracting Meaning from Bay Area Sentences

Thanks, Chris.

Your questions require some thought and self-evaluation, and some others may reach the same conclusion as me: "Am I swimming in the stream or just dipping my feet into it?"

Sometimes you have to be the ones starting the conversation, and creating the buzz, too.

This video from Yahoo, presenting Jon Kleinberg of Cornell University talks about how information flows through social sites, blogs, emails, and has some interesting ideas in it:

Diffusion and Cascading Behaviors in Social Networks

I think those are excellent points regarding brand and name information.

[Chris Boggs]

Bill thanks for the links. That first one is great, right?

I have just finished watching the first of the five clips in the Kleinberg series. Very forward thinking, and in my opinion is an insight into the great head start that Yahoo! has on the social front. I'll watch the rest and get back with some more. I hope some other's will join in and share some ideas...

[SEOigloo]

Dear Bill -
I read you post, and read Animas post, and I just went and wrote this one:
http://www.solaswebdesign.net/wordpress/

I hope this isn't considered self-promotion, but you have struck upon a keen point with me - the vagaries of the English language!

I'm so glad to know I am not the only one who thinks about HOW people say things!
Thanks, Bill!
Miriam

This post has been edited by SEOigloo: Aug 25 2006, 03:55 AM

[bragadocchio]

Nice blog post.

I'm thinking about how the word "so" was used in those conversations, and I think that it was more a signal that what was being said was a hypothetical more than anything else. The equivalent of perhaps starting a sentence with "Imagine that..." or, "We can speculate..."

In a recent Matt Cutts blog post, SEO Advice: Writing useful articles that readers will love, a lot of folks in the comments section start discussing the words that Matt uses rather than the advice he provides. I think that they might have cared a little too much about the signal itself, and not the information that it was carrying.

But the search engines do look at those signals. They consider the words used, and consecutive sequences of words (n-grams), in understanding messages sent.

I referred to the Jon Kleinberg video, and I agree with Chris on its importance. Getting past the math in the video, the message in there is important - that trying to understand how information spreads from person to person may be helpful in finding that information for someone who might be looking for it.

I'm so glad to know I am not the only one who thinks about HOW people say things!

I think that's part of what has me interested in seeing how a search engine tries to understand and interpret signals.

Afterthought

Don't miss this excellent spoof (I think it's a spoof) of Matt's post from Dave Naylor - Spam: Scraping useful articles that Search Engines will love