43 replies to this topic

[randfish]

Let's say I was launching a site targeting the chiropractic industry. How would one go about determining, on the keyword research side, how big (or small) the "tail" of the search terms in that industry are?

Basically, I'm looking for a methodology to measure the value of being listed in the top for a few, hig volume search terms in the head vs. the value of having rankings for the thousands of smaller terms making up the tail. Other than simply using intuition and logic, I can't seem to think of a good way to do it. I know that the real estate industry has a very long tail and the webdev industry has a very long tail, but I don't know about more specific or niche fields... i.e. chemical engineering, commercial re-refueling, veterinary insurance, etc.

[earlpearl]

I recently saw this question elsewhere. On the one hand I refer you back to DD's thread about the power of 2ndary keywords at seo chat. The long tail works. (and I don't know what the topic of her site is).

In practice if I add content to my site on a vast number of topics and kewords that do not specifically cover the topic, but are long tail and add traffic and some of that converts... great. And the cost of adding and carrying content is less than the revenues...then it pays.

The caveat on that, is Black Knight's comments about measuring resources in site development and work versus the payback.

Typically, one never knows the payback on long tail, or any keywords until you have made the effort. ( Of course Google is learning this --due to analytics--but that is another issue.)

Finally I had a recent conversation with a webmaster that has dramatically increased conversions/sales. How. He added tons of content on his site. Much of the content covers issues not directly relevant to the main point of the site...but it fills in tangential information. Much of the content covers topics that are not highly competitive so the site keeps showing up in serps for all kinds of phrases.

People searching on lots of topics that are distantly connected to the main topic...keep seeing the web site. Total conversions are up about 50% from last year. Probably the conversion rate is way down.

I don't think you know till you do the work. Secondly, my experience is that consistent work on long tail comments will find some real winners. They may not be high volume traffic phrases but they may be killer conversion phrases. You definitely won't know about them unless you do the work.

Dave

[rmccarley]

I asked about this recently and EP was kind enough to give me the same advice as the above. End result though is test and see. It would be great to cut about 80% of the tail out of my work so I could just beef up on the juicy terms...

[earlpearl]

The lesson learned from the other webmaster was that he added all this voluminous content after studying results on his sites for over a year. He put a lot of things together after reviewing all the traffic from search phrases and links to the site. Then he went after the long tail impact. I have heard this same story in different ways from other webmasters...and I have applied it. It works.

[AbleReach]

What does your intuition say? ;-)

Just guessing, but I'd imagine that providers requiring professional certification may have a shorter tail with less WOM and diversity of paths to conversion. However, anything touching on people skills, personal taste, fame or controversy would probably have a longer tail.

To approach an "industry," I'd want to get a look at what works for that industry's longer standing niche publications. What concerns re-surface regularly? Are they adequately addressed? Would these concerns be of more interest to customers or colleagues?

Would it be helpful to look at how a customer hears about and chooses who to hire or buy from?

I'm a fan of looking at real life FAQ, if at all possible. What does the end user kvetch about or dream of? Are they asking about concepts or facts? For example, concepts like "why chiropractic?" could lead to a diverse collection of content-rich keywords. Facts like "how many years in business?" or "what are your professional affiliations?" would be less related to content than trust. Bridge topics like "chiropractic after injury," or "low back pain" may include both concepts and facts, and could be good long-tailed link fodder.

This is an interesting question, Rand.

[EGOL]

Hey Rand.... Instead of wondering about this... Just attack both of 'em. lol


No, I'll give you the answer..... the long tail on chiropractic is *infinite*... and there are massive misspellings of "sacroiliac".

Edited by EGOL, 20 March 2006 - 09:09 PM.

[earlpearl]

Hey EGOL. long time no see. Spoken like a true mod. congrats.

[EGOL]

The root keyword for one of my sites has an proper term, and a slag term and a ignorant term (which has a tail on it that you would not believe). Lots of people with similar site go only after the proper term. I know about the other two because I spent 15 years at a government agency listening to the average person trying to explain something simple but they didn't know what words to use.

If you want to know about the chiropractic tail, ask the person at one of those offices who takes the incoming calls!

Edited by EGOL, 20 March 2006 - 09:19 PM.

[sanity]

I find running an AdWords campaign using broad match and then running an analytics program to see all the varied phrases people come to the site from works well.

It's a good way to find unique, niche terms as well as weed out obvious duds.

[eKstreme]

This is a very interesting question Rand. Here are some comments and a hot-off-the-press answer.

1. I think we need to formulate the question better. IMHO, it is better to ask about the tails of a keyword, not an industry. The difference is subtle: Each industry has a set of keywords that define it, and each would have varying tail lengths. In a way, it is wrong to ask about the industry-specific tail, because there are many independent tails.

Further, the broader the industry, the more keywords there are. If you are targeting the keyword "software development", then you will see a different tail length when you compare it to "ASP developer" or "C# developer" etc. These keywords, although all related to software dev, define very different industries.

In short: we need to define our industry, i.e., our niche within a realm of keywords. As we should do anyway Of course, our definition can be very broad.

2. Now suppose we've defined our niche as an industry defined by a set of keywords. How can we estimate the tail lengths of each of those keywords?

I propose the following method:

a. We ask our favorite search engines for all pages related to our keyword. For example, we ask real estate to see all pages related to real estate.

b. We next ask, which pages really are related to "real estate". Notice the quotes? This gives us another number.

I propose that the ratio of a/b is an estimate of the tail length of the keyword. In essense, the ratio is this: of all the pages you think are related to a certain keyword, which ones are definitely related to that keyword?

To rephrase: If you plot the power graph showing the long tail, the x-axis would be the number estimated in "a" above. The estimate "b" is the percentage of "a" along the axis that overlaps the search term. If the overlap is minimal, then there is no tail. If the overlap is significant, there is a long tail.

Maximally, the long tail ratio is 1, that is a/b = 1. In the case of real estate, the ratio is almost 1 (!). In the case of veterinary insurance, the ratio is very very small. By this method, I think we can assume that the keyword "real estate" has a long tail but the keyword "veterinary insurance" has a very short tail. This makes sense intuitively.

I would love to see how other known short-tail keywords perform using this test.

One final remark: the necessary disclaimer I'm writing this at 2:50 AM because I can't sleep. It is likely I could be very wrong and might lose whatever respect you guys have for me. Please be gentle when you shoot me down.

Having said that, good night and happy tailing.

Pierre

[EGOL]

Maybe estimating the length of a KW tail is easy. You plot a graph with frequency on the Y axis and keywords (numbered 1 2 3 4 ..... ) on the X axis.

The area under the curve is the search volume and the point at which the curve hits the X axis is the tail length.

I had calculus over 30 years ago but remember estimating the point at which a curve would finally converge on a value. There is certainly a formula out there to fit this curve and estimate that point.

My bet is that lots will be asymptotic as there are infinite terms that people will think up. If I run my logs with one day of data - for a tiny site - the list of querys will be fairly long... and it always amazes me that the terms grow in number with every passing day - even if that log accumulates for months.

Edited by EGOL, 21 March 2006 - 12:27 AM.

[eKstreme]

Maybe estimating the length of a KW tail is easy.  You plot a graph with frequency on the Y axis and keywords (numbered 1 2 3 4 ..... ) on the X axis.

The area under the curve is the search volume and the point at which the curve hits the X axis is the tail length.

I had calculus over 30 years ago but remember estimating the point at which a curve would finally converge on a value.  There is certainly a formula out there to fit this curve and estimate that point.

My bet is that lots will be asymptotic as there are infinite terms that people will think up.  If I run my logs with one day of data - for a tiny site - the list of querys will be fairly long... and it always amazes me that the terms grow in number with every passing day - even if that log accumulates for months.

Well you don't need calculus to find the x-intercept (as it's called): where the curve crosses the x-axis is when the y-axis value is zero. So if you have a formula that describes the graph (you can best fit one given a model), then you can set y=0 and solve for x.

As for the asymptote issues: given that our data set is finite, no matter how long we accumulate data, then we will always find an intercept. The problem of course is that the estimate will keep changing all the time as we add more data. I'm pretty sure there is a statistical method to counteract this, but I kinda slept through stats lectures Best sleep I ever got!

[AbleReach]

Wouldn't the ratio of "real estate" versus (no quotes) - real estate - have more to do with how often the two words are used together? "The real value of his estate," is different than "the value of his real estate."

If chatter on a topic is an indication of "tail," then would variety and repetition of associated keywords be an indication of the tail of that keyword?

How would terms with a lot of synonyms be different than those that are usually said one way?

[rynert]

If the site is live make sure there is a search box where you are caputuring the 'searches' done - a good starting point. \

As Egol mentioned, also check the log files, daily, and where appropriate produce content for anything that appears numerous times.

[bragadocchio]

If the Design Observer can call Wilson Pickett a Design Theorist, I can call him a marketing strategist. Especially when he said things like this:

"You harmonize; then you customize."

When conducting keyword research, I initially try to avoid getting too caught up in specfic words or phrases, and try to capture categories instead. It's a method that helps me find the general categories that I need to address, and then review more specific ones that I may miss if I focus too much upon keywords themselves.

Taking the chiropractic industry as an example, I'm going to try to find a number of sites that deal with the industry, articles, forums for doctors of chiropractic and for patients, news stories, and more. I'll explore those not so much for keywords, but rather for categories.

I might come up with a list for the chiropractic industry like this one:

Practitioners
Patients
Chiropractic schools
Licensing
Alternative health
Equipment

The list of categories might be bigger than that, but I'll leave it there for now.

I'll then take those bigger categories, and try to create subcategories from them. If another broad category reveals itself to me while I'm doing this, I will add it.

Lets take "patients" as a category. Subcategories might include:

Fears about chiropractic
Symptoms of patients
Public figures who see a chiropractor
Concerns for the young
Concerns for the elderly
Work related injuries
consumer education
Athletes and chiropratic
Patient education

After I come up with a list of these subcategories, I might drill down even further. I listed "work related injuries" as a subcategory. Does it make sense for me to look at different types of work, to see if the types of ailments that might be helped by chiropratic approaches involve some jobs more than others? It probably does, so I would start looking at material involving work related injuries, and chiropractic.

Some keywords, and keyword phrases may suggest themselves to me as I'm collecting categories, and I might make a separate list of some of those. If the list starts getting big, I might see if I have categories that match those. I would try to sort them into my existing categories, or even create new ones.

But my focus is defining my range, and making sure that I expand upon it intelligently, without getting lost focusing upon a handful of words and phrases.

Once I think I have the essential categories, and subcategories, and even smaller categories, I'll start focusing upon keywords and keyword phrases, and alternative and related words. I've got the harmony, now I customize.

[Black_Knight]

Nice post, Bill (like you do any other kind).

In terms of size, the long tail is indeed effectively infinite. People can always add one more word to a query, or use a different order to their words.

It was not long ago that one of the original guys at Excite, who has recently joined Yahoo, was discussing how the long tail was visible even in the early and mid nineties. He explained that although the top few hundred phrases drew immense traffic, around 97 percent of all enquiries on any given day were not for those types of terms and instead were far more likely to occur only once in a few million searches. He went further and gave us a hard statistic that really shows how big the tail is.

If they were to calculate the average keyword occurrence in any given day, the average occurrence for all phrases came down to 1.1 occurences. Despite the top thousand or so keyword phrases occuring hundreds of thousands of times, and despite the SEO stats of the day telling us that one and two word searches were the norm. If you picked any given search made in any given day, it statistically almost certain to be unique to that day. 97% chance.

[rynert]

If you picked any given search made in any given day, it statistically almost certain to be unique to that day.  97% chance.

Out of curiosity I just looked at a random day on one of my sites and of 340 phrases in the logs, 90.5% were unique on that day - not quite 97% but close enough to support the figure

[earlpearl]

Fascinating discussion with both theory and practice.

This has helped frame and articulate my efforts. I've approximated what Bill described. But first some stats.

To date for March through the 20th. 2022 keyword searches. 1,069 individual phrases. Only 143 phrases were used more than once leaving 926 phrases that hit the site one time. (ooph!) The most active keword phrase - 216 visits - about 10%. Certainly a long tail.

Of the keyword phrases I counted about 700 that were generic industry phrases from the first 70 terms. A more critical keyword phrase in my case is generic industry phrase with a geo term (state/city/town names) These are the best conversion phrases because it is a local site. (can be anyone of 3 states - 3 cities - and a myriad of towns).

Typically the industry/geo terms total about 70-80% of the industry total. If that were the case this month than about 60% of the traffic would have a very high (industry/geo) and high (industry) level of relevancy to conversions.

Of interest, the highest single phrase with an industry/geo phrase was 10; about 5% of the single highest industry phrase. That is about typical. What is difficult is that the best conversion phrases are the industry/geo phrases. Frankly I could have lots of individual phrases being critical conversion phrases...and darn if they don't come in 1's and 2's.

What's the point?

I currently analyse keywords from a categorization basis a la Bill and Wilson Pickett's ( ) suggestions. I wish, though, that I had come about that via learning or reading from Bill's suggestions before spending well over a year looking through endless analysis.

so if nothing else here is a plug for Bill's suggestions.

[bwelford]

Yes, as usual, Bill has got it right.

I go along with the direction of thoughts here. Just a further nuance I might add from my own researches. Any given multi-keyword phrase will see all sorts of variants used by searchers. The order of words may change, plurals may be used instead of singulars, etc. Of course each different variant will give a different ranking of the entries in the SERPs.

That's why the category approach is so much better. I must admit as I read it I thought about mind-maps as a help to the creativity process involved in this approach.

[Respree]

I'd never heard the term 'tail' in the context of keywords until yesterday (are you guys making these words up?).

I couldn't help but think, doesn't this fall under the 80/20 rule? Does it make sense expend 80% of your time/energy trying to market thousands of keywords that will only produce 20% of your results (traffic, conversions, etc.)? Or am I mistaken in my assumption that it's only 20%?