Keyword Research and Related Tools, What do you use? Options

[jendy]

Hi,

I'm looking into revamping my keyword research and analysis process and am considering investing in some tools to help the cause.

Currently I just use nichebot but realise I need to get a bit more serious.

I have been looking into the following and wondered if anyone has any useful comments on them or recommendations of other tools / services they use and why.

Research

- http://www.keyworddiscovery.com $390 per year
- http://www.wordtracker.com $262 per year

Analysis

- http://www.keywordsanalyzer.com/ $97

Thanks,
jendy

[A.N.Onym]

I have been using free trials both of KD and WT.
However, since WT has introduced email free trial veritification system, I only use trial KD and it is more or less enough if you know what you are looking for.

[Black_Knight]

Clipboard, Pen, Questionaire, and willingness to ask people in your target demographic is under $10, or priceless, depending on point of view.

Keyword Research (In the real world)

If you were looking for this (information/product) how would you go about it?
* Ask friends
* Use Search Engine

If you found yourself searching for it online, how would you do this? What search engine would you use? What words would you search for?

What other ways might you try to locate this product/info?


That's real keyword research. Look how much more it tells you than that some people, perhaps none of whom are in your real demographic, perhaps none of whom have a credit card, use keywords X in search engines supplying data to tool Y:)

[FP_Guy]

Wordtracker has the best results, but only indexes search phrases for the past two months.
Trellion (keyword discovery) has almost as good results, but keeps their search queries for an entire year.

I use both and use only the keywords that are favorable in both.

[jendy]

Hi A.N.Onym:

I'll have a look into the free trials, It has always been in the back of my mind that you can conduct this area successfully combining free services.

Have you tried http://www.nichebot.com/, its quiet a good free service.

Hi Black_Knight:

Good point.

It's very easy to forget about manual techniques in the technology world - I certainly do!

I would imagine that using that technique, the quality of the research would be excellent (targeted and exact) and the only down side would be the time involved.

I'll definitely give that some more thought, thanks.

[A.N.Onym]

True, WT sometimes has unique great keywords, but sometimes it lacks the volume of searches for a particular phrase to estimate its competitiveness.

I'll be probably using nichebot since it returns fast feedback on queries, but I'm still disappointed with WT a tad. Not that I am a valuable customer, more like a constant visitor

Doing extensive keyword research, either with people or tools, requires quite a lot of time. It probably doesn't make sense if you are not in a competitive industry or just want to know basic keywords (like 'blue widget' or 'green widget') to get started.

[Black_Knight]

The time taken can seem extreme, but on the other hand, the time saved in testing less-proven keywords tends to be many months longer.

Although a keyword tool may take only an hour or so to properly mine for an average site, it will take months to perform fair analysis of effectiveness.

Compared with a day or two of focused research with questionaires, or even user-testing, which also gives so much more information on where else to advertise, often the tools are actually by far the less efficient use of time. The tools are simply easier, just like instant coffee is easier than fresh ground and percolated. But we all know where the quality lies.

[dgeary9]

Let me add support for BK's wise words.

A good friend of mine works at a small technology company that didn't want to cough up for user research. He suggested they do some informal research - everyone on the senior management team called 3 recent customers and asked questions like "how did you hear about us", "why were you looking", "why did you decide to buy". Then they had a meeting. Out of that informal research came:

1) a good sized budget for user research
2) a very different focus for their website, including SEO efforts
3) a brand new product
4) a deep awareness that customers think differently than employees!

As a data geek, it pains me to say this , but sometimes there really is no substitute for actually talking with people.

[Ruud]

If I can't do it otherwise I will ask people around me. But I prefer to ask them if they have a minute or two and would be so kind as to help me find "a program which does...", "a review for...", etc. Seeing them search and watching the kind of revisions they make is very interesting and helps me better determine which trail(s) I could follow.

I use several online tools to find related searches etc. Then I use WordTracker to guesstimate volume and thus where to start putting my focus.

After that I start to re-enforce searches which led to conversions and follow-up on searches which either converted or give me a hint what people might use to find this XYZ.

[A.N.Onym]

Ruud, how do you track which keywords led to conversions?
ClickTracks, visitor path analysis from site stats software or some other tool(s)?

[dgeary9]

Nettracker Lite was the best for this, and it has vanished in the Unica buyout. sigh...

ClickTracks has a nice report that shows short visits, which I actually find really helpful at the keyword analysis stage. I divide keywords into 3 categories: 1) keywords that generate conversions, 2) keywords that generate interest (decent length visits - these may be future sales or perhaps I can tweak the site to improve conversions), 3) keywords that generate a high % of short visits (probably only good for adsense sites ).

[Ruud]

A.N.Onym, it depends on the site/project and whether or not I'm the one installing software and scripts or not. NetTracker (for deep views) and ClickTracks are my favorites together with some home-rolled scripts. I've also used Google Analytics and Indextools (very nice).

The thing to remember is that if one doesn't have any many to put behind this kind of pays-for-itself research (and remember, something like Indextools can start from 20 bucks/month) or if you find Google Analytics too complex, you can still get this type of information in other ways.

If you know how to start sessions you can do a quick script to simply track who comes from where and eventually did what.

If all else fails, make different landing pages and put a free, invisible tracker on it like Statcounter.

Nettracker Lite was the best for this, and it has vanished in the Unica buyout. sigh...

Yeah, that used to be pretty sweet. Some download site which host their own files might still have it. Archive.org has the odd file too.

Have you tried ClickTracks Appetizer? Haven't had time to try it out to see how useful it is to recommend.

[A.N.Onym]

Thanks for the reply, Ruud.

Agreed, one can track conversions in many ways.

I have tested ClickTracks Appetizer and I was pleased with it.
I only tested the paths visitors go to and from a page (it is shown as an overlay on a page, with percentages of click-throughs shown near every link). I intend to continue using it for my sites, too.

It says it has more imptions, like tracking where visitors came from to an action page (submit, order), I'll be trying this out when I get more visitors and, in fact, conversions.

Though I was testing on its Web Analytics day, I didn't use any other options.

[Black_Knight]

how do you track which keywords led to conversions?

Thoroughly.

Seriously, the more of your conversions, even ones where the customer spends weeks considering the purchase, and eventually orders by telephone, well, the more you can attribute right to the specific campaign and keywords the better.

How to do it depends on the effort and creativity you apply.

At the very least, you should at least attempt to cookie users to keep track of how they first found your site, and possibly how many other times they came by, and by what methods.

Alternatively (or better yet, additionally) create an incentive scheme that allows customers to get extras even on their first purchase - provided they create a virtual loyalty card - meaning they'll log in before purchasing. That way you can have them 'refer a friend' for a small amount off their purchase, or an added free gift with first purchase, which both encourages them to spread the word, and enables you to create an incentive to log in to your tracking system.

Off Topic

Human nature can work in your favour if they choose to be sneaky and refer themselves, since they will now have provided you with assurance that they'll log in - they obviously want that incentive pretty strongly

You may well find (especially with high-ticket items, or purchases that require more thought in deciding on) that the period from first encountering your offer, to finaly accepting it, is many months in duration. This is most notably true of the kind of purchases where someone will want to consult their spouse, or ask friends opinions, etc.

For all this, you really want a bespoke tracking solution a lot of the time, as most of the commercially available pre-packaged tracking solutions are rather generic, and focus too much on sessions, rather than on interactions.

I highly recomend adjusting your server to log to a database rather than to a flat text log file. That way you can adjust the database to your own requirements. IIS servers provide an option in the setup to do this, while Apache servers can use any of a range of mods to log to an SQL database such as MySQL or PostgreSQL.

From then on, you can work with your stats through standard SQL queries, and even have your site dynamically adjust to the information in your tracking database.

To track offline sales, well, you need to think creatively. Special offer vouchers with reference numbers can be used to trace any order back to the database item it was referred in/by. Just have your tracking database system tie the reference number to the data for that visitor.

There is no limit to ways to track conversions but your own creativity and desire to do so. The effort is very soon justified by the far more accurate picture you develop of just how your customers shop, decide, and order.

Tying each and every sale back to the keywords used, on which engine, etc, really can show you the classic 80:20 rule in action, where many of the highest-profit deals come from some very specific keyword combinations. It can highlight precisely which words indicate a high-value customer from a lower-value customer.

Then add ways to track repeat custom, and customer referrals, and you start to build up a truly invaluable picture of your customers, and can start to manage these things you can now measure. You can find just what type of incentive rewards scheme gives maximum ROI in repeat custom. You can measure the effects of incentivising customer referal through loyalty programs or free gifts, etc.

[bsaric]

I use all tools above + finding public statistics, you check for 10 or 20 competitor where they are hosted, check if they hosting has public statistic and just open they statistics It works all the time for me. Or just try something like this:

domain.com/stats/
domain.com/logs/
domain.com/awstats/
domain.com/webalizer/
...

you can also find old sites, some have counters on page, extreme or something else

research

[A.N.Onym]

Thank you for such an inspiring post, Ammon.

However, I was expecting to learn about established methods of tracking conversions, like ClickTracker.

Apparently, this is relatively new and everyone's free to do whatever he wants here. Fine by me, I suppose.

bsaric, I don't think it's ethical to hack into others' stats, even if you don't need a password to do that.

In my mind, just stats won't get you much. Site usability and factors, influcing conversions, matter much more than that.

Btw, there is Competitive Intelligence from Trellian to learn everything about a competitor.
It costs about $60 per domain and $250 for setup as far as I recall. Gives all the keywords, PPC campaigns, links, a list of sites that drive the most traffic to the domain, etc. Pretty good if you do need that.

This post has been edited by A.N.Onym: May 30 2006, 07:19 PM

[bragadocchio]

Keyword tools are fine and good, but I don't really start using them until late in the process.

Ammon noted the importance of user testing, and focused research. Of course, he left some unsaid about defining client and site objectives, understanding who the targeted audience or audiences are, capturing information about perceived and actual competitors, finding information about industry journals and trade publications, reviewing any past marketing efforts (successful and not so successful), developing an understanding of metrics to be used to achieve success, and exploring the niche as thoroughly as possible.

Those are all steps that can, and possibly should be undertaken before plugging words into tools.

Before even beginning to try to locate keywords, I try to develop categories (conceptual categories) related to the subject and objectives of the site, and of the people who will hopefully be visiting it. Categories can have sub categories, and so on. Define what the site does, and what visitors can do when they find the site. Consider how information will be arranged within the site. How will information be arranged to help the site fulfill its goals? What concepts are covered and related from industry publications, from competitor's sites, from user forums and emails and so on.

Then begin the search for keywords.

Tracking conversions means understanding what a conversion actually is for the site in question. Is a conversion a phone call, or a sale, or the downloading of a certain whitepaper, or registration of a newsletter?

[bsaric]

bsaric, I don't think it's ethical to hack into others' stats, even if you don't need a password to do that.

And why this is not ethical? Most of public statistics you can find with Google, so its unethical to see their stats?

This post has been edited by bsaric: May 31 2006, 02:02 AM

[A.N.Onym]

Well, the following

domain.com/stats/
domain.com/logs/
domain.com/awstats/
domain.com/webalizer/

assumes that you just check out the competitors stats without using publically accessible stats.

Though AWStats is free and open source, I don't think it is considered public. Moreover, I don't think your competitors consider their stats public.

Hacking into one's AWStats stats is as ethical as entering your neighbours bedroom when you please.
Is the backdoor open in his house? Most likely. Will he enjoy seeing you there? Don't think so.

Technically, even checking out public statistics is not fair sport, but since its public, you can do that.

This post has been edited by A.N.Onym: May 31 2006, 02:22 AM

[bsaric]

Hacking into one's AWStats stats is as ethical as entering your neighbours bedroom when you please.

I think this is really drastic example . Its not like that.