Log analysis consistancy Options

[Adrian]

I'm currently having a go with the trial version of Net Tracker, to see how it compares a bit to what I've been using before, as I'm now after something a bit more advanced (I don't really think they are going to let me spend Net Tracker kind of money, but I thought it was worthwhile checking out the difference).

Now, I've analsyed some logs, spanning a time period already analysed by my current software (the much cheaper and more basic Weblog Expert). Looking at the results, they are both telling different things.

At a very basic level, Net Tracker is suggesting that for September last year (an easy one I can compare) we had an avergae of 200ish less visitors a day than Weblog Expert says. The trends are generally the same, but still some big enough differences to make me very dubious about placing any importance on running these kinds of apps.

In at least one point Net Tracker is suggesting there were something like half the number of visitors (600ish compared to 1200ish) in a day.

Now Weblog Expert is worth about £50, while Net Tracker Pro is worth more like £400, so I'm guessing Net Tracker should be the one that is accurate. And I know that you can't rely on these kinds of analysis entirely, because tracking a 'visit' exactly in standard Apache logs can be difficult.

But I am a bit annoyed, to be honest, at the diference between the 2. I've paid good money for the software I've already got, I can't see that I'm going to be allowed to spend the kind of money needed for Net Tracker (and I certainly can't afford it myself, or even need it personally), so it seems I'm left with reports that are effectively useless because the information is innaccurate. Assuming that is, that Net Tracker is the more accurate one.

I'm now wondering whether I should even bother looking for another application that's more adavanced than what I already have, but also more affordable than Net Tracker level stuff, because how can I know how reliable it is?

How do other people work round this?

[bwelford]

Adrian, I relate to your problems. I'm not sure what might be a better solution, but I use Funnel Web Analyzer, which is free. I'm reasonably happy with the results. However I cannot be sure that it will have correctly identified which visitors are spiders and which are likely to be 'human'. I use it to compare results over time and so I get some indication of trends.

I find the results most illuminating. Still I know I have minor problems that I haven't managed to resolve. For example, it does not identify the Yahoo! Slurp robot, although it seems to recognize many of the others.

[Black_Knight]

Adrian, the difference can often lie in the way that apps cope with browser caching. As such, usually the lower count of 'unique visitors' is the accurate one. The higher counts usually demonstrate differences in how they assess uniques, or in whether or not they worry over-much about everytime a visitor goes back to a page over whether they went there at all.

[Adrian]

Yeah, never expected them to agree exactly, just demoralising when I've seen traffic double in one programmes reports, due to some great changes we made to the crawlability of the site, and now another app is saying those figures are basically a fair way out

I've already decided I need to change the way I analyse them. Weblog Expert is under powered for some of the stuff I want to know now, and eats RAM like nothing I've seen when processing the logs.

And now, to my shopping list, I need to add not just functionality and price, but also accuracy :roll:

[Black_Knight]

Accuracy just doesn't exist. Your best bet was what you first asked for - a uniform and consistant bias.

Make no mistakes - no form of tracking, nor any combination thereof, is ever truly accurate. Inaccuracy is completely unavoidable because of the way the web works. Come to terms with that fact.
http://www.analog.cx/docs/webworks.html

[Adrian]

Oh yeah, I wouldn't expect any of then to be truely accurate just based on apache logs, it aint gonna happen, and I've never trusted the numbers exactly.

I do expect them to be able to provide numbers that are fairly good though.
Basically means I should ignore any figures about traffic numbers and just look at the general trend lines as a rough guide instead.
Which makes it more difficult to suggest improvements or to give feedback on how successful a change has been in that sense.

Ah well, just have to live with it I guess.

[rustybrick]

Funny thread, I actually wrote a blog entry on this topic a while back named Comparing Web Traffic Between Different Web Analytictical Tools, and that was comparing the same product, just different versions.

I still need to give NetTracker a try, but I am so happy with Urchin and all its add ons.

[Mike521]

a couple of my clients are using hitbox, which I am happy with. I can't say how accurate I think it is because I'm not comparing it to any log analyzers. what I like about hitbox is that it tracks visitors who get a cached copy of the site from proxy servers. AOL does a lot of proxy stuff from what I hear, meaning a lot of AOL subscribers are visiting sites but never leaving log traces.

Another thing I like about it is that all the overhead is on the hitbox server. My company has a dedicated server, and who wants to be bothered with all the overhead that these log analyzers require... sometimes it's not a lot, sometimes it's massive though. then they come out with upgraded software and you need a massive hardware upgrade... I'm very happy letting this be hitbox's headache.

Anyway, has anyone compared log analyzers to hitbox stats?

[Adrian]

Even more frustrating, I'm getting to quite like Net Tracker. I like being able to look at an individual days visits and where they are from, and the detailed view of SE Bot visits, as wel as some of the general stats it produces.

Using it, I think I've just found out why we're not getting more exposure in Yahoo for example. Not sure why it's happening, but that's for another thread I'm about to start

Perhaps if it can help fix that, I can start to justify the expense...... Perhaps....

[Black_Knight]

Another convert in process. Can you see why I say NetTracker is such a serious piece of kit now? The ability to spot something in any report, zoom right in on it, turn it around to view it another way, zoom back up to a different broad view, etc. It somehow feels like it gives you a deeper view. Three dimensional.

[Adrian]

Hehe, yes indeed, serious it most deffinately is.

Certainly makes me feel a bit more guilty about it not looking quite right in Firefox when I look through the visits and see someone using Firefox to have a look around several pages.....

To be able to drill down right into an individual visit and see what that person/bot was doing is quite interesting. And then yeah, as you say, twist it around a bit, look at it in different ways, and then zoom back out to a different view is all very clever. I could sit looking at these things all day

[shalin]

Adrian, I have been into web analytics since over 30 months and understand that problem that causes inconsistency in the number of visits for an apache log file. Unlike W3C apache don't carry that extra bit of information to provide you with accuracy. Typically, if accuracy is mission critical here - I would suggest you to go for a page tag solution. The chances of error there are usaually less unless the solution is buggy.

However, I can explain you some major factors why product tend to lack accuracy.

1. They do not differeniate between a robot visit and an actual human visit.
2. They consider a very short span of time to consider a visitor unique. For instance, if you were on a site for 70 minutes the tool will count that as 5~10 visitors. Some products like Livemon allows you to configure the session time out period. So if you were running a gaming site where users can stay for hours on a single page playing a flash game - session timeout close to 240 minutes would give you the best result. And for a small site 30 minutes would be ideal.
3. Cross check with the number of the other reports within in the same product for instance total entry pages + total exit pages should roughly give you the total real visitors to your website. Another report which can help is the total Site Navigation or Site Path report which will tell you the total number of site paths followed. Again this should sum to number of visitors. Leave 10% as margin of error.
4. The other way to try few more products and the one most close to the avg of all would be the one to pick.

I would welcome you to try our product Livemon - http://www.tenmiles.com/livemon/ - it's an server side software log analyzer solution.

regards,
Shalin Jain
http://www.tenmiles.com

[sanity]

Accuracy just doesn't exist. Your best bet was what you first asked for - a uniform and consistant bias.

I'm coming late to this but what Ammon said is so true.

What I've found is that as long as you use the same program for a site you can at least see reasonably accurate trends. The other thing to measure is the conversions - orders, number of sign ups, leads and so forth.