A|b Split Test, How does this affect search engines? Options

[Rhea]

Out of curiosity, if we're running an A|B split test on the entire site, not just one landing page (like Google' web optimizer), how would this be seen by the search engines?

Thanks!

[Jozian]

Need more info, Rhea. How are you implementing the test? Different page names? I think way you do it will be critical to determining impact.

I would like to know what the SEO gods here think on this as weel - getting ready to do my own.

-Jeff

This post has been edited by Jozian: Apr 2 2007, 03:07 PM

[Rhea]

Using cookies to split traffic half and half (when needed for full site, sometimes just certain areas) and just for new visitors.

[Jozian]

So you are landing and then redirecting based on cookie... or are you dynaimcally changing the page based on cookie value? If url and page name are the same, you are good, else...

1) I think that the redir could hurt your organic position/PR - how much I'm not sure.

2) If you are bidding on keys through Google too, and they check and find a redir, the key could be dropped at least temporarily.

But, if you are only doing the testing for a couple days, I would think it you would be fine. If you plan on doing this a lot, you need input from someone higher up the organic food chain than me!

What say you SEO gods?

-Jeff

This post has been edited by Jozian: Apr 2 2007, 03:31 PM

[Rhea]

URL and page name are the same.

I'm basically fishing for possible solutions to a bigger problem. I think Google is confused about our site and I don't know how to make things clearer. Sigh.

[phaithful]

Typically with most A/B or multivariate testing, it's usually best practices to implement changes via javascript to reduce your search engine impact.

The major search engines don't index or execute javascript (at least not on a regular basis) which makes it a great way to conduct your testing without impacting the content.

In addition, if you are using cookies to divide up your traffic, then you'll have less to worry about. The major search engine bots won't accept cookies, so the challengers that you're testing won't be displayed to the bots... only the control will be exposed.

For Example: I want to test some UI treatments; one with a red ad (Challenger A), one with a blue ad (Challenger B ), and one with our standard green ad (Control).

At the server level I distribute 10% traffic to Challenger A, 10% to Challenger B, and 80% to the Control.

The user arrives at my page somesite. com/webpage.php. Now I cookie the user with a unique identifier to make sure they always get the Challenger A example until they've performed a conversion action. In this case a conversion action would be clicking on the advertisement.

Based on the cookie, my server provides the appropriate javascript file to do a javascript.write, placing the Red advertisement over the standard Green ad (Control).

So the user sees the red ad unless they have javascript disabled.
-----------
Rinse and repeat for Challenger B, the blue ad.
-----------
Rinse and... for the control you won't serve any javascript file.

Now, you're covering yourself via javascript, which the bots don't read, and then you're protecting yourself with cookies, which the bots don't accept.

If you want to be even more cautious, you can serve your Challengers only to User Agents that don't identify themselves as search bots. Some people consider this cloaking.... but I believe this to be a gray area because your intent is not to deceive. You can even take this further by cloaking via IP, but it shouldn't be necessary.

This post has been edited by phaithful: Apr 7 2007, 07:25 PM

[Jozian]

phaithful-

Thank you for sharing that detailed testing methodology. I like the logic you use.

-Jeff

[AtlasOrient]

JavaScript certainly makes sense. If you're interested in a very exhaustive, yet expensive, solution, take a look at Offermatica.com

[phaithful]

Offermatica is a good one. So is Optimost. Both of which will cost you a pretty penny depending on your traffic levels.

The free solution is Google Optimizer, but I'm always wary about giving Google too much of my info

[A.N.Onym]

I wonder if serving different site themes (HTML code) will give the bots shivers. I guess it is best to exclude the bots from the test, but it'd be some sort of cloaking (which is the use of Javascript, too).

Btw, Javascript takes some time to load, right?

[AbleReach]

PageLab has a free trial for the first month or 1,000 page views.

I haven't used them, but I have followed Dana Todd somewhat.

[phaithful]

I wonder if serving different site themes (HTML code) will give the bots shivers. I guess it is best to exclude the bots from the test, but it'd be some sort of cloaking (which is the use of Javascript, too).

Btw, Javascript takes some time to load, right?

You're correct that Javascript does take time to load, so you'll want to be selective to how much you change.*

With regards to the bots, since Google has their own multivariate testing platform which uses Javascript like Offermatica and Optimost. I believe the engines see this as an acceptable form of testing.

Even if you were going to do a complete theme change, which would essentially be calling 2 webpages in 1 request.... you're still only doing it to a fraction of your users, which hopefully won't be a bot (but then again they don't run javascript or cookies ).

*caveat: load time will significantly decrease if you package your javascript and css, and have the ability / resources to cache the content across multiple servers (e.g. akamai)

This post has been edited by phaithful: Apr 13 2007, 11:11 AM