New Sites Put Into a "Sand Box" by Google Options

[sanity]

The only pattern I see from the threads is that these are new sites. I see a wide range of backlinks reported, a wide range of styles of on-page optimization. Only pattern is the site was launched after December.

Yup that's what I'm hearing and seeing too. There has been speculation that all new sites are on "hold" for a period of time before they rank - 3 months has been suggested. I too have been reading the WMW threads and a couple have people have recently commented that after 3 months their sites have started to rank.

[tosheroon]

I've had a similar problem with a site that went live at the end of january. For its main keyphrase both my web design site and the google directory category rank above it and on those pages the text only appears as a links to the original site. It has in the last two weeks started to improve so perhaps 3 months probation idea is right. Originally within the first week of launch it was number one for its keyphrase and several other related phrases. To make a cross for me to bear, my partner bragged about that to the client, who consequently developed a taste for checking his serps every day. :|

[projectphp]

Why would this \"sand box\" effect be happening to sites with very little backlinks, where there is no \"manipulation\" flag being raised?

Maybe it pays to think like Google for a bit. Not creditting links straight away could potentially:

- Hinder (not stop though) the a) old buy domains, build spam sites, c) get banned, d) repeat methodology, as it makes it far riskier (will the site last three months??), and far more expensive (need more sites in teh "pipeline").
- Hinder link buying. If you buy a PR 9 link and see no ranking improvement for three months, will you continue? Could become a very expensive investment very quickly to buy a lot of links for a PR boost, but no ranking benefit.
- Fly by night scam stores. If a site wont ranlk for three months, makes setting up a scam eCommerce store harder.
- Lessen PageRank manipulation. Long term links are harder to get than short term links, and links that come and go quickly wont effect rankings overly much.

By throwing all links, or all new sites, in together, teh assumption is taht if a site is good enough, it will last. If it isn't, good night sweatheart, good night.

So, my questions would be: does this phenomenon also apply to older sites (but no one has noticed)? Could it be all PageRank is not creditted for 2-3 months? I assumme that Google timestamps links (the expired domain issue, that zeroes all links on expired domains that change hands), so it wouldn't be a very hard SQL statement to write to seed any PageRank calculation with longer term links. Visa versa, it wouldn't be too hard to store old PageRank values for three months. Do PageRank now, display current value in toolbar, use old value in algorithm.

All pure speculation, but my $0.02 of places to look. I don't know of many new sites, so hard to comment much more from experience.

[bragadocchio]

There's likely not a connection, but there was a time when Google was showing categories for search results. When they went through a site redesign a few weeks back, they stopped doing that.

It's mentioned in this article:

Google revisits policy on hate sites

The policy would be part of a broader change at Google to bring back category labels to all search results, not just for Web sites that some people find offensive. Until a Web site redesign a few weeks ago, the company included such labels for everything from cars to cats to camping. It stopped because of quality issues, according to David Krane, a Google spokesman.

This paper (Next Generation Web Search: Setting Our Sites) shows that Google had been using category labels for quite a while. It's from September, 2000.

The fact that search engines show hits on category labels is significant, because web directory categories, and their associated links, are manually selected to be representative starting points for search.  

...

However, Google also now incorporates category information into its search results, listing category labels beneath a search hit if that category had been assigned to it. An informal test on short general queries brought up on average 2.7 categories alongside the top 10 hits.1

We've been seeing changes to Google over the past year or so that might indicate more of a reliance upon categories, whether through hilltop or local rank, or advanced semantics.

I hadn't seen an admission from Google before about a "problem" with the quality of their categories.

Is there a relationship between that problem, and this problem with newer sites failing to show up in categories that they might otherwise appear within (and be ranked within)? Don't know.

[Jean_Manco]

Bill - Google certainly was close-lipped last year about the problems with its directory. It never explained in public what was causing the unusual delays in updating from Dmoz. (Very diplomatic. The problems were almost certainly with the errors in the ODP's RDF.) The relationship between the 'labels' in the results and where a site actually was in the directory went haywire, because one was being updated and not the other.

I can't see any connection though between these issues and the one discussed on this thread.

[bwelford]

Since we're all confused, perhaps I should add to the confusion.

I have always had a problem with the notion of categories. This implies that we can write a tree structure of categories, subcategories, etc. where any given web page can be correctly classified into an appropriate sub-sub-sub-sub-category. Clearly it doesn't make sense. A given web page has associations, some weak, some strong with a huge group of somewhat related web pages. It's more like one of those complex mindmap or synaptic circuit diagrams. If you had a map, you would see some points (web pages) close together in clumps. Other points would be by themselves. If you like you can try to put a name on some of these clumps and call them a certain sub-sub-category. However it's highly arbitrary.

The one thing this approach has going for it is that it helps us humans. We can see a short list and try to understand the implications of what is on this short list. That's one of the advantages of the Teoma SE reports. However I'm not sure that the clusters help the search engines in any meaningful way.

I thought the semantic analysis approach tries to decide "how far apart" any two web pages are in semantic terms. For any keyword phrase search, it then tries to pull out a long list of those web pages that are "close" to the keyword phrase. The human mind would have a problem trying to work with such a huge list. The computer handles it all then delivers whatever summary we humans want: say the 10 that are "closest" to the keyword phrase.

I never saw the necessity for categories in the semantic analysis stuff. However to make the computations doable, it may have been advantageous to start off in a certain slice (category) of the database.

So with this view, I was not surprised to see Google downgrade the visibility of the Directory and its categories. I guess we'll see how the future unrolls. :roll:

PS BTW projectphp, I really related to your analysis. It makes a lot of sense.

[Jean_Manco]

I grant you there are problems with the scalability and even the concept of Web directories Barry. The fall in their popularity is the main reason that Google demoted its directory off the front page.

But I'm still not seeing a connection to whatever Google is doing with new sites. Maybe I'm missing something.

[rustybrick]

I tend to like projectphp's view but I don't think Google will do something that drastic. Its like what is going on at the airports in the US and other countries. Profiling all new sites in one bucket and making them stand on long lines.

The point of putting these sites on a hold is to discourage the spammers. But that is obviously not fair to those non-spammers. Would Google lump everyone together? It is possible. All new sites is the profile. Doesn't seem like a good profile to me.

And if this is a case to ward of spammers, don't you think they will pick up on this and then buy one or two links right away. 90 days later buy all the good links when their site gets out of the sand box. So SEO's will need to just think 90 days ahead with their campaigns.

[bwelford]

You're quite right, Jean. I was developing a thought that Bill had which he acknowledged was not necessarily connected. That's the way minds work, we don't always stay Cartesian in our thinking.

The point of contact for me on categories and the real theme of this thread was a general feeling that perhaps Google doesn't know where it's going. Today is, I believe, the day when Google must publish details of its financial arrangements given that it has more than 500 shareholders. Everyone seems to be saying that the brains behind the outfit, Brin and Page, are worrying about this IPO and questions of control, etc., etc. Perhaps those two captains up on the bridge of this Titanic have forgotten to look out of the window to see where the ship is heading. :roll:

[John AlphaOne]

Returning to the original question, is it not possible that these sites are just settling precisely where they belong in the current scheme of things once the fresh bonus has expired.

These are new sites. In general a new site won't have many backlinks, so if it performs poorly in the rankings then that's no surprise. Alternatively if it has lots of backlinks then how has a new site managed to acquire these in a short space of time? Chances are these are poor quality "manufactured" links, scoring badly with respect to relevant PR, LocalRank or whatever.

[rustybrick]

John, its possible with my example but in the larger sample it is not.

Some of these sites have really nice PR and tons of quality links. Thought is that how can a brand new site get these links so quickly? But then when you think of the Internet, there are tons of sites that explode virally and obtain thousands of links in a weeks period.

So again, I don't know.

[peter_d]

A wise observer noted that Google possibly decided that the policy of rewarding fresh sites was rather prone to abuse. Think about why that might be.

So, reverse the policy (except for perhaps news-y style sites).

Unfortunately, the new policy has a side effect of not ranking legitimate sites for a while. That's ok, reasons the 'plex - a legitimate site (i.e a site that isn't overly concerned with ranking manipulation) won't worry about short term non-ranking.

Only SEOs are overly concerned about that...

[HHI Golf Guy]

Even though the site we recently developed appears to still be in the "Sandbox", we are seeing 2nd page results on Froogle.

See my recent post on the Google vs Froogle thread if you're interested in the details.

[Conqueror]

I agree 100% with Rusty Brick. I have a site that I uploaded to the Internet on Dec.28th. 2003. So the site is now 4 months and 3 days old. It had lots of websites linking to it and Google "found" it in a couple of days. It originally placed high on some important keywords (so far this is TYPICAL) and then dropped out of sight (still TYPICAL) but instead of returning at aroun the 8 week mark for the really competitive keyword phrases it didn't.

In fact even though the site was a PR6 on the Google Toolbar for months and based on competitiors sites should have been on page 1 for major industry keywords it wasn't even in the top 1,000 results! No where. All the while being a PR6, listed on about 50 directories (one way links) including the Yahoo! Directory, LookSmart, Business.com, YellowPages.com, etc. Did paid inclusion also with all the major search enignes. The only search engine I did NOT submit to was Google.

Also, the #3 site on Google for this major industry keyword is only a 14 page website and has only 28 backlinks that show on the Google Toolbar!
My site on the other hand is a 156 page website and has 501 backlinks that show on the Google Toolbar. Go figure.....

My site just jumped from 66 Google backlinks to the 501 about a week ago when Google updated my backlinks. Curiously after getting credt for the 501 backlinks my site actually DROPPED on the Toolbar from a PR6 to a PR5; but I am confident that this is a temporary thing.

My site recently showed up in the SERPS for one major industry keyword and currently ranks #21. It should move as high as #3 based on my analysis of the 10 websites presently on page 1.

But this has all taken 4 months and I once had a site I got on page 1 of Google in 2 months. So definitely something has changed. Almost like a penalty for brand new sites. I know for me it has felt like a penalty. It hurts knowing that here's a site #3 on Google with only 14 pages to the site and only 28 backlinks being ranked where my site should have been a couple of months ago and should be now, but still isn't. Also, my 501 Google backlinks are of the HIGHEST QUALITY, being industry related and PR's of 4-7 and I even have one PR8 site linking back to my site.

My site should be a PR7 after Google updates the toolbar pagerank and I should FINALLY be on page 1 for the major keywords after the next Index update.

But like Rusty said it is taking twice as long as it used to. I think from here on out that SEO's better tell their clients that it could be 3-4 MONTHS before their site is FULLY indexed by Google. Instead of the 6-8 WEEKS it used to take. I think this could be directly related to all those new billions of websites that Google has added to their index. It's most likely that, rather than a new site penalty of some sort.

[AmanM]

Take a little bit of this:

1.

SEO's better tell their clients that it could be 3-4 MONTHS before their site is FULLY indexed by Google. Instead of the 6-8 WEEKS it used to take. I think this could be directly related to all those new billions of websites that Google has added to their index. It's most likely that, rather than a new site penalty of some sort.

And a little bit of this:

2.

Not creditting links straight away could potentially: 

- Hinder (not stop though) the a) old buy domains, build spam sites, c) get banned, d) repeat methodology, as it makes it far riskier (will the site last three months??), and far more expensive (need more sites in teh \"pipeline\"). 
- Hinder link buying. If you buy a PR 9 link and see no ranking improvement for three months, will you continue? Could become a very expensive investment very quickly to buy a lot of links for a PR boost, but no ranking benefit. 
- Fly by night scam stores. If a site wont ranlk for three months, makes setting up a scam eCommerce store harder. 
- Lessen PageRank manipulation. Long term links are harder to get than short term links, and links that come and go quickly wont effect rankings overly much.

3. Factor in that the senior staff at Google (most are probably engineers?) will have been distracted by an IPO for say, the last 3 months (I used to work in M&A for a bank, so have a pretty good grasp of timeframes and complexities of normal IPOs; Google's online share bidding process is anything but normal and knowing Google's love of technological innovation, probably involved a lot of work on the part of their own engineers in bulidng the aforementioned online bidding system they hope to implement i.e. resource allocation away from radical algo changes in the short term [let radical refer to algo changes that would greatly filter out spammy sites]).

In other words, more sites to deal with leading to technological constraints in terms of providing a timely service, combined with concern over spammy sites and the fact that Google has probably had a lot of corporate decisions to make which has influenced the allocation of resources in the short term.

This would lead to a plausible conclusion that Google doesn't like having to exclude new sites, but at the moment doesn't have time to deal with the problem fully. So they use this as an interim option while the company deals with it's IPO then BHAM! - they get back to working on search and suddenly have a few hundered million to pump into developing a new algo that would solve this problem (and possibly numerous other algo flaws that SEOs are unaware of).

A viable explanation? I am not an SEO, so thought I'd offer a business cost/benefit perspective on this (strictly speaking not my speciality either, as I'm an Economist by trade).

In case you're wondering why I'm writing this, I'm writing some personal finance content as a, well, 'hobby' i suppose, for a site that is being developed by some SEOs. Launched in late January, around 180 pages of content, plenty of quality backlinks, new content every couple of days, frequently spidered, PR6 (I've picked up quite a bit of search knowledge perusing these boards ). The results? Two weeks in the SERPs at launch on several keywords, now nowhere. The answer? Add good content and backlinks. Wait. Add more content and back links. Wait some more. Repeat this process indefinitely. Why? Because spammers won't.

Aman

[peter_d]

I pretty much agree, Aman. It is happening, and the "why" is probably due to a number of factors.

Wait. Add more content and back links. Wait some more. Repeat this process indefinitely. Why? Because spammers won't.

I wouldn't be so sure about that It will only affect the amateurs.

[Conqueror]

Aman:

That's what I have been doing: adding more content and links, waiting, adding more content and links, waiting some more, adding more content and links, waiting still more, adding more content and links and still WAITING, WAITING, WAITING.....

I've read so many theories as to what Google is up to that I have decided that Google is just plain BROKE and has been for quite some time. If Google truly has something like 4 BILLION websites to index then this certainly could happen. At the very least really slow things down.

[AmanM]

The idea of Google being 'broke' wasn't suggested by me but was mentioned earlier in the thread.

I just wanted to point out that:

a) There is not necessarily a single (SEO) cause for this 'sand box' effect.
Beyond this, there are numerous other factors to consider beyond SEO-only explanations, such as management issues within Google.
c) A tentaive conclusion that this 'sand box' effect is not necessarily a particularly long term phenomenon.

With regards to c), the IPO raises a couple of interesting questions. The primary reason for an IPO is usually to get some cash to invest in the business. Many would argue that Google now makes a ton of money (what $100m profit last year? - I'm guessing - I haven't really paid that much attention to Google's figures), suggesting that perhaps Google doesn't need the money for investment.

However Google seems to spend a huge chunk of any profits on R&D, specifically on related technologies rather than their main search.

Alongside this existing high spend on Google Labs, there is likely to be a pretty large cost involved in the rollout of Gmail, which holds strategic importance for Google when it comes to fighting the competition in search (see my post with an excerpt from the Economist for more info on this).

The other strategic imperative for Google is to remain a credible search engine in the longer term and this involves removing spam from the results. The combination of draconian measures in the short term ('sand boxing') and the decision to float to bring in more money suggests that Google are about to launch a huge R&D campaign on a drastically new algo (innovation leading to discontinous change in search). Again looking at this strategically, this would make sense as Google have an 18-24 month window of 'first mover advantage' before Microsoft launch Longhorn and go on an all-out assault on the search market.

What I'm suggesting as a possiblity here (incredibly tentatively), is that many of the techniques employed (deployed?) by SEOs could, in essence, be reduntant in the short- to medium-term rather than longer-term. Sandboxing may be a problem for SEOs with a new site at the moment, but a radical change with Google's technology (such as removing PageRank - finding a way that users cast votes rather than webmasters 'voting' with links - and I mean an effective and efficient user vote system rather than a smiley face in the Google Toolbar) could mean that a lot of existing techniques could be out the door. Some of the SEOs I'm working with attended the NYC SES conference and were saying that this user-orientated approach rather than webmaster-orientated approach was brought up as one of the key aims of Google (personal search working on the principle of websites visited by your peers).

I'm not saying SEO will disappear or anything like that, but it will change the need for PR building and hoarding, and focus more on the on-page techniques like ensuring you have well written SEO'd content and well built sites, rather than link strategies.

If future search technology is geared around users rather than webmasters and links, then site's return to the fundamentals. In other words, having a good site that users like and want to use. Worryingly (for lefties like me), this would play into the hands of big blue-chip players who could use 'old media' advertising to entice users to their sites, thus in part removing the egality of the internet where small players can compete with large. That would be the price you paid for effectively eliminating spam from the SERPs*.

Or you could say that I'm talking nonsense. Sorry, I've strayed somewhat.

Aman

*Edited: Meant to say until SEOs and spammers worked out the new algo. Then it starts all over again.

[John AlphaOne]

Here's an interesting tip picked up from another forum. If you have a site afflicted in this way try adding:
-dfsdgsdsd -sdfgsdgsdfg -sdfgsdgsdg -sdfgsdfgsdfg -dsfgsdgsdg -sdfgsdfgsdfg -sdgsdfgdsfg
to the search terms.

Voila - I'm back at #1 where I belong. Works also for the "Negative Pressure Isolation Rooms" case mentioned earlier in the thread. In the latter case the total pages goes down from 79,000 to 41,600.

I think this maybe gives a clue as to what is happening - adding that garbage on the end defeats the Semantics. Why are new sites not performing well - maybe because indexing the semantics takes a shed load of processing power and it's all way backlogged. And maybe every time you tweak your page to try and improve matters you go to the back of the queue.

Just speculation of course - keen to hear your views.

[rustybrick]

Nice find! Interesting stuff.