Rate To Add New Pages Options

[bloard]

I have a site with around 1k pages that does very well in google on some excellent keywords. I have just finished a new section that will contain about 30k new pages. At what rate do you think I can publish these new pages so as to not set off alarm bells at googleplex? 1k per day? 500 per day? 5k per day etc.?

The last thing in the world I want to do is get the pages that currently exist and rank well penalized by google suddenly finding that the site is 30x's as big as it was before, yet at the same time I want my new pages to be found quickly and start generating revenue. What is the balance?

BTW - long time lurker, first time poster. Forum is excellent. Thanks in advance.

[EGOL]

I think that the relevant question is not the rate at which you can add pages but instead the quality of the pages you are adding. If these pages look like cookie cutter content - such as about plastic surgeons in 30,000 different cities with the city name swapped in and out from page to page then adding 100 of them total on your site would not be a good idea.... in that case google will probably filter most of them from the search results - sooner or later.

Another problem would be posting 30,000 totally different articles that each appears on dozens of different sites. In that situation why should google index and serve dozens of different versions of the same content - or why should you do that knowing that you will be in competition with dozens of other sites with exactly the same keywords to attract traffic.

In either of the cases above you can probably find a more effective way to build a quality site that will generate long term income.

[bloard]

I agree that neither of the above scenarios would be a good way to build a long term business strategy. Therefore, the content does not fit into either category. Each page is unique content and extremely pertinent to the site topic. Thus, back to the original question... is google going to see a problem with increasing the size of my site thirty fold overnight?

[A.N.Onym]

I'd just add it in a way that is beneficial to your visitors. In logical chunks, in parts they'd have time to notice and study and get ready for more, etc.

I have to say, I am slightly suspicious on the quality of those 30k pages, too.

To create real quality articles of that amount it'd cost several millions of the US currency, I'd suspect.

This post has been edited by A.N.Onym: Dec 19 2006, 10:13 PM

[bragadocchio]

Hi bloard,

Welcome to the forums. Glad that you've decided to start taking part in some of the conversations here.

I think that I heard someone ask one of the search engineers at either Pubcon or SES answer the question you ask with this answer (which I've also seen Yuri use on his blog):

It depends.

It depends on what the pages are about. It depends upon whether or not they are pages that people will visit, and link to, and write about, and bookmark, and so on. It depends on what the 1,000 existing pages are like, and how popular they are. It depends on how well built those new pages are, and how search engine friendly they are. It depends on what kinds of topics the pages cover, and whether the topics are popular.

How unique are the pages, and the content they contain? Are they filled with content that can be found on other pages on the web, such as manufacturer's product reviews or distributor supplied copy?

How easy are the pages to index by the search engines? Do they fit well within the existing hierarchy of the site? Are there site map or site index pages that make it easier for spiders to follow them? Will there be links from existing pages of the site pointing to the new pages?

In addition to unique content, will each page have a unique page title, and unique meta description tag? Will it be easy for people to find stuff on your pages?

You're concerned with not "setting off alarms at Google" which is something that you probably don't need to be worried about if everything is above board. What you should be concerned about is whether or not adding so many pages will diffuse the link popularity value of your pages, or overwhelm visitors or change the things that people may like about your site.

[NameCritic]

Very nice forum. found it in Search Day. Good questions and answers about this topic. My 2 cents worth.

Had my company provided that many articles to you, it would cost around $450,000 so I understand the post from the person that says it would cost a fortune to do what you suggest.

I provide about 1000 articles per week for one client and they have not experienced any difficulty with google as a result of that, but that is with real original and informative content with seo in mind but not spammy text.

I would be very careful to make sure the content is high quaility if you plan to add even 1000 articles per week.

Most of my smaller clients add 1 article per day to their website and even then they make sure it is quality. Thats the same thing a blog does basically if you write to it every day so we know that rate works easily.

That is why you see so many people here skeptical about the number of pages you are asking about adding.

[JohnMu]

Google doesn't care if your content is high quality or not . It can't read, it can at most extract keywords and try to determine context, but for the it needs a push ...

The push Google needs to index new pages is mainly "value" from inbound links. If you have the link-value to support 30k new pages, then by all means go for it. I've put test sites online with 200k pages and gotten 90% indexed within a month (and no, the content was definitely not quality - but it was unique ).

If you don't have the link-value to support indexing of 30k new pages, then Google will just crawl / index as much as it can and ignore the rest. It will pull the average pagerank of the site down a bit, but it won't give you a penalty.

Do you have the value to support 30k more pages? If you don't, then I would go slowly and add pages in about the same rate as you build link-value (say from the new content).

John

[send2paul]

NameCritic - hello and welcome to the Cre8asite forums and community

Thank you for that insight into the article content world. I think, as far as this discussion goes, (so far), that was a vital contribution from the article provider side.

Most of my smaller clients add 1 article per day to their website and even then they make sure it is quality

I think, for me anyway, that this piece of info is an excellent guide for those with smaller websites/businesses who are considering using articles to add content to thier websites.

Thanks again NameCritic for dropping into this discussion.

Paul

p.s. if you want to, come and say a few words about yourself in the Introduce Yourself forum, and maybe have chat in After Hours ?

[A.N.Onym]

John, Google can detect duplicate, nearly duplicate and other kinds of duplicate content. If you scrape from other sites, that's not the best content. They'd rather show unique content to their users. That's what I meant by quality content.

And you are right that it takes links to help the content go up. It may be a good idea to show content to your visitors in such a way that they'd have time to read it and link to it, instead of dumping all the thousands of pages you have, so they'd walk in, see the huge pile of pages waiting for them, and walk out.

[JohnMu]

Yuri, I 100% agree with you on that . I was just assuming that those pages have legitimate, unique content; I think - at least in our forum here - we should expect that, right? I think there are certainly situations where you could put 30k unique and valuable content online at a time (moving an offline collection online, etc), but there are also a lot of situations where people just put 30k of "junk" online. I hope / assume that bloard's pages fall into the "unique + valuable" category. He'll know .

John

[A.N.Onym]

Well, what I meant is that our member may not necessarily scrape content, but still have duplicate content.

Just having a paragraph of text on a page makes it unique for the visitor. But not necessarily for the search engine (with all that top nav, secondary nav, footer, etc). That's why having unique titles, meta tags and more unique main content body should be helpful.

Becides, I believe it was mentioned that the 30k of pages are listings of some sort, which is perfectly acceptible within our forums.

P.S. So many kinds of duplicate or low quality content, eh?

This post has been edited by A.N.Onym: Dec 23 2006, 07:45 AM

[NameCritic]

hi, finally came back to introduce myself. lol

[Halfdeck]

I'd suggest that first you add a tracker to all your pages (if you haven't already) to monitor SE traffic to those pages which can give you a clue as to whether or not they're in the main index.

Second, publish all your pages but use robots.txt to only let Google crawl 1K pages. Wait for them to get indexed. Look at your site's supplemental percentage using site:domain.com/* query (which isn't too reliable) or other means.

That way, your users have access to the new content which is more important than anything else.

If 1k pages stick in the main index, unblock 1k pages more, and so on, till you start to see more supplemental results pop up. At that point, keep the rest of your pages disallowed for the time being while you gain more backlinks. Remove disallows as you gain more link equity.

You can also just let your pages turn supplemental and see how much traffic they pull, since Google is promising to drive more traffic to pages in the supplemental index.

If your site is huge, one internal linking tactic you might try is nofollowing sitewides (including links to the home page) and using random internal links to preventavoid a top-heavy PageRank distribution setup.

This post has been edited by Halfdeck: Aug 23 2007, 11:11 PM

[NameCritic]

Wow. Great answer. Uhh . . . I take it you have done a little seo before?

[bwelford]

Very informative and useful answer, Halfdeck.

Re one small point:

Look at your site's supplemental percentage using site:domain.com/* query (which isn't too reliable) or other means.

AFAIK Google no longer identifies which web pages are in the supplemental index. I don't believe there is a way of getting a count at this time.

[Halfdeck]

"AFAIK Google no longer identifies which web pages are in the supplemental index. I don't believe there is a way of getting a count at this time."

Yes, Google doesn't label supplemental results anymore. Still, the site:domain.com/* has a good chance of returning URLs that are in the main index. site:domain.com/& has a good chance of returning URLs that are in the supplemental index, though they're label-less. Keep in mind, those queries are no more reliable than site:domain.com *** -sdfsd used to be.

[bwelford]

the site:domain.com/* has a good chance of returning URLs that are in the main index. site:domain.com/& has a good chance of returning URLs that are in the supplemental index

Good one, Halfdeck. That seems to work reasonably well in the test I've just done. I hadn't seen the site:domain.com/& trick. Who spotted that?

[Halfdeck]

I hadn't seen the site:domain.com/& trick. Who spotted that?

I first read Danny Sullivan mention it though he told me WMW announced the hack first.

This post has been edited by Halfdeck: Aug 25 2007, 02:47 PM

[glyn]

It depends on what the pages are about. It depends upon whether or not they are pages that people will visit, and link to, and write about, and bookmark, and so on. It depends on what the 1,000 existing pages are like, and how popular they are. It depends on how well built those new pages are, and how search engine friendly they are. It depends on what kinds of topics the pages cover, and whether the topics are popular.

How unique are the pages, and the content they contain? Are they filled with content that can be found on other pages on the web, such as manufacturer's product reviews or distributor supplied copy?

How easy are the pages to index by the search engines? Do they fit well within the existing hierarchy of the site? Are there site map or site index pages that make it easier for spiders to follow them? Will there be links from existing pages of the site pointing to the new pages?

In addition to unique content, will each page have a unique page title, and unique meta description tag? Will it be easy for people to find stuff on your pages?

This is all what smoke and mirrors of what the SE wants you to believe the index to be. Although these guidelines do help defect spammers In answer to your question, and assuming your pages are good quality, I've never had any problems with 250 links per page, linked from an entrance page. That's one single html page with links off to 30 or 40 other

Glyn

[bragadocchio]

Glad that you dredged this thread back from the past, Glyn.

Bloard didn't really ask about the number of links from any one page. The question instead was, might there be an issue with too many new pages being added to a site at any one time.

I was really pleased to see Yahoo come out with a patent application which provides us with a little insight into how at least four of Yahoo's search engineers might feel about the rate of growth of a site, including links:

Using exceptional changes in webgraph snapshots over time for internet entity marking

Here's the abstract:

Techniques are provided through which “suspicious” web pages may be identified automatically. A “suspicious” web page possesses characteristics that indicate some manipulation to artificially inflate the position of the web page within ranked search results.

Web pages may be represented as nodes within a graph. Links between web pages may be represented as directed edges between the nodes. “Snapshots” of the current state of a network of interlinked web pages may be automatically generated at different times. In the time interval between snapshots, the state of the network may change.

By comparing an earlier snapshot to a later snapshot, such changes can be identified. Extreme changes, which are deemed to vary significantly from the normal range of expected changes, can be detected automatically. Web pages relative to which these extreme changes have occurred may be marked as suspicious web pages which may merit further investigation or action.

The method in this patent application focuses upon links, but it could just as easily look at the number of pages of a site:

Other attributes, other than the number of links associated with a node, also may be used. For example, in one embodiment of the invention, each snapshot captures the number of distinct web pages that are hosted by each host. Thus, the increase in the number of web pages hosted by a particular host may be used as the basis of determining whether that host is a suspicious host.

Suspicious changes from one point in time to another may result in automatic banning or penalization or flagging of pages or a site for human review.

Is Yahoo doing this? I would guess that they might be. I would also guess that Google would be, too.