8 replies to this topic

[Aaron Pratt]

New pages usually get indexed in 1-3 days if you have enough PageRank but it has been 5 or more days for a few of my sites. Can Google Multitask or is it in need of some serious upgrades? Even Matt Cutts admits that many of the datacenters are old and this is why the visible PR update takes so long to show. Does it really lag the entire network?

I like to think of Google as the massive complicated matrix but really it is only a few years old. It's like growing up and realizing you might be smarter than your dad, oh well.

Notice this bottleneck?

[JohnMu]

Hi Aaron
Is it a bottleneck or is it on purpose? If you ask me, PR exports have a priority just slightly above making sure that the comments in the javascript are all politically correct. Why waste resources on something that is a "toy"?

If you look at all the high-resource services they've started in the past year or so I really doubt that they're running low on storage or horsepower (otherwise they'd slow that pace, but then again, perhaps they are already running slower than they could?).

Have you noticed how fast search results are? If you know what it usually takes to query that much data then you wouldn't be saying that Google is getting slower . It's friggin' amazingly fast.

John

[bwelford]

Google only puts enough effort behind the publiched PageRank to keep it alive as a marketing gimmick. Real search algorithms no longer use a simple PageRank number but have a cloud of factors that contain the essence of PageRank.

Your question got me thinking of another side to all this, Aaron. That's the paradoxical timing effects of regular web pages versus blog entries. With regular web pages, they presumably are largely time independent. So you can consider mechanisms like sand-boxes to check out the authority of web pages as others link to them. Age of a regular web page may have some bearing on its relevance.

Blogs on the other hand are designed to be very time dependent. It's really all a form of news. So old news is no news. In consequence we have pinging mechanisms and news feeds to give instant visibility to blog entries. Google feeds this process with its Blogsearch process.

So how does/should Google multi-task on this one? After a period of time, clearly blog entries are treated as regular web pages if they have sufficient authority. How does Google treat them in the first few days with respect to regular search? How do others see blog entries stacking up against regular web pages in Google's eyes?

[Aaron Pratt]

Those are very good questions and I also do not have the answers. I do know that all my sites are now blogs, they just communicate better with engines and other humans out in the social matrix. Blogs establish relationships and for those of us who do not engage in link building or digg manipulation natural links are made easier via pingbacks.

[Adrian]

Just a note, the visible PR isn't old because of datacentres having older infrastructre, they just only update visible PR every few months anyway. Nothing to do with the state of the Datacentres.

New pages being indexed is going to depend on several things. How mnay pages you already have indexed, what your PR is, if the new content coincides with a regular deep crawl anyway.....

Besides which, I don't see it matters greatly if a page takes 3 days or 5 days to show up in Google.

If it's a blog post you've got things like technorati that you can notify of new content etc...

I wouldn't be looking for large amounts of google search traffic within a day or 2 a new blog post, unless it was getting listed in Google News.
Which is pretty rapid.

[rustybrick]

Aaron, did you see Matt post a few hours after you posted this?

http://www.mattcutts...fo-on-pagerank/

[bwelford]

Adrian, I agree with your comments but think it's worth adding a positive word for Google BlogSearch. This picks up new blog posts within an hour and I've seen reasonable volumes of traffic from this source. So there is a slice of the audience which is looking at the newsfeeds created by blogs via such search tools (including Technorati, Sphere, etc.).

An interesting question is whether the PageRank value that applies to the domain in regular keyword searches also comes into play in Google BlogSearch. Or would there be a separate PageRank system at work in the news feed space. Perhaps some of our members can comment. ;-) ;-)

[graywolf]

Whenever that big google patent came out a few years ago (ack I'm getting old) they mentioned comparing growth over a certain time periods, fancy maths people refer to this as delta or difference. The shorter the time period you measure delta over the less likely you are to spot "aberrations" or "anomalies", however if you make the time period too long you stunt growth.

Having a large data set for multiple year time periods google has a reasonable range for growth. For example they have a range for [delta pages] over time period x. Thanks to Matt we also know that adding lots of pages will trip up some sort of filter. So if a site had a large [delta pagerank] with out a corroborating amount of [delta links] and [delta traffic] (gathered from toolbar, adsense, analytics or other methods) it might look "manipulated".

Additionally by updating less frequently they make it much more difficult for people who are actively trying to manipulate it to establish a direct cause and effect relationship between events.

[Aaron Pratt]

rusty - yes, thanks I enjoy matt and keep an eye on all of his posts as you also do.

graywolf - yes indeed, I dread pagerank updates and one of my sites is going PR7 with little content, that can be worse I believe.

The reward of high PR is not always what you expect, if you are rewarded with high PR you better offer something useful, it's do or die time which is challenging and scary at the same time. Oh well, got to get back to writing stuff.