5 replies to this topic

[bobbb]

Just noticed in my logs that Googlebot is crawling out of Godaddy IP addresses [208.109.8.2xx]. All of December 2006 and now since February 2007. My site just happens to be on Godaddy servers.

Anyone else noticed this? Or is it fake?

[JohnMu]

Did you verify it? There are a lot of people surfing as a Googlebot, you'd be surprised

John

[eKstreme]

<shamelss link drop>
SE Bot Authentication in PHP
</shameless link drop>

Basically it is a code implementation of the page John linked to above.

Pierre

[bobbb]

Yes I did. ARIN tells me the block belongs to GoDaddy and the IP acts like a robot by just getting the pages (php/html) and nothing else. It does ID itself as "Mozilla/5.0 (compatible; Googlebot/2.1;"

I know the agent can be spoofed and if it came from China or Korea then I would understand (and not be surprised).

[JohnMu]

Hi Bob
It's likely some script hosted on a GoDaddy server by someone who wants to crawl the web scrape content from other people's sites. Google certainly does not host their crawlers within GoDaddy's IP space .

If you are worried about scrapers (or if the load is too high), I would block the IP or even the IP range.

Also, you might want to consider contacting GoDaddy's abuse department to report this issue - though you probably shouldn't expect too much - it might be a local proxy that is used within their hosting network and might have been used by any of the sites they host... hard to track (hard=probably not interesting for them unless you are a "big name").

John

[bobbb]

Just analyzed all my data. Only 164 pages read.

Guess it's no big deal but I will try the trick of blocking just for the practice.

consider contacting GoDaddy's abuse department to report this

Yea right. I agree, I could probably get more response from the wall in front of me.