Announcement
Collapse
No announcement yet.
The OFFICIAL Chatting Thread
Collapse
This is a sticky topic.
X
X
-
^^^^
I was Right!!
Search engines use automated software programs that crawl the web. These programs called "crawlers" or "spiders" go from link to link and store the text and the keywords from the pages in a database. "Googlebot" is the name of Google's spider software.
Many webmasters have noticed that there are now two different Google spiders that index their web pages. At least one of them is performing a complete site scan:
The normal Google spider: 66.249.64.47 - "GET /robots.txt HTTP/1.0" 404 1227 "-" "Googlebot/2.1 (+http://www.google.com/bot.html)"
The additional Google spider: 66.249.66.129 - "GET / HTTP/1.1" 200 38358 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
Why does Google do this?
Google hasn't revealed the reason for it yet. There are two main theories:
The first theory is that Google uses the new spider to spot web sites that use cloaking, JavaScript redirects and other dubious web site optimization techniques. As the new spider seems to be more powerful than the old spider, this sounds plausible.
The second theory is that Google's extensive crawling might be a panic reaction because the index needs to be rebuilt from the ground up in a short time period. The reason for this might be that the old index contains too many spam pages.
2008 325i coupe
Stock
Comment
-
As i should be! It's strange, because i've just noticed that my office car park is really empty...like there's about 25% of the cars that are normally here. Is there something happening today?theeldestboy
Silver 2005 E46 M3 Convertible
Alpine White 2004 E46 M3 Coupe
Topaz Blue 2002 M-Spec 330ci Sport Convertible
Blue 1998 318ti M-Spec Compact Sport
Comment
Comment