Google’s Bigdaddy Update

Google’s Bigdaddy Update is Live

By Doug Williams - February 2006

Google is rolling out their next major update which has been called Bigdaddy. For months now Google has been testing some different crawling and indexing characteristics in an effort to take web searching to a new level. Bigdaddy is much more than another update. This is being termed a structural overhaul. So this contains new code for sorting and examining web pages. Seven year old Google now accounts for over 46% of all searches.

According to Search Engineer Matt Cutts of Google, “Bigdaddy involves new infrastructure and is not just a data push or algorithm update.” So far three data centers have been converted over and new ones are being converted at the rate of one every 10 days. This conversion is expected to accelerate here soon.

On January 4, we started getting our first glimpses of the new Google search. By early February Matt Cutts announced: “Bigdaddy continues to roll out and is now available at three data centers. In addition to 66.249.93.104 and 64.233.179.104, Bigdaddy is now up at 216.239.51.104.” These data centers can be accessed by entering these IP addresses directly into the address bar of your search browser.

Google reportedly has 85+ data centers. The data center conversion to Bigdaddy is supposed to be complete at the end of February or in early March. Google uses a network of data centers with different IP addresses to answer search queries. These decentralized servers share the workload of indexing web sites. The more data centers there are using Bigdaddy, the odds of you hitting a Bigdaddy data center in the normal rotation go up

How did this Bigdaddy” get its name? At a meeting last year, one of the webmasters (JeffM) suggested “Bigdaddy.” Bigdaddy is his nickname because that’s what Jeff’s kids call him. The name has stuck and has become the name for this update.

Bigdaddy fixes issues

The biggest problems Google plans to fix with Bigdaddy are hijacking redirects of URLs and what Google calls "canonicalization."

Hijacking refers to someone redirecting to his Web site, a request for another site. Webmasters will set up a server-side redirect so the old URL points to the new one. Hijackers are sometimes able to intervene, and steer the traffic to their own sites.

According to search engine specialists, Bigdaddy is intended to provide a solution to the problem of illicit redirects, known as " 302 redirects".

Canonicalization, that’s a fancy Google word for instructing a search engine how to decide which of a series of related URLs is the proper one to insert into the Google index. For example, most people would consider these the same URLs:

• www.example.com
• example.com/
• www.example.com/index.html
• example.com/home.asp

But technically all of these URLs are different. A web server could return completely different content and rankings for all the URLs above. When Google canonicalizes a URL, they try to pick the URL that seems like the best representative from that set.

What will Bigdaddy do?

Bigdaddy has different crawling and indexing characteristics. This will lay groundwork for future improvements that will allow Google to develop more advanced algorithms, larger databases and being able to index content faster and more effectively.

Another reason for the new data center infrastructure is that Google wants to be able to index different content types. Google is now testing a new search engine spider that is based on the Mozilla browser. That should mean that in time, it will be able to read links within images and even within Flash video.

Are you Up or Down with Bigdaddy?

What impact will this update have on search results today? We have spent this past week looking at the primary search terms for our SEO clients and comparing the results between “Big Daddy” and “non-Big Daddy data centers.” We found that 67% of the keyword phrases checked saw an improvement, 22% stayed the same and only 11% saw a decline in rankings.

We have also noticed that the Big Daddy data centers are reporting a much higher number of search results. As an example, searching for the term “landscape software” there are approximately 30 million results on the non-Bigdaddy data centers and over 60 million results from the Bigdaddy data centers. We found similar results on other search terms.

We interpret this to mean that sites with good on-page SEO methods and targeted reciprocal linking methods will do well with this update. It's hard to tell how the Bigdaddy update will affect your web page rankings. If you have a spam free web site with good content and many incoming links, the update should have a positive effect on your Google rankings. There are also higher numbers of websites being included in the data center’s search results.