Google Spider - Googlebots

When you have you pages ready and after you have submitted your site to Google, how Google is going to collect data from your sites? Yup! Let me introduce to you Google spider - Googlebots A Googlebot is a search bot, used by Google. It fetches all the web pages from your sites and build the search index for the Google search engine. If you don't want Googlebot to crawl some of your page, you can set up a robots.txt under your root domain of the site. There are two versions of Googlebot - deepbot and freshbot. Deepbot, same as the name, it is a deep crawler, it tries to download all the pages from you by following all the links in your pages, it completes this process monthly. Another bot, freshbot, it crawls the updated web pages, it only visits the page that changes frequently. Some rumors suspect that freshbot is the one in which collect the Freshness of the page. Typically, Freshbot would visit a daily newspaper's website every day and a weekly ezine would get crawled once every 7 days. But Google does not announce the specification of its crawlers so far.