Google have finally released a new web indexing system, Google Caffeine. It may seem a bit odd for the world’s number one search engine to change something that wasn’t broken before but with the continuous evolution of the web, Caffeine promises to return searchers with more up to date, relevant content then before from not only text based content but from video, images, news and real-time updates.
The old index method had several “layers” which all refreshed at different rates to others which would mean Google had to analyze the entire web causing a delay between finding a page and making it available to searchers.
With the new system, Caffeine analyzes in small portions and updates the search index on a continuous basis so new pages or information are added straight to the index giving searchers fresher information faster.
Every second Google Caffeine processes hundreds of thousands of webpages in parallel. To give an impression of how fast that is, Google have compared it to some real life situations.
If this were a pile of paper it would grow three miles taller every second. Caffeine takes up nearly 100 million gigabytes of storage in one database and adds new information at a rate of hundreds of thousands of gigabytes per day. You would need 625,000 of the largest iPods to store that much information; if these were stacked end-to-end they would go for more than 40 miles.”
