“Can I Download The Entire Internet?”
And now for some useless stats.
I know, I know – it’s a stupid question. You can’t really download the Internet. It’s so ridiculously huge and messy that even Google hasn’t indexed all of it. But what if a clueless fool eccentric multibillionaire came by and asked you to do it, stating that “money is no object”? Could we do it, and how long would it take?
How big is the Internet?
The first thing we need to determine is how much data we’d need to download. We can calculate this by examining some historical data points and estimating liberally :
- In 2005 Google estimated that the Internet contains about 5 million TB of data.
- Also in 2005, VeriSign released a news report saying that there are about 77 million unique domains registered.
- Currently there are about 110 million active domains.
Assuming the amount of data on the Web grows in linear proportion to the number of domains, the Internet now holds about 7 million terabytes of data. Divide the difference between now and then by the number of years that have passed and we get the rate of increase – 500 000 TB per year.
How fast can we download it?
Depending on which source you trust, the fastest Internet connection that is available commercially is either 160 Mbps or 1 Gbps. Lets be generous and use the latter number. Given that and the above assumptions, it would take approximately 1817 years to download the entire Internet (as it is now) using currently available technology. Of course, your personal copy of “The Complete Internet Of 2009″ will probably feel slightly dated in the year 3826.
Addendum
Connection speeds of 1 Tbps and more will probably be available in the next decade. Throw in femtosecond lasers for incredibly fast data storage, and downloading a significant portion of the Internet (in a reasonable timeframe) may actually be a plausible idea one day.
If you had all the money in the world, why would you even settle for a 1Gbps line? I would just buy a major backbone ISP company since they have fiber laid out to all the other major carriers. Combine that with having a humungous number of multithreading jobs doing GET requests would decrease your proposed number of 1817. Still, I don’t think you can achieve downloading the entire internet in one’s lifetime.
I know you have chosen simple numbers so that readers can understand your point. I’m just adding my .02.
Yeah, I considered that possibility, but it’s harder to find reliable numbers for the backbone connections.
I think the real problem with “downloading the Internet” would be that the amount of data would always grow faster than your available bandwidth. As more people get high-speed internet we also get more bandwidth-intensive, user-created content like YouTube videos and so on.
I like the style of your writing. Nice article…
I knew a German GSM focused guy called “w-shadow” years ago, this isn’t you?
Nah, that’s probably some other shadow.