“Can I Download The Entire Internet?”
And now for some useless stats.
I know, I know – it’s a stupid question. You can’t really download the Internet. It’s so ridiculously huge and messy that even Google hasn’t indexed all of it. But what if a clueless fool eccentric multibillionaire came by and asked you to do it, stating that “money is no object”? Could we do it, and how long would it take?
How big is the Internet?
The first thing we need to determine is how much data we’d need to download. We can calculate this by examining some historical data points and estimating liberally :
- In 2005 Google estimated that the Internet contains about 5 million TB of data.
- Also in 2005, VeriSign released a news report saying that there are about 77 million unique domains registered.
- Currently there are about 110 million active domains.
Assuming the amount of data on the Web grows in linear proportion to the number of domains, the Internet now holds about 7 million terabytes of data. Divide the difference between now and then by the number of years that have passed and we get the rate of increase – 500 000 TB per year.
How fast can we download it?
Depending on which source you trust, the fastest Internet connection that is available commercially is either 160 Mbps or 1 Gbps. Lets be generous and use the latter number. Given that and the above assumptions, it would take approximately 1817 years to download the entire Internet (as it is now) using currently available technology. Of course, your personal copy of “The Complete Internet Of 2009” will probably feel slightly dated in the year 3826.
Addendum
Connection speeds of 1 Tbps and more will probably be available in the next decade. Throw in femtosecond lasers for incredibly fast data storage, and downloading a significant portion of the Internet (in a reasonable timeframe) may actually be a plausible idea one day.
Related posts :
Subscribe via RSS
Get Updates by Email
If you had all the money in the world, why would you even settle for a 1Gbps line? I would just buy a major backbone ISP company since they have fiber laid out to all the other major carriers. Combine that with having a humungous number of multithreading jobs doing GET requests would decrease your proposed number of 1817. Still, I don’t think you can achieve downloading the entire internet in one’s lifetime.
I know you have chosen simple numbers so that readers can understand your point. I’m just adding my .02. 🙂
Yeah, I considered that possibility, but it’s harder to find reliable numbers for the backbone connections.
I think the real problem with “downloading the Internet” would be that the amount of data would always grow faster than your available bandwidth. As more people get high-speed internet we also get more bandwidth-intensive, user-created content like YouTube videos and so on.
I like the style of your writing. Nice article…
I knew a German GSM focused guy called “w-shadow” years ago, this isn’t you?
Nah, that’s probably some other shadow.
Well what if we have multiply lines?? 1 GBPS = 1817, 1 GBPS x 2 = 908.5 , 1 GBPS X 3 = 605.66 and so on…… Assuming that money is not a problem ………..he he he
I think we can do it in less than a year if we have Multiply lines
i dont think the problem will be the bandwith, the disk speed will slow it down ALOT. no harddrive can make 1gbps lol
Another 1,000 years for the Porn!
IT CAN BE DONE.
Nothing is Impossible.
If money was no object, I’d use a CDN-like setup with EVERY ISP (big AND small) in the whole world and mirror all the data requested by the users of the ISP. Between them they would probably account for 50-70% of the internet.
The remaining, I could run a script that parses the ICANN registry & compares for sites that haven’t been pinged by the ISPs and well…
I am talking absolute rot, aren’t I? Sigh.
This really is what I have been finding all day. I should have discovered your blog post faster.
@Shrikant if yo uput it on a CDN, surely that means you just copied the internet to the internet?
it is done, there are multiple techniques by which it can be done, most ISPs download the frequently views sites on their network and it is done autonomously, and that there are many online data storage centers who are currently downloading the internet and making a backup so in the near future they can renovate the internet. in the next decade after the era of web 3.0 internet will be different and that we will be getting the data from authorized nodes only rather than wandering from one server to another. The investers are now planning to invest in those mainframes. The domains will be allocated from those nodes and the data will be on those interconnected nodes, unlike servers that we have nowdays.
The situation you described (censored/100% controlled internet?) seems to me completely different than where the internet technology, specially all the protocols behind it are pointing to since the beginning 20 years ago. what kind of protocols will the web 3.0 be based on at your opinion?
Updating this info: http://www.usernetsite.com/webmaster/the-new-size-of-the-internet.php
And internet is growing and growing
Too many viruses everywhere. Downloading the internet is like your computer sleeping with the whole planet, you WILL get every disease that ruins computers.
It is possible if you can stop time.
Which I will be able to do.
If money is not a problem, divide and concur is the solution. Break the workload down to multiple machines with maximum bandwidth and it could be done in a reasonable amount of time.
Assume you have 100 PC’s downloading at 160 mbps each, it would take a 145 years..
@One issue
Already very outdated, as “Justin Bieber ft. Ludacris – Baby” (the worst song in the history of humanity imo) has 698,643,317 at the time of writing vs the 344,194,152 at the time you wrote your coment. That’s an increase of 345,449,165 or 100% in just 10 month! And that’s on the most viewed video on YouTube. If this goes on in the same speed, it will have about 1,4 BILLION views at November this year. Unbelieveable… I would like to see an updated size of the internet 🙂