“Can I Download The Entire Internet?”

Internet on a stick!And now for some useless stats.

I know, I know – it’s a stupid question. You can’t really download the Internet. It’s so ridiculously huge and messy that even Google hasn’t indexed all of it. But what if a clueless fool eccentric multibillionaire came by and asked you to do it, stating that “money is no object”? Could we do it, and how long would it take?

How big is the Internet?

The first thing we need to determine is how much data we’d need to download. We can calculate this by examining some historical data points and estimating liberally :

Assuming the amount of data on the Web grows in linear proportion to the number of domains, the Internet now holds about 7 million terabytes of data. Divide the difference between now and then by the number of years that have passed and we get the rate of increase – 500 000 TB per year.

How fast can we download it?

Depending on which source you trust, the fastest Internet connection that is available commercially is either 160 Mbps or 1 Gbps. Lets be generous and use the latter number. Given that and the above assumptions, it would take approximately 1817 years to download the entire Internet (as it is now) using currently available technology. Of course, your personal copy of “The Complete Internet Of 2009″ will probably feel slightly dated in the year 3826.

Addendum

Connection speeds of 1 Tbps and more will probably be available in the next decade. Throw in femtosecond lasers for incredibly fast data storage, and downloading a significant portion of the Internet (in a reasonable timeframe) may actually be a plausible idea one day.

Related posts :

22 Responses to ““Can I Download The Entire Internet?””

  1. twan says:

    If you had all the money in the world, why would you even settle for a 1Gbps line? I would just buy a major backbone ISP company since they have fiber laid out to all the other major carriers. Combine that with having a humungous number of multithreading jobs doing GET requests would decrease your proposed number of 1817. Still, I don’t think you can achieve downloading the entire internet in one’s lifetime.

    I know you have chosen simple numbers so that readers can understand your point. I’m just adding my .02. :)

  2. White Shadow says:

    Yeah, I considered that possibility, but it’s harder to find reliable numbers for the backbone connections.

    I think the real problem with “downloading the Internet” would be that the amount of data would always grow faster than your available bandwidth. As more people get high-speed internet we also get more bandwidth-intensive, user-created content like YouTube videos and so on.

  3. nope says:

    I like the style of your writing. Nice article…

    I knew a German GSM focused guy called “w-shadow” years ago, this isn’t you?

  4. White Shadow says:

    Nah, that’s probably some other shadow.

  5. Himanshu Gajwani says:

    Well what if we have multiply lines?? 1 GBPS = 1817, 1 GBPS x 2 = 908.5 , 1 GBPS X 3 = 605.66 and so on…… Assuming that money is not a problem ………..he he he

    I think we can do it in less than a year if we have Multiply lines

  6. felix says:

    i dont think the problem will be the bandwith, the disk speed will slow it down ALOT. no harddrive can make 1gbps lol

  7. Anything Is A Possibility says:

    Another 1,000 years for the Porn!

  8. CLEGG says:

    IT CAN BE DONE.

  9. Nike says:

    Nothing is Impossible.

  10. If money was no object, I’d use a CDN-like setup with EVERY ISP (big AND small) in the whole world and mirror all the data requested by the users of the ISP. Between them they would probably account for 50-70% of the internet.

    The remaining, I could run a script that parses the ICANN registry & compares for sites that haven’t been pinged by the ISPs and well…

    I am talking absolute rot, aren’t I? Sigh.

  11. This really is what I have been finding all day. I should have discovered your blog post faster.

  12. @Shrikant if yo uput it on a CDN, surely that means you just copied the internet to the internet?

  13. qazibasit says:

    it is done, there are multiple techniques by which it can be done, most ISPs download the frequently views sites on their network and it is done autonomously, and that there are many online data storage centers who are currently downloading the internet and making a backup so in the near future they can renovate the internet. in the next decade after the era of web 3.0 internet will be different and that we will be getting the data from authorized nodes only rather than wandering from one server to another. The investers are now planning to invest in those mainframes. The domains will be allocated from those nodes and the data will be on those interconnected nodes, unlike servers that we have nowdays.

  14. nope says:

    The situation you described (censored/100% controlled internet?) seems to me completely different than where the internet technology, specially all the protocols behind it are pointing to since the beginning 20 years ago. what kind of protocols will the web 3.0 be based on at your opinion?

  15. Chaky says:

    Updating this info: http://www.usernetsite.com/webmaster/the-new-size-of-the-internet.php

    And internet is growing and growing

  16. One issue says:

    Too many viruses everywhere. Downloading the internet is like your computer sleeping with the whole planet, you WILL get every disease that ruins computers.

  17. Timelock says:

    It is possible if you can stop time.

    Which I will be able to do.

  18. Robert Nard says:

    If money is not a problem, divide and concur is the solution. Break the workload down to multiple machines with maximum bandwidth and it could be done in a reasonable amount of time.

  19. Fireprufe15 says:

    Assume you have 100 PC’s downloading at 160 mbps each, it would take a 145 years..

  20. Sniffer says:

    @One issue
    Already very outdated, as “Justin Bieber ft. Ludacris – Baby” (the worst song in the history of humanity imo) has 698,643,317 at the time of writing vs the 344,194,152 at the time you wrote your coment. That’s an increase of 345,449,165 or 100% in just 10 month! And that’s on the most viewed video on YouTube. If this goes on in the same speed, it will have about 1,4 BILLION views at November this year. Unbelieveable… I would like to see an updated size of the internet :)

Leave a Reply