Feeds

Fedora servers are that big

Download Your Way

  • alert
  • submit to reddit

Boost IT visibility and business value

Letters Yesterday's article on Fedora's FC6 download travails prompted a torrent of emails, not all abusive.

Much to our surprise we stirred up something of an ontological debate, as readers attempted to answer that age-old question: how big is big? Choice cuts below.

The Fedora project has apparently been subject to a sustained DDoS attack. That someone decided to attack on the scheduled release date is unlikely to be a coincidence, and would therefore probably be in the hope of getting Fedora bad press. Well done for falling for it and supporting hackers! Jonathan Larmour

Umm...always happy to oblige, Jonathan.


Well they did say that BitTorrent was the preferred method of downloading FC6, and sure enough I was able to use this method so that the ISOs were sitting waiting for me this morning. FWIW the site breakage also stopped most of the update applications, like yum and Yumex etc, from being able to get any Fedora updates at all, and as ever, there are a whole collection of day 0 updates for FC6 due to the extended package freeze before the release date. Yours enthusiastically, Brian

The Redhat/Fedora download sites were having issues before FC6 was announced. I was helping a friend early last week with a dedicated server install, adding some software he needed, and the tool that talks to fedora servers to get lists of current software (yum) was timing out roughly 3/4 attempts in any given hour.

My conspiracy theory: a competitor bought some zombie boxes and is DDoSing fedora judiciously.

More realistic: I didn't check but I suspect that maybe the FC6 downloads were available (in some fashion) as early as last week ... and the enthusiasts were being so damned enthusiastic.

Jacob L E Blain Christen


To keep things even more interesting, existing users of FC5 have not been able to update their servers for 2 days now :)

Wayne


Red Hat's 'low availability OS' - hardly.

I hit the torrent yesterday, got it in a couple of seconds (it's less than 100 KB), and the ISO came roaring down the line at 600 KB/sec, about as fast as it ever is for me. There were over 5000 users in the swarm I was part of.

Anyone (especially geeks) downloading a 3.25 GB file using HTTP needs a bitchslap, not a faster RedHat pipe. Torrents are very much the right way to do it.

65 TB an hour by the way is a *very* big chunk of bandwidth in anyone's book (2000*3.25GB) - it's 1.8 GB a second (14.4 Gbit/sec) before you consider any overheads (which using HTTP is significant), other page loads, resends for people with dodgier links, etc.

Cut them a little slack please, your headlines are sometimes in danger of heading from friendly irreverence over the line into Daily Mail (spit) "how could this happen?"-style headlines, and that would be a shame, as I usually like your style and the info I get from you.

Dan

Consider us bitchslapped. But the Daily Mail? We didn't mention immigrants once. I've doublechecked.


Don't be misinformed. Linux can handle unreal web loads on one machine. Redhat Just was not setup to scale downloads to that level. The Worlds busiest single machine/server is one (1) linux box. :)

"1.4 The Numbers At the time of writing (May 2005), ftp.heanet.ie; can sustain at least

  • 27,000 simultaneous HTTP downloads
  • can saturate a gigabit interface at the same time
  • has roughly 3.2 million downloads per day
  • ships about 3.5 Terabytes of content per day
  • mirrors over 50,000 projects
  • as 3.7 Terabytes of unique content
  • has circa 6 million unique files ... (on IDE hard drives!)

    http://www.stdlib.net/~colmmacc/Apachecon-EU2005/scaling-apache-handout.pdf

    anon-o-mouse


Well big is always just a matter of persepctive. The thing to bear in mind is that it's 10,000 copies of probably a dvd iso, 3.5 or 4 Gb depending on what your favorite flavor is. So that's about 37Tb of data downloaded over the 5 hours. To manage such bandwidth the other open source tool out this week Firefox would need to have 7.5 million downloads over the same time period, on the page your news story was served on would need to serve up 1.2 billion page views (at 30k a page I realise that there's css and images, but any devoted reg reader probably has them cached ;-) )

To keep up with that kind of data rate they'd need what 1.7 Gbit of outgoing bandwidth, so even to have managed what they did they need more than a unix box or two I'd have thought, or at least more than two network adaptors. And all of this ignores thier normal traffic, and the related traffic (people browsing the fedora pages about the download, and then browsing other stuff on redhat's network while they wait for the download) Not that I disagree with your general feel that they could have done better, but it's not exactly a small challenge.

Rob

NB these numbers are fabrications of my brain and a piece of paper and my be horribly inaccurate, or possibly even accurate. Take nothing for granted, and I accept no responsibility over what these numbers may chose to do on their night off.

®

Build a business case: developing custom apps

More from The Register

next story
Why has the web gone to hell? Market chaos and HUMAN NATURE
Tim Berners-Lee isn't happy, but we should be
Microsoft boots 1,500 dodgy apps from the Windows Store
DEVELOPERS! DEVELOPERS! DEVELOPERS! Naughty, misleading developers!
Mozilla's 'Tiles' ads debut in new Firefox nightlies
You can try turning them off and on again
'Stop dissing Google or quit': OK, I quit, says Code Club co-founder
And now a message from our sponsors: 'STFU or else'
Apple promises to lift Curse of the Drained iPhone 5 Battery
Have you tried turning it off and...? Never mind, here's a replacement
Uber, Lyft and cutting corners: The true face of the Sharing Economy
Casual labour and tired ideas = not really web-tastic
Linux turns 23 and Linus Torvalds celebrates as only he can
No, not with swearing, but by controlling the release cycle
prev story

Whitepapers

Top 10 endpoint backup mistakes
Avoid the ten endpoint backup mistakes to ensure that your critical corporate data is protected and end user productivity is improved.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Backing up distributed data
Eliminating the redundant use of bandwidth and storage capacity and application consolidation in the modern data center.
The essential guide to IT transformation
ServiceNow discusses three IT transformations that can help CIOs automate IT services to transform IT and the enterprise
Next gen security for virtualised datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.