Feeds

Fedora servers are that big

Download Your Way

  • alert
  • submit to reddit

Protecting against web application threats using SSL

Letters Yesterday's article on Fedora's FC6 download travails prompted a torrent of emails, not all abusive.

Much to our surprise we stirred up something of an ontological debate, as readers attempted to answer that age-old question: how big is big? Choice cuts below.

The Fedora project has apparently been subject to a sustained DDoS attack. That someone decided to attack on the scheduled release date is unlikely to be a coincidence, and would therefore probably be in the hope of getting Fedora bad press. Well done for falling for it and supporting hackers! Jonathan Larmour

Umm...always happy to oblige, Jonathan.


Well they did say that BitTorrent was the preferred method of downloading FC6, and sure enough I was able to use this method so that the ISOs were sitting waiting for me this morning. FWIW the site breakage also stopped most of the update applications, like yum and Yumex etc, from being able to get any Fedora updates at all, and as ever, there are a whole collection of day 0 updates for FC6 due to the extended package freeze before the release date. Yours enthusiastically, Brian

The Redhat/Fedora download sites were having issues before FC6 was announced. I was helping a friend early last week with a dedicated server install, adding some software he needed, and the tool that talks to fedora servers to get lists of current software (yum) was timing out roughly 3/4 attempts in any given hour.

My conspiracy theory: a competitor bought some zombie boxes and is DDoSing fedora judiciously.

More realistic: I didn't check but I suspect that maybe the FC6 downloads were available (in some fashion) as early as last week ... and the enthusiasts were being so damned enthusiastic.

Jacob L E Blain Christen


To keep things even more interesting, existing users of FC5 have not been able to update their servers for 2 days now :)

Wayne


Red Hat's 'low availability OS' - hardly.

I hit the torrent yesterday, got it in a couple of seconds (it's less than 100 KB), and the ISO came roaring down the line at 600 KB/sec, about as fast as it ever is for me. There were over 5000 users in the swarm I was part of.

Anyone (especially geeks) downloading a 3.25 GB file using HTTP needs a bitchslap, not a faster RedHat pipe. Torrents are very much the right way to do it.

65 TB an hour by the way is a *very* big chunk of bandwidth in anyone's book (2000*3.25GB) - it's 1.8 GB a second (14.4 Gbit/sec) before you consider any overheads (which using HTTP is significant), other page loads, resends for people with dodgier links, etc.

Cut them a little slack please, your headlines are sometimes in danger of heading from friendly irreverence over the line into Daily Mail (spit) "how could this happen?"-style headlines, and that would be a shame, as I usually like your style and the info I get from you.

Dan

Consider us bitchslapped. But the Daily Mail? We didn't mention immigrants once. I've doublechecked.


Don't be misinformed. Linux can handle unreal web loads on one machine. Redhat Just was not setup to scale downloads to that level. The Worlds busiest single machine/server is one (1) linux box. :)

"1.4 The Numbers At the time of writing (May 2005), ftp.heanet.ie; can sustain at least

  • 27,000 simultaneous HTTP downloads
  • can saturate a gigabit interface at the same time
  • has roughly 3.2 million downloads per day
  • ships about 3.5 Terabytes of content per day
  • mirrors over 50,000 projects
  • as 3.7 Terabytes of unique content
  • has circa 6 million unique files ... (on IDE hard drives!)

    http://www.stdlib.net/~colmmacc/Apachecon-EU2005/scaling-apache-handout.pdf

    anon-o-mouse


Well big is always just a matter of persepctive. The thing to bear in mind is that it's 10,000 copies of probably a dvd iso, 3.5 or 4 Gb depending on what your favorite flavor is. So that's about 37Tb of data downloaded over the 5 hours. To manage such bandwidth the other open source tool out this week Firefox would need to have 7.5 million downloads over the same time period, on the page your news story was served on would need to serve up 1.2 billion page views (at 30k a page I realise that there's css and images, but any devoted reg reader probably has them cached ;-) )

To keep up with that kind of data rate they'd need what 1.7 Gbit of outgoing bandwidth, so even to have managed what they did they need more than a unix box or two I'd have thought, or at least more than two network adaptors. And all of this ignores thier normal traffic, and the related traffic (people browsing the fedora pages about the download, and then browsing other stuff on redhat's network while they wait for the download) Not that I disagree with your general feel that they could have done better, but it's not exactly a small challenge.

Rob

NB these numbers are fabrications of my brain and a piece of paper and my be horribly inaccurate, or possibly even accurate. Take nothing for granted, and I accept no responsibility over what these numbers may chose to do on their night off.

®

New hybrid storage solutions

More from The Register

next story
New 'Cosmos' browser surfs the net by TXT alone
No data plan? No WiFi? No worries ... except sluggish download speed
'Windows 9' LEAK: Microsoft's playing catchup with Linux
Multiple desktops and live tiles in restored Start button star in new vids
iOS 8 release: WebGL now runs everywhere. Hurrah for 3D graphics!
HTML 5's pretty neat ... when your browser supports it
'People have forgotten just how late the first iPhone arrived ...'
Plus: 'Google's IDEALISM is an injudicious justification for inappropriate biz practices'
Mathematica hits the Web
Wolfram embraces the cloud, promies private cloud cut of its number-cruncher
Mozilla shutters Labs, tells nobody it's been dead for five months
Staffer's blog reveals all as projects languish on GitHub
SUSE Linux owner Attachmate gobbled by Micro Focus for $2.3bn
Merger will lead to mainframe and COBOL powerhouse
iOS 8 Healthkit gets a bug SO Apple KILLS it. That's real healthcare!
Not fit for purpose on day of launch, says Cupertino
prev story

Whitepapers

Secure remote control for conventional and virtual desktops
Balancing user privacy and privileged access, in accordance with compliance frameworks and legislation. Evaluating any potential remote control choice.
WIN a very cool portable ZX Spectrum
Win a one-off portable Spectrum built by legendary hardware hacker Ben Heck
Storage capacity and performance optimization at Mizuno USA
Mizuno USA turn to Tegile storage technology to solve both their SAN and backup issues.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
The next step in data security
With recent increased privacy concerns and computers becoming more powerful, the chance of hackers being able to crack smaller-sized RSA keys increases.