Apache packages: creating a support vacuum

OSS in the real world

Choosing a cloud hosting partner with confidence

Comment Right at the end of OSCon in September, I got the opportunity to harangue Ubuntu's Mark Shuttleworth about the support vacuum distros such as his create. He didn't have an answer. OK, I was putting him on the spot, but I don't think he'd have done much better if he'd had notice of the question. Frankly, I don't think there is an answer.

So what's the problem? We at Apache know our own software, and are prepared to support it in fora such as the Apache Users mailinglist, and the #apache IRC channel.

Among the diverse issues we have to deal with are the many users who have installed Apache from a third party package. Many of these packages are rearranged so as to be essentially unrecognisable; and in particular, some packagers have very bizarre ideas about how to organise Apache's configuration files. That makes it hard, sometimes impossible, to support the end-users: we may be talking completely at cross-purposes. And for a change, the worst problem is not with Windows packages, nor with commercial Apache packagers like IBM and Oracle (for which, at worst, we can disclaim all responsibility), but with Linux distros. By far the biggest offender, in terms of the support headache it throws on us, is the Debian family, notably Debian itself and its near-clone Ubuntu.

Now, if we at Apache stick to what we know about, we'd tell the questioners to go to a Debian support forum. But the Debian people will tell them to use an Apache forum. Where should the burden of support lie?

newbie: How do I stop it listing all my files when there's no index.html?
niq: Options -Indexes
fajita: Options -Indexes turns off directory indexing. Ask me about mod_autoindex and Options for more details.
newbie: So where do I set that?
niq: httpd.conf
...fajita explains how to find httpd.conf ...
newbie: httpd.conf is empty. Do you mean apache2.conf? Or something in conf.d/?
niq: Probably. See your distro's docs.
newbie: huh? it's just a standard apache installation.
niq: distros
fajita: distros sometimes mangle your apache installation, putting apache and its files in different places to the standard. It's hard for us on #apache to help if your installation is mutilated this way.

And no one is satisfied. And that's just a simple example: others involve a lot of confusion with nonsense like, for example, per-module conditional .conf files, and distro-specific ways of manipulating them.

They've configured it right, but Apache ignores it because it needs some extra distro-defined magic to activate the configuration file in question. Now they're banging their head in desperation, and we can't help because we don't know the magic. And we get the blame.

You might of course reply: therein lies a great opportunity; a niche market waiting to be exploited. And yes, there are people and companies (your humble scribe included) who earn our bread and water in something not far from this niche. Though far from being as lucrative as, say, supporting Oracle (with open source and decent documentation, the barrier to entry is far lower), it works. But that's at the top end: medium and large businesses, with custom needs and a budget to meet them. Not all Apache users fall into that category! So, that still leaves a support vacuum.

Back to Ubuntu and my encounter with Mark Shuttleworth. I put the question to him in general terms, citing Apache merely as the example I've had experience with at the sharp end of support. His answer, in similarly general terms, was that Ubuntu would expect to be the first point of support, but he would hope that upstream providers would also get involved. Sorry, but that just doesn't work.

First, Debian/Ubuntu would have to provide a complete set of documentation, at the level of Apache's own. Otherwise, people will naturally come to apache.org, and get confused when it appears to bear little relation to what's installed on their computer. And even if they did provide a full handbook, that still leaves a Tower of Babel amongst third-party articles.

Second, sorry, but speaking as an Apache developer, I make the time to chip in to the task of supporting our users. But I draw the line at taking the time to figure out the quirks of someone else's gratuitously much-mangled package. If there was just the one distro, things might be different, but it's not just Debian/Ubuntu - it's Gentoo, Redhat, SuSE, FreeBSD, Windows, Solaris, etc, etc. If you want me to learn the quirks of your distro so I can support it, you can bloomin' well pay me! And to be fair, the commercial distros do pay Apache people - which may be one reason why the biggest support headache thrown back on us comes from the pure freebies.

Well, OK, so don't support distros then. Is it that hard?

Actually, yes it is, unless we stop supporting anything at all. We can't stop people with distro-induced trouble asking in our fora, and they will blame us (sometimes loudly and obnoxiously) if they don't get what they want. This creates real tension.

Now, there's an experimental effort underway to try to close this gap with documentation. The core of it is a wiki page, now at http://wiki.apache.org/httpd/Recipes/DistrosDefaultLayout. It's driven by Apache support people, frustrated with a constant stream of "huh? what are you talking about?" posts. It's not yet clear how useful this will be in closing the support gap; at the moment, it's essentially a quick-reference card for mapping between distros.

Having sounded off at distro packagers, I should say that they do of course, play a vital role. Packaging Apache, including as they do a lot of third party stuff, lowers the bar to entry for the many users who would be uncomfortable with the idea of compiling a package from source. Some third party packages can be challenging for a non-expert admin to build, and distro packagers can be exactly what they need.

Given all that to contend with, do we really need confusing packages to thrust extra complexities onto both us and our users? A plea to Debian and Ubuntu: if you must repackage Apache in a radically different manner, please document it properly! That includes, along with the manual, a prominent notice to your users, that Apache and third-party documentation and support describe something different, and will not always be applicable!

There's feedback on this article on Nick's blog here

Intelligent flash storage arrays

More from The Register

next story
Netscape Navigator - the browser that started it all - turns 20
It was 20 years ago today, Marc Andreeesen taught the band to play
Sign off my IT project or I’ll PHONE your MUM
Honestly, it’s a piece of piss
Return of the Jedi – Apache reclaims web server crown
.london, .hamburg and .公司 - that's .com in Chinese - storm the web server charts
Chrome 38's new HTML tag support makes fatties FIT and SKINNIER
First browser to protect networks' bandwith using official spec
UNIX greybeards threaten Debian fork over systemd plan
'Veteran Unix Admins' fear desktop emphasis is betraying open source
Admins! Never mind POODLE, there're NEW OpenSSL bugs to splat
Four new patches for open-source crypto libraries
Torvalds CONFESSES: 'I'm pretty good at alienating devs'
Admits to 'a metric ****load' of mistakes during work with Linux collaborators
prev story


Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Cloud and hybrid-cloud data protection for VMware
Learn how quick and easy it is to configure backups and perform restores for VMware environments.
Three 1TB solid state scorchers up for grabs
Big SSDs can be expensive but think big and think free because you could be the lucky winner of one of three 1TB Samsung SSD 840 EVO drives that we’re giving away worth over £300 apiece.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.