Feeds

Tackling Apache zombies

Myths and legends laid to rest

High performance access to file storage

The net is full of Apache knowledge. Tips and tricks, discussion fora, experts of all kinds, and innumerable "how-tos" and tutorials on a range of subjects. Some of these are worth reading; others may be otherwise.

Among all this wisdom are some myths and half-truths that are so well known as to be "common knowledge". They regularly need to be "un-explained" in the support fora. They typically have origins in the very early days of Apache and its predecessor the NCSA server, and were either true in the distant past, or appeared in tutorial examples which gathered momentum to become, in cargo cult terms, the One True Way. Now, zombie-like, they refuse to die.

So, let's tackle some of them here. I've selected four that can lead you into trouble if misunderstood. But before launching into them, here's a public service announcement, particularly for those who missed ApacheCon in Dublin for reasons of geography. Two more ApacheCon conferences are coming up shortly:

  1. Asia: Colombo, Sri Lanka, 14-17 August
  2. USA: Dallas, Texas, 9-13 October

Password Protection and .htaccess

One of the most pervasive myths around Apache is that password protection means .htaccess, and vice versa. While not entirely false, this belief is misleading, and often leads to configurations that are unnecessarily complex and slow. Apache itself may be guilty of confusing users by virtue of the naming scheme: the similarly-named htpasswd and htdigest programs are indeed concerned with password protection.

To understand the role of .htaccess, we need to take a look at Apache configuration. The majority of Apache's configuration can be set per-directory, so that http://www.example.com/example/ may behave entirely differently to http://www.example.com/, etc. This is based on the <Directory> container in httpd.conf, and its cousins <Location>, <Files>, etc. Password protection is just one of many aspects of configuration that can be determined per-directory.

Now, the true role of .htaccess files is to enable some aspects of Apache configuration to be determined outside httpd.conf, so it can be delegated to users who have no privilege to mess with the core of the system. The AllowOverride directive determines what aspects of configuration can be set in .htaccess files, and AllowOverride is itself something that can be set per-directory.

Now, you may sometimes encounter instructions to create a .htaccess file, containing something like:

  AuthType Basic   AuthName "My Application"   AuthUserFile /path/to/my/userfile   Require valid-user

together with setting

  AllowOverride AuthConfig

in httpd.conf to enable it.

This is harmful for two reasons. Firstly, it's complex: it introduces new and unnecessary issues concerning management of, permissions on, and (not least) security of, the .htaccess files themselves. Secondly, it's slow: as soon as you set AllowOverride to anything other than its default setting of None, Apache has to look for and read the .htaccess files (possibly more than one, where subdirectories are involved) for every hit!

Just say no to .htaccess! Whatever goes in a .htaccess file can always go directly into a corresponding <Directory> stanza. The only people to use .htaccess should be end-users who want control without having to bug the server admin.

AddType Abuse

The AddType directive serves to associate a filename "extension" with a MIME type, so the server sets the correct Content-Type header for the Client to handle the contents.

In the NCSA server and in Apache 1.0, this was extended in a rather dirty hack to create "magic" types that would invoke special processing on the server. Common instances were CGI scripts and server-side includes, and later PHP.

This hack became redundant with the introduction of the AddHandler directive in Apache 1.1 (1996). Now we have a clean distinction between handlers (serverside processing) and MIME Types (which describe contents so that browsers and other Clients can handle them correctly). But ten years on, the abuse of AddType to determine serverside processing continues to muddy the waters. This, perhaps more than any other Apache zombie, has advocates who really should know better.

Don't limit the limits!

Another password-protection myth that is mercifully near-dead concerns the use of <Limit ...> sections. These appeared in some early example documentation, from where they were widely copied out of context and grew into a myth.

The <Limit> container serves to say "this limit applies only to the HTTP methods listed". So you're granting unlimited access using other methods. This is rarely what you want, although it does have valid uses. The most common valid case is where you use Limit alongside a matching LimitExcept to determine different access policies for different operations. WebDAV and subversion commonly use this form, and provide good examples. For other password protection policies, you should normally just ignore it.

The case-insensitive URL

The case-insensitivity myth goes something like "Apache on Windows is case-insensitive; on *X it's case sensitive". And so it appears, though Apache's mod_speling(sic) - or an AliasMatch or RewriteRule - can change this behaviour.

Underlying this is a half-truth: that URLs correspond to the filesystem. They may indeed do that, but this is no more than a convenient convention. URLs are in fact defined unambiguously by RFC2396 to be case-sensitive (in the "file" part). A server that treats them otherwise is in fact serving the same contents at multiple different URLs.

Servers that give the illusion of being case-insensitive are actively harmful to the 'net infrastructure. That's because agents such as caches and search engines MUST treat each URL as different. So creating multiple versions is in effect spamming them (and may get a site penalised in search engines that watch out for spamming).

To see the potential extent of this spam, note that each "case-insensitive" letter doubles the number of spammed URLs. So for example, an eight letter "case-insensitive" URL is in fact 256 URLs, and at 20 letters, it rises to over a million!

And now the light is failing and I'm pulling the covers over my head – before the zombies get me... ®

High performance access to file storage

More from The Register

next story
Windows 8.1, which you probably haven't upgraded to yet, ALREADY OBSOLETE
Pre-Update versions of new Windows version will no longer support patches
Android engineer: We DIDN'T copy Apple OR follow Samsung's orders
Veep testifies for Samsung during Apple patent trial
OpenSSL Heartbleed: Bloody nose for open-source bleeding hearts
Bloke behind the cockup says not enough people are helping crucial crypto project
Microsoft lobs pre-release Windows Phone 8.1 at devs who dare
App makers can load it before anyone else, but if they do they're stuck with it
Half of Twitter's 'active users' are SILENT STALKERS
Nearly 50% have NEVER tweeted a word
Windows XP still has 27 per cent market share on its deathbed
Windows 7 making some gains on XP Death Day
Internet-of-stuff startup dumps NoSQL for ... SQL?
NoSQL taste great at first but lacks proper nutrients, says startup cloud whiz
US taxman blows Win XP deadline, must now spend millions on custom support
Gov't IT likened to 'a Model T with a lot of things on top of it'
Microsoft TIER SMEAR changes app prices whether devs ask or not
Some go up, some go down, Redmond goes silent
prev story

Whitepapers

Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.
Five 3D headsets to be won!
We were so impressed by the Durovis Dive headset we’ve asked the company to give some away to Reg readers.
HP ArcSight ESM solution helps Finansbank
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Mobile application security study
Download this report to see the alarming realities regarding the sheer number of applications vulnerable to attack, as well as the most common and easily addressable vulnerability errors.