How do you make the electricity bill for your home computing disappear? I decided to try to find out.

Historically computer kit had been responsible for about 90 per cent of our electricity bill at my home office for my ISP and consultancy business. I ran one of the UK’s first ISPs, and even in my new, smaller home, I was getting a bill of £1,000 a year.

In late 2007 I caught the energy-efficiency (and climate-change) bug. I had become worried by equipment that was overheating in summer, and anyway we needed the space for another child on the way in 2008.

The two rounds of therapy I describe here almost entirely eliminated that bill; my annual business 'leccy use at home now probably costs less than one purple drinking voucher, ie £20, and it is practically invisible. I also got a room back without moving or building an extension.

Do your homework first

You need to measure to manage. A simple plug-in power meter showed that all the servers used well over 600W, with the networking drawing another 40W. That didn't include the air-con that I needed in summer to keep it all from melting.

Monitoring the servers confirmed that none of them was then heavily loaded for CPU, I/O, or any other parameter.

All I needed now was essentially a small home office (SoHo) style-server, and *nix being what it is, consolidating all of the functions on to a single machine was not going to be too tricky. None of my main applications were tied to a particular machine architecture or OS – eg sendmail, Web servers including a big Java/JSP site, and DNS.

In megacorps, especially on Windows boxes, it can be very helpful to virtualise servers so that they still think that they have the whole machine to themselves, but with my relatively well-behaved apps that wasn't necessary.

Bringing everything to one box also would also eliminate the need to run a complex internal network.

So, in the first round of consolidation, I brought all the applications onto a single laptop running Linux, with a few GB of memory. This consumed somewhat under 30W on average, as little as 17W when quiet.

Some of the apps, especially the Java Web server, had to be reined in a little, especially with regards to memory and disc usage, and storage has proven to be the most tricky recurring themes in this downsizing story.

Tweaking

Various techniques were used to trim power consumption. I'll describe them in detail below.

I minimised filesystem activity by turning off pointless logging. I used solid-state storage, spinning up a magnetic disc only occasionally. I turned on "laptop mode" to minimise disc spin-ups. I used a "tickless" Linux kernel and using the "ondemand" CPU speed governor. I also made sure that apps let the CPU sleep as much as possible.

I also spent time with the devices - autosuspending as many devices as possible. The major applications – such as Apache, sendmail, NTP, etc – were tuned for CPU efficiency, and any necessary applications and hardware (eg the backlight and WiFi) were disabled. The networking was simplified to reduce consumption to 8W for one ADSL router. The upshot of this part of the story is that it all worked. It also saved the cost of the new equipment in electricity in the first year and every year since. These are the kind of savings that both a bean-counter and a tree-hugger will love (over 6MWh per year of electricity saved and 3 tonnes of CO₂ emissions) [The Advertising Standards Authority says that you can only claim 430 grammes of CO₂ per kilowatt-hour. Try 2.58 tonnes - Ed]. Indeed the power consumption was so dramatically reduced that I could run the server off a solar panel in my back garden up to 12 hours per day.

Interested in the gruesome details?

Storage, storage, storage

I expected that memory might be an issue, but a bit of tweaking of configuration of sendmail (maximum simultaneous incoming connections) and Apache and my big Java app overcame most of that fairly easily at this stage.

Having less storage available after moving away from big networked RAID discs was more of a problem. In the end it turned out that I didn't need to keep years' worth of logs for possible analysis on my death-bed for example. But it did take quite a lot of admin time and soul-searching to discard them.

Also, I set the system up to try to boot and run from solid-state media as much as possible to minimise power consumption (an SD card for the laptop), with the hard disc only spun up when necessary for "bulk" storage.

This brought up a feature of excessive write to storage in particular: such writes might prematurely wear out the SD card.

So a significant amount of effort went into reducing unnecessary logging and writing without losing anything essential if the system went down. For the laptop, forcing "laptop mode" most of the time was very helpful. And making sure that /tmp was in tmpfs, ie RAM, was also good.

Java is not enough: introducing Sheeva

I was filled with the zeal of the reformed, and the circa-30W consumption of the laptop was a target that I thought I should be able to beat easily.

In particular I wanted to be able to run the server off-grid (ie from solar PV panels and battery) all year round. I also wanted to get my system fully solid-state and silent.

I found out about SheevaPlug, an "embedded" System-on-Chip (SoC) Linux box, small enough to plug into a mains power socket at about the size of a chunky power transformer for a games console or similar, though a discussion of the Shark/Zero Java port to the ARM chip inside the SheevaPlug.

A SheevaPlug can consume under 4W if quiet and carefully configured, despite packing 512MB of Flash and 512MB of RAM, SD and USB slots for more memory and other peripherals, and an 100Mbps Ethernet socket.

The step down from the Linux laptop represented about a halving of CPU speed (similar CPU GHz at 1.2GHz for the SheevaPlug ARM, but half the work per Hz) and RAM chopped by 75 per cent, but it looked do-able, and with a many-fold reduction in power. Can you say "challenge"?

To get away from mag disc was going to be tough, and powering such a disc through spin-up would also be tricky, even a "portable" or USB-powered disc, so in the end I settled on the biggest (32GB) SD card that I could plug in to hold the OS and most of the frequently-used data, and a 128GB USB thumb-drive (yes GB, not MB) to hold the "bulk" data that used to be on the laptop's mag disc, with some room to spare.

Both were pretty expensive per GB compared to smaller devices, and the Kingston DataTraveler 200 USB 128GB flash drive (£260) was only just about available when I needed it. I had just under 60GB of bulk data to put on it at that point, growing slowly.

System on a plug: Unboxing the Marvell Sheeva

The community at plugcomputer.org was very helpful with booting a usable system, including a new enough kernel (2.6.31) to contain much of the important energy-saving features such as the "tickless" clock (which wakes up only as necessary, not every millisecond just in case).

One unfortunate detail is that the Sheeva came with Ubuntu 9, and Ubuntu abandoned support for the Sheeva version of ARM (armv5tel) in 10. So any significant upgrade would require a move to another distribution such as Debian (upon which Ubuntu is based). But I haven't felt the need, and many of my mirrors elsewhere in the world run much older Linux releases.

The memory issue was quite pressing, and I had to, for example, limit Apache and sendmail to fewer threads/children than usual, and to run with a smaller stack (much less than the default 8MB), though this only required config and script hacking, no (re)compilation, in part because I am running the system with no swap. Using solid-state media as swap could wear it out in no time at all.

Java (the Tomcat Web server) had to be run with much less heap than ideal (about half of physical memory, ie 256MB, compared to a more usual 1GB), and I had to spend a lot of effort over many months tracking and squashing slow and small-but-deadly memory "leaks" (over-eager cacheing in many cases) in my code. The upside of this work is that on my bigger mirror hosts the code is now super-efficient with its memory. In fact the code adapts at run-time to both the available heap/memory and the number of CPUs, ranging from 1 CPU and 256MB on the SheevaPlug to 24 CPU threads and several GB on a Sun Niagara host, which all adds to the fun (yes, it's what I like doing).

For all-flash storage, reducing "disc" activity to minimise power consumption is a must, and reducing write activity to maximise lifetime is also important. I also laid out the filesystems (efs3) as suggested by Theodore Ts'o to line up allocation boundaries with the likely flash block boundaries to minimise wear, as well as setting long timeouts between write flushes and avoiding noting each time a file is read (accessed) using the mount options noatime,commit=600.

For example, here is the set-up of one partition on flash media:

fdisk -H 224 -S 56 /dev/sda

To create cylinders of 49*128kB, with a small leading VFAT utility partition:

mkfs -t vfat /dev/sda1

To allow the main/second partition to be cylinder- and erase-block- aligned, then to make that labelled main ext3 partition with better-than-default sizing (no space for expansion, directory indexes, sparse superblocks, an inode per 128kB, which is a decent safety margin over the current 500kB/file usage). I ran:

mkfs -t ext3 -i 131072 -O ^resize_inod,dir_index,sparse_super /dev/sda2

I also rigged the flash thumb-drive partition to be powered up and automounted on use, and unmounted and powered down when idle to reduce the otherwise considerable power draw from USB hubs with downstream storage keeping them awake.

More worryingly, though, without any major catastrophe so far, is that the USB flash drive and ext3 combo seems occasionally to lock up, fail to write a bunch of data, and then requires some manual surgery after an fsck. Alarmingly, some files that initially seem to have been written correctly look very broken after a restart/fsck. As far as I know, I've not lost anything major, and I systematically keep checksums (and backups) of all the important files, but it's a major issue that I will need to resolve.

Embedding the Shark

On the Java front, one downside of going to ARM on the Sheeva is that the "standard" Sun-supplied Intel Linux JVM builds are not available.

I looked at the free options such as Zero/Shark on IcedTea, and I've tested various versions of it with the help of the developers, and it is getting better over time. It's certainly a good fall-back.

But Sun made available a copy of their "embedded" ARM JRE available to me (before being swallowed up by Oracle), which works pretty well. It's not a "server" VM (it could probably run a little faster if it were), and not being a full JDK I had to upgrade from Tomcat 4 to Tomcat 6 to be able to use it. It also seems that the garbage collectors other than the default serial GC are still a little buggy, causing the occasional crash, but I'm happy with it.

Note that the ARM chip in the Sheeva has no hardware floating-point support, but then floating point is pretty slow on my Niagara box too, so I'd already converted everything performance-critical to fixed point.

For me WORA (Write Once Run Anywhere) really does work, as my Java Web server runs from the same WAR file on the SheevaPlug (ARM) and on Solaris SPARC and various Linux Intel x86 mirrors.

Even with nominally enough solar PV and battery last year (more than 100Wp and 250Wh respectively, for a 4W typical consumption) I had to run the SheevaPlug on mains for most of the winter of 2009/2010. This year I've about doubled the PV and battery capacity in the hope that I can stay off-grid even in the bleak mid-winter...

The system monitors its own battery state via a k8055 USB I/O card, avoiding optional tasks (thus energy consumption) when the battery gets low, and conversely cranks my AI-based image-rating system when the sun is out and the batteries bursting at the seams.

Lessons Learned

After a year of relatively reliable and silent operation, the SheevaPlug hosts all my main Internet-facing servers in the space of a pack of cards and consumes less power than a typical PC in standby!

I'm glad that I went in two steps via the Linux laptop else the effort might have been overwhelming; the SheevaPlug has been in the end just more of the same, evolution not revolution.

I need to resolve the USB flash issue one way or another: maybe by experimenting with a different brand of drive, or a different file system (such as btrfs), or even by trying out a purpose-built SSD external USB drives. I want to find out why I'm seeing this transient flaky behaviour.

I may also try having a second SheevaPlug mirror come on as "dump load" when the sun is out and the batteries are full, to provide more bandwidth to users and make use of the otherwise "lost" energy. I have a spare sitting waiting, and at only about £100 why not?

Memory and storage have been the unexpected jokers in the migration from a bank of conventional Sun Solaris servers down to a single embedded Linux box. But for a couple of orders of magnitude of reduction in energy consumption saving the better part of £1,000 every year, the liberation of a room in our smallish house, and quite a lot of fun, I'd be happy to do the project all over again. And if a lower-power successor with just a bit more memory becomes available, maybe I will... ®

For more gruesome detail see my more detailed SheevaPlug write-up, and its predecessor, the Linux laptop.

Damon Hart-Davis is an IT consultant who has spent most of the past 15 years in the City, with a long-standing interest in small and low-power devices and various flavours of UNIX. He founded one of the first British ISPs, virtual credit card company, and once edited a supercomputer magazine.