Victoria & Albert overwhelms museum SAN

London's Victoria & Albert museum has overwhelmed its original SAN with digitised images of its collection and is moving to a new one with room to grow more than 50 times larger.

The V&A has found that by becoming a museum without walls on the Internet, it has found there is a huge demand for on-screen access to its collection of more than 30,000 pieces of art and design, including ceramics (puzzle jug below), fashion, furniture, glass, metalwork, paintings, photographs, prints, sculpture, and textiles. It is a veritable encyclopeadia of arts and crafts and, naturally it has a storage problem - there are more stored artefacts than gallery space to display them.

By adding interactive computerised displays in its galleries and images of its collection on the web it also ran out of digital space. The physical storage problem had a £110 million program applied to it with new galleries opening from 2006 onwards. The digital space problem has proved far less expensive to solve.

What happened was that from 2001 onwards the museum had a SAN (storage area network) for its Digital Asset Management System, which provided access to images, interactive displays and such like. It meant that images of artefacts, for which there was no physical room to display, could be seen on a screen, digital access augmenting and replacing the limited physical access.

The software ran on NEC and Dell servers connected via gigabit Ethernet to EMC Clariion storage. Arcserve and Sun's Solstice backup software on a Solaris box were used to protect data on LTO-2 tapes in an Overland Storage library. It was great at the time, especially as the 1Gbit/s Ethernet replaced 10 and 100Mbit/s Ethernet links previously used. Data whizzed around the LAN like Ferrari racers in a Formula 1 Grand Prix instead of asthmatic delivery vans.

Time passed and digital data did what it always does: It grew and grew, until the nightly backups began overlapping the daytime work periods and the increasing digitation of images could be seen, inevitably, to be filling up the SAN. In 2001, 8 terabytes was a huge silo of data space. In 2008 it is a drop in a bucket.

The gigabit Ethernet has grown to a 4Gbit/s Ethernet courtesy of Nortel Networks so that is okay, but the digital storage, the SAN backend, is runing out of room.

The V&A's Sarah Winmill, its head of information systems services, says that the program to take photos of its collection can result in 30MB photos - these can be used for research activities - and 50,000 of them a year are being added. That's 1.5TB per annum. It gets worse. Three-dimensional aretefacts need multiple images. Artefacts need multiple images at different resolutions so researchers can zoom into details.

The museum is also involved in 3D photo developments and in the searching of photos with recognition software, partnering researchers at Southampton University.

For example, it is using QuickTime so web viewers can see a rotating 3D image of a Meissen vulture, looking suspiciously like the original El Reg mascot.

And then - here's a kicker - the V&A has films, analogue films, of theatre productions from London's West End. These could be digitised in future, meaning that a SAN capacity that maxes out at 8TB is simply not enough. The museum has to have tens of terabytes now and headroom to grow to 100s of terabytes when needed.

A tender was put out, a shortlist of three was built from the replies, and Hitachi Data Systems emerged as the preferred choice. The SAN is now based on a 60TB AMS 1000 array, which can hold up to 450TB using today's 1TB SATA drives, and an identical array mirrored to it. This is destined to go a nearby separate site as a disaster recovery and business continuance resource with data asynchronously replicated from the primary AMS box.

Sara Winmill says that the data transfer from the original SAN is about 70 per cent complete and the secondary AMS will go to its remote site when the migration finishes. The old SAN won't be discarded, UK public institutions "having to sweat their assets." But the future is disk-based replication of data for protection and away from tape. LTO-2 is, anyway, two generations behind the tape curve.

The V&A now has a SAN silo that can hold hundreds of terabytes should it need to. We can imagine that all museums and art galleries will espouse the same basic IT infrastructure. The possibilities for a virtual 3D tour of the V&A's galleries are quite staggering. Imagine Google street view applied to the inside of a building and 3D photographs of artefacts within it. The future digital storage and networking needs of these places could grow a hundredfold, a thousandfold, on a never-ending journey of digitisation.

It could get to the point where wealthy benefactors no longer build extensions to museums, like the Sainsbury Wing of London's National Gallery, but fund a digital tour of it and its collection instead. Victoria & Albert would not be amused at all. They'd be astonished. ®