Feeds

British Library tracks rise and fall of file formats

Analysis of 2.5 billion online files suggests software obsolescence slowing

  • alert
  • submit to reddit

Business security measures using SSL

File formats and the software capable of reading them are living longer than previously thought, according to a British Library and UK Web Archive study.

Formats over Time: Exploring UK Web History (PDF, slides as PDF) considers 2.5 billion files author Andrew N Jackson retrieved with the help of the Internet Archive and the Joint Information Systems Committee (JISC). All the files come from “the UK web domain” and come from the period between 1996 and 2010.

Jackson used Apache Tika and PRONOM's DROID tool to inspect the files and determine the format they use. Central to the research was Jeff Rothenberg's 1997 prediction that “Digital Information Lasts Forever – Or Five Years, Whichever Comes First.” Jackson is also keen on a rebuttal from David Rosenthal, who he quotes as saying: “When challenged, proponents of [format migration strategies] have failed to identify even one format in wide use when Rothenberg [made that assertion] that has gone obsolete in the intervening decade and a half.”

Jackson's take is that file formats seem to last rather longer than five years even if they don't survive forever.

“While there were just two active versions of HTML in 1996 (2.0 and 3.2), all six were still active in 2010,” he writes. “Similarly, there were three active versions of PDF in 1996 (1.0-1.2) and eleven different versions in 2010 (1.0-1.7, 1.7 Extension Level 3, A-1a and A-1b, with 1.2-1.6 dominant). In general, it appears that format versions, like formats, are quick to arise but slow to fade away.

HTML versions found online in the UK between 1996 and 2010

Jackson attributes formats' longevity to the Network Effect, but also writes that he is uncomfortable drawing firm conclusions about software obsolescence given the sample is UK-centric and the tools used to analyse data identify files imperfectly.

He nonetheless concludes:

Our initial analysis supports Rosenthal's position; that most formats last much longer than five years, that network effects to appear to stabilise formats, and that new formats appear at a modest, manageable rate.

But he also warns that “a number of formats and versions that are fading from use, and these should be studied closely in order to understand the process of obsolescence.” ®

Choosing a cloud hosting partner with confidence

More from The Register

next story
'Windows 9' LEAK: Microsoft's playing catchup with Linux
Multiple desktops and live tiles in restored Start button star in new vids
Not appy with your Chromebook? Well now it can run Android apps
Google offers beta of tricky OS-inside-OS tech
New 'Cosmos' browser surfs the net by TXT alone
No data plan? No WiFi? No worries ... except sluggish download speed
iOS 8 release: WebGL now runs everywhere. Hurrah for 3D graphics!
HTML 5's pretty neat ... when your browser supports it
Greater dev access to iOS 8 will put us AT RISK from HACKERS
Knocking holes in Apple's walled garden could backfire, says securo-chap
NHS grows a NoSQL backbone and rips out its Oracle Spine
Open source? In the government? Ha ha! What, wait ...?
Google extends app refund window to two hours
You now have 120 minutes to finish that game instead of 15
Intel: Hey, enterprises, drop everything and DO HADOOP
Big Data analytics projected to run on more servers than any other app
SUSE Linux owner Attachmate gobbled by Micro Focus for $2.3bn
Merger will lead to mainframe and COBOL powerhouse
prev story

Whitepapers

Providing a secure and efficient Helpdesk
A single remote control platform for user support is be key to providing an efficient helpdesk. Retain full control over the way in which screen and keystroke data is transmitted.
WIN a very cool portable ZX Spectrum
Win a one-off portable Spectrum built by legendary hardware hacker Ben Heck
Storage capacity and performance optimization at Mizuno USA
Mizuno USA turn to Tegile storage technology to solve both their SAN and backup issues.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Security and trust: The backbone of doing business over the internet
Explores the current state of website security and the contributions Symantec is making to help organizations protect critical data and build trust with customers.