Software

Open source plugin aims to defeat link rot

The death of URLs greatly exaggerated

Chains image via Shutterstock

A new open source plugin designed to prevent the creation of dead content links online – so called "link rot" – has launched.

Amber has been designed by Harvard's Berkman Center for Internet and Society and it provides what it calls a "persistent route" to information on the internet by automatically taking and retaining a snapshot of every page on a website and storing it on the same website's server.

In other words, it's a Wayback Machine for your own website. Or perhaps think of it as a mirror of your website with minimal fuss.

If for whatever reason a URL goes dead, rather than returning a 404 error page, the tool should provide visitors with the relevant snapshot. The snapshots are stored on the same server as the website but can be configured to save them on third-party systems or in archival systems.

Perhaps unsurprisingly, the project has been supported by the Internet Archive (Wayback Machine), Amazon Web Services, and Perma.cc (another online archiving service).

How and why

There are lots of reasons for link rot: websites are restructured or shifted to a new content management system and break all the previous URLs; articles get moved behind a paywall; people delete social media accounts or change their privacy settings; or links contain information that goes out of date (such as session info).

Highlighting the depth of the problem, a paper back in 2013 written by one of the instigators of the problem, Harvard academic Jonathan Zittrain, highlighted that nearly half – 49 per cent – of the links in Supreme Court decisions went nowhere. Likewise over 100,000 Wikipedia articles contained dead links.

There is also the risk of DDoS attacks on big storage providers or hosters taking down millions of webpages, and of course censorship where government or other entities limit or block specific URLs because of the content they contain.

The Amber system is available as plugins for the two most popular CMSs out there: Wordpress and Drupal. It is also available, however, as modules for Apache and Nginx for people running custom systems.

The Berkman Center says Amber is a work in progress and is asking for feedback through email or via GitHub. ®

Sponsored: Customer Identity and Access Management