Google devs try to create new global namespace
Universal file sharing is hard because Services and CDNs scramble HTTP. Enter 'Upspin'
Wouldn't it be nice if there was a universal and consistent way to give names to files stored on the Internet, so they were easy to find? A universal resource locator, if you like?
The problem is that URLs have been clunkified, so Upspin, an experimental project from some Google engineers, offers an easier model: identifying files to users and paths, and letting the creator set access privileges.
By the by, yes: Dropbox, Google, Apple, Microsoft, and World+Dog offer file-sharing – but Upspin's authors reckon file sharing that way is also clunky.
Most of all, however, it's the big names (Google included) that Upspin's engineers reckon are making it hard to share stuff. As they note in their documentation:
Photos that one uploads to Facebook or Twitter are managed by, and effectively belong to, those providers.
Other than occasional workarounds using a URL, information given to these services becomes accessible only through those services. If one wants to post a Facebook picture on one’s Twitter feed, one does that by downloading the data from Facebook and then uploading it to Twitter. Shouldn’t it be possible to have the image flow directly from Facebook to Twitter?
This 'information silo' model we have migrated to over the last few years makes sense for the service providers but penalises the users, those who create and should therefore be in charge of their data.
The Google software engineers who put Upspin together (Andrew Gerrand, Eric Grosse, Rob Pike, Eduardo Pinheiro and Dave Presotto) add that users need more granular access control than “private” and “public”, plus better security and privacy controls.
Hence Upspin: not an application, but a framework to create something like a global namespace based on a user, identified by their e-mail address (hello,
firstname.lastname@example.org, you never answer my messages any more).
As explained at the project's repository (there's a mirror at GitHub), “it is not a file system, but a set of protocols and reference implementations that can be used to join things like file systems and other storage services to the name space.”
The naming framework is ridiculously simple: it's based on filepaths that attach to the user, thus
email@example.com/dir/file is what an Upspin service will evaluate to reach the file Ann@example.com shares.
There's no concept of a “local path” in the system – every file including local files, have to be fully qualified. And, in a borrowing from Unix, files can also be links (for example, to directories or other users' files).
Users grant access to their files with a file which, per-directory, identifies who has access. With no access file,
firstname.lastname@example.org/dir/file is private to Ann; if Access exists in the directory, Ann can give others access to that directory with a simple:
read: email@example.com, firstname.lastname@example.org
The structure that puts this together is also straightforward: a key server holds the public keys for all users in the system; a storage server holds the data, indexing it using a “reference, which for the default implementation is a hash computed from the contents of the data.”
Finally, there's a directory server (or, as the system scales, many directory servers). They keep the links between data on the storage server, and the user who owns the data.
Upspin's authors have also designed it so if someone were to compromise the infrastructure, individual users' data is still safe: the user holds the key to their files, and encryption happens in the user's machine, not in the cloud.
And yes: it was unfair to compare this to re-inventing the Web. It's just a source of wry amusement that “how do I share this?” has been so intractable for so long. ®