Original URL: http://www.theregister.co.uk/2008/07/14/dziuba_google_protocol_buffer/

Google releases serialization scheme

Pedantic programmers hold love-in

By Ted Dziuba

Posted in Developer, 14th July 2008 14:24 GMT

Fail and You Protocol buffer: it’s the object serialization scheme the pretentious little shit on your development team has been talking at you about during lunch hours for the past couple of days. You’ve been feigning interest with a steady stream of “oh-yeahs” and “that-sounds-cools”, so you don’t really know what it is.

Well today is your lucky day because I’m about to drop some science up in this bitch.

It’s Like XML But From Google, Therefore It’s Awesomer

A protocol buffer is a way to serialize an object such that it can be shared across platforms. You define your object in terms of its primitives in a special language, run the protocol buffer compiler over your definition, and it will produce code in your language to serialize and unserialize this object. With Google’s code, you can share objects across any programming languages you want, as long as they are either C++, Java, or Python.

If you’re putting two kids through college, you likely know this technology by the name ASN.1. Otherwise, it’s just XML with a Google pedigree.

What makes the protocol buffer so popular with the pretentious little shits? Aside from wanting to put it on the CV they send to Google every three months, it’s got scalability written all over it. Oh, scalability: the problem that tens of thousands of engineers yearn for, but only six actually have.

Google invented the protocol buffer because they found XML parsing to be too slow, and XML messages too large. Now this is a good optimization in an environment that needs to process tens of millions of requests per minute, but it is unlikely that your company’s CRM system will benefit from it.

Even if you do have a scalability problem that Protocol Buffers could help, you’ll still be unlikely to use them, because putting your prize-winning testicles out on display as you draw boxes and lines on a whiteboard is most of the fun of solving scalability problems. You’ll be damned if Google is gonna cuckold you on this one.

Can you replace XML with protocol buffers? If you’re just at the outset of a project, it may be worth a look. XML, like a venereal disease, spreads from developer to developer by direct contact. The conservative school of thought suggests that the best way to avoid XML is to simply abstain from programming, but a more pragmatic approach can help to contain the spread of the pestilence while still allowing us to do our jobs. Since programmatic reflection is built into the implementation, it is easy to translate protocol buffers to other formats such as XML, so you can do it if you really have to.

It’s All About Makin’ That R-P-C

The protocol buffer definition language allows you to specify RPC servers that take protocol messages as input and output. The compiler will generate some code to help you with this RPC, but it will not give you an actual server. Google’s engineering team, overgrown with developers from academia, has left this as an exercise to the reader. This isn’t so much a shortcoming as it is another opportunity for you, the developer, to swing your dick around. You get to write a multithreaded RPC system with connection pooling and load balancing and all that shit.

Think of how scalable that shit’s gonna be. You’ll put a real hurt on all that imaginary load your system is taking. Then, you get to go home and fuck the prom queen.

If you really like the protocol buffer idea but aren’t so hot on writing your own RPC system, there is relief. About a year ago, Facebook released a project called Thrift that does essentially the same thing as protocol buffers. Thrift failed to gain heavy traction because its name isn’t terribly cool, nor does it give way to an acronym that contains the letters J or X. Nonetheless, Thrift does include a variety of RPC implementations.

Interestingly, one of Facebook’s engineers on the Thrift project, Mark Slee, worked at Google before his arrival at Facebook. It’s therefore unsurprising that Thrift and protocol buffers share many design decisions. Unfortunately, because of heavy inbreeding within Silicon Valley engineering teams, the two products, although functionally similar, are not compatible with one another.

If you want to do it, writing your own RPC layer isn’t a herculean task. I managed to hack something together on top of Tomcat in a couple of hours. It didn’t make me feel as manly as I hoped it would, so to supplement, I suggest you have two cigars, a glass of Maker’s Mark, no ice, and a copy of The Godfather trilogy within reach.

Alright, Wrap It Up, Windbag

Since the release of protocol buffers, the tech blogosphere (long regarded as the dominant scholarly force of the internet) has been chattering. Of course, none of these bloggers have actually used protocol buffers in any code that matters to anybody, but as you know, that is no reason to prevent the offering of a strong opinion on the matter. If you work with a pretentious little shit, you know this phenomenon all too well.

The Web 2.0 startup circle, being mostly composed of pretentious little shits, is likely to adopt protocol buffers as a first-class data interchange format. You, the legitimate developer at a corporation that actually earns money, have nothing to worry about from this uprising. These trends rarely escape from the echo chamber.

Considering everything about protocol buffers, the bottom line is this: you can continue to do the least possible amount of work and make the most possible amount of money just as you are doing. This technology is unlikely to move into widespread use, but if it does, you’re looking at a loss of 3 to 4 YouTube hours to learn enough of it so that you don’t look like an idiot at the next team meeting. If this sort of thing does suit your fancy, it provides a few good opportunities to passive-aggressively flex your nuts to co-workers. ®

Ted Dziuba is a co-founder at Milo.com You can read his regular Reg column, Fail and You, every other Monday.