Google's Grumpy code makes Python Go

With a transcompiling tool, YouTube aims to overcome Python's limitations

By Thomas Claburn in San Francisco

Posted in DevOps, 5th January 2017 00:20 GMT

Google on Wednesday introduced an open-source project called Grumpy to translate Python code into Go programs.

The company's front-end server for YouTube and its YouTube API are mainly written in Python, and run on CPython 2.7, as opposed to other implementations (Jython, PyPy, or IronPython).

The issue for Google is performance. Python in its various forms doesn't excel at concurrency – running multiple, independent threads at the same time – explains YouTube engineer Dylan Trotter in a blog post.

Grumpy is a source code transcompiler and runtime. It compiles Python source code into Go source code that gets compiled into native code rather than bytecode that requires a VM.

The compiled Go code makes calls to the Grumpy runtime, which is a Go library that functions similarly to the Python C API.

Trotter characterizes Grumpy as experimental, but that doesn't mean support for Python 3 is forthcoming as the project matures. In response to a request for Python 3 support, Trotter on GitHub said, "We have a large Python 2.7 codebase so that's what we've been focused on. I definitely would like to support Python 3. It's just a bunch of work." He suggested those interested in Python 3 support should fork the project.

One possible rationale for creating Grumpy may be the planned end-of-life date for Python 2.7 in 2020. Rather than trying to update all its Python code from version 2.7 to 3 – which has been slow to catch on – Google appears to be planning to convert at least some of its Python code to Go over the next few years.

Google didn't immediately respond to a request for comment.

In a post to Hacker News, Trotter offered his thoughts on the matter. "The idea is to continue to write code in Python," he said. "The transcompiled code is not suitable for working with directly. That said, there is the possibility of rewriting bits and pieces in Go (eg, performance-critical stuff) and then call into it from Python. Sort of a hybrid approach."

Three years hence and beyond, that hybrid approach can be expected to include a lot more Go code, provided Google's developers become as proficient with Go as they are with Python.

Go, as Trotter suggests, has performance advantages over Python (though benchmarks can vary significantly, depending upon optimizations and alternative implementations). It also has brand advantages: Go was developed at Google and the company's willingness to eat its own dogfood, so to speak, could convince other organizations that Go is the way to go.

Google doesn't benefit directly from Go adoption, but broader usage will deepen the talent pool of Go developers, who could in time end up working at Google or improving Go libraries that Google uses. There's also the opportunity cost imposed on competitors – the more time developers spend with Go, the less they have to spend with languages backed by Apple, Microsoft, Mozilla, Oracle, or others. ®

Sign up to our NewsletterGet IT in your inbox daily

55 Comments

More from The Register

Data scientist wanted: Must have Python, spontaneity not required

Review of job ads pins average salary at £47k

Frenchman comes eye to eye with horror toilet python

'I could very well have been bitten in a sensitive place, if you know what I mean'

Python explosion blamed on pandas

Data science fad just won't die

Everyone loves programming in Python! You disagree? But it's the fastest growing, says Stack Overflow

It's a grower not a, er, yeah...

Q. What's today's top language? A. Python... no, wait, Java... no, C

Just learn them all and stop worrying about the popularity contest

Python 3 in Google cloud

Pretend Python packages prey on poor typing

Typosquatting attack hits the PyPI registry

The future of Python: Concurrency devoured, Node.js next on menu

Analysis Programming language keeps getting fatter amid awkward version 3 split

Worry not, Python devs – you can program a quantum computer

Schroedinger's snake slithers into Bank's quantum sim

Former UK.gov IT man and Python king's guide to neural networks

Interview Tariq Rashid on programming AI