DevOps

Google's Grumpy code makes Python Go

With a transcompiling tool, YouTube aims to overcome Python's limitations

By Thomas Claburn in San Francisco

55 SHARE

Google on Wednesday introduced an open-source project called Grumpy to translate Python code into Go programs.

The company's front-end server for YouTube and its YouTube API are mainly written in Python, and run on CPython 2.7, as opposed to other implementations (Jython, PyPy, or IronPython).

The issue for Google is performance. Python in its various forms doesn't excel at concurrency – running multiple, independent threads at the same time – explains YouTube engineer Dylan Trotter in a blog post.

Grumpy is a source code transcompiler and runtime. It compiles Python source code into Go source code that gets compiled into native code rather than bytecode that requires a VM.

The compiled Go code makes calls to the Grumpy runtime, which is a Go library that functions similarly to the Python C API.

Trotter characterizes Grumpy as experimental, but that doesn't mean support for Python 3 is forthcoming as the project matures. In response to a request for Python 3 support, Trotter on GitHub said, "We have a large Python 2.7 codebase so that's what we've been focused on. I definitely would like to support Python 3. It's just a bunch of work." He suggested those interested in Python 3 support should fork the project.

One possible rationale for creating Grumpy may be the planned end-of-life date for Python 2.7 in 2020. Rather than trying to update all its Python code from version 2.7 to 3 – which has been slow to catch on – Google appears to be planning to convert at least some of its Python code to Go over the next few years.

Google didn't immediately respond to a request for comment.

In a post to Hacker News, Trotter offered his thoughts on the matter. "The idea is to continue to write code in Python," he said. "The transcompiled code is not suitable for working with directly. That said, there is the possibility of rewriting bits and pieces in Go (eg, performance-critical stuff) and then call into it from Python. Sort of a hybrid approach."

Three years hence and beyond, that hybrid approach can be expected to include a lot more Go code, provided Google's developers become as proficient with Go as they are with Python.

Go, as Trotter suggests, has performance advantages over Python (though benchmarks can vary significantly, depending upon optimizations and alternative implementations). It also has brand advantages: Go was developed at Google and the company's willingness to eat its own dogfood, so to speak, could convince other organizations that Go is the way to go.

Google doesn't benefit directly from Go adoption, but broader usage will deepen the talent pool of Go developers, who could in time end up working at Google or improving Go libraries that Google uses. There's also the opportunity cost imposed on competitors – the more time developers spend with Go, the less they have to spend with languages backed by Apple, Microsoft, Mozilla, Oracle, or others. ®

Sign up to our NewsletterGet IT in your inbox daily

55 Comments

More from The Register

New Python update slithers into release

Behold, the new, faster version 3.7, with nanosecond timing, data classes and docs in more (human) languages

New Monty Python movie to turn old jokes into new royalties

You silly English k-niggits will probably flock to see Spamalot the musical movie

Python creator Guido van Rossum sys.exit()s as language overlord

‘Benevolent dictator for life’ tired of the hate, leaves behind no successor or governance

Frenchman comes eye to eye with horror toilet python

'I could very well have been bitten in a sensitive place, if you know what I mean'

Data scientist wanted: Must have Python, spontaneity not required

Review of job ads pins average salary at £47k

Python explosion blamed on pandas

Data science fad just won't die

Everyone loves programming in Python! You disagree? But it's the fastest growing, says Stack Overflow

It's a grower not a, er, yeah...

Python 3 in Google cloud

Q. What's today's top language? A. Python... no, wait, Java... no, C

Just learn them all and stop worrying about the popularity contest

Pretend Python packages prey on poor typing

Typosquatting attack hits the PyPI registry