DevOps

Google's Grumpy code makes Python Go

With a transcompiling tool, YouTube aims to overcome Python's limitations

By Thomas Claburn in San Francisco

55 SHARE

Google on Wednesday introduced an open-source project called Grumpy to translate Python code into Go programs.

The company's front-end server for YouTube and its YouTube API are mainly written in Python, and run on CPython 2.7, as opposed to other implementations (Jython, PyPy, or IronPython).

The issue for Google is performance. Python in its various forms doesn't excel at concurrency – running multiple, independent threads at the same time – explains YouTube engineer Dylan Trotter in a blog post.

Grumpy is a source code transcompiler and runtime. It compiles Python source code into Go source code that gets compiled into native code rather than bytecode that requires a VM.

The compiled Go code makes calls to the Grumpy runtime, which is a Go library that functions similarly to the Python C API.

Trotter characterizes Grumpy as experimental, but that doesn't mean support for Python 3 is forthcoming as the project matures. In response to a request for Python 3 support, Trotter on GitHub said, "We have a large Python 2.7 codebase so that's what we've been focused on. I definitely would like to support Python 3. It's just a bunch of work." He suggested those interested in Python 3 support should fork the project.

One possible rationale for creating Grumpy may be the planned end-of-life date for Python 2.7 in 2020. Rather than trying to update all its Python code from version 2.7 to 3 – which has been slow to catch on – Google appears to be planning to convert at least some of its Python code to Go over the next few years.

Google didn't immediately respond to a request for comment.

In a post to Hacker News, Trotter offered his thoughts on the matter. "The idea is to continue to write code in Python," he said. "The transcompiled code is not suitable for working with directly. That said, there is the possibility of rewriting bits and pieces in Go (eg, performance-critical stuff) and then call into it from Python. Sort of a hybrid approach."

Three years hence and beyond, that hybrid approach can be expected to include a lot more Go code, provided Google's developers become as proficient with Go as they are with Python.

Go, as Trotter suggests, has performance advantages over Python (though benchmarks can vary significantly, depending upon optimizations and alternative implementations). It also has brand advantages: Go was developed at Google and the company's willingness to eat its own dogfood, so to speak, could convince other organizations that Go is the way to go.

Google doesn't benefit directly from Go adoption, but broader usage will deepen the talent pool of Go developers, who could in time end up working at Google or improving Go libraries that Google uses. There's also the opportunity cost imposed on competitors – the more time developers spend with Go, the less they have to spend with languages backed by Apple, Microsoft, Mozilla, Oracle, or others. ®

Sign up to our NewsletterGet IT in your inbox daily

55 Comments

More from The Register

Mega-bites of code: Python snakes into 1st place for cyber-attacks

Hackers share general public's love of popular programming language

A spot of Python in your Azure automation? Step right this way, sir

Python 2 support for runbooks slithers out of preview

Using Python in Visual Studio Code? Microsoft has new toys for you

You will use the new debugger and you will like it, OK?

New Python update slithers into release

Behold, the new, faster version 3.7, with nanosecond timing, data classes and docs in more (human) languages

Microsoft Visual Studio Code replumbed for better Python taming

Python Language Server an option for those that code

Pleasant programming playground paves popular Python path

Shrew'd thinking: Code Shrew helps peeps who want to, or need to, gobble a slice of Py

New Monty Python movie to turn old jokes into new royalties

You silly English k-niggits will probably flock to see Spamalot the musical movie

Hooray: Google App Engine finally ready for Python 3 (and PHP 7.2)

'OG of serverless' gets modern makeover

Python joins movement to dump 'offensive' master, slave terms

Programming language bites its tongue to be more inclusive

Python wriggles onward without its head

Analysis The software's just fine, annual codefest agrees