The future of Python: Concurrency devoured, Node.js next on menu

'There's twice as much Python as you know'

The thing is, asynchronous code is Node.js's reason for being. Node, a JavaScript runtime environment, was created to allow non-blocking, event-driven programming. Python is moving rather quickly into the same territory, and to do so, the size of the language – in terms of the standard library – has expanded considerably.

"There's twice as much Python as you know," as Hettinger put it.

Simeon Franklin, a Python technical instructor at Twitter, said there's a lot of enthusiasm about new features, but suggested it's hard to keep up. He said he used to know the entire standard library very well. Now, he said, there's a lot of new material.

Event-driven programming was available in Python long before Node existed, through the Twisted framework, created by Glyph Lefkowitz in 2002, and later the Tornado framework. Now, with Python 3 showing signs of strong adoption, it has become a central focus among Python developers.

Event-driven programming relies on an event loop that runs continuously. Presented with asynchronous requests, it can process them without blocking other requests.

Certain applications may be suited for handling requests in a series, but this approach isn't ideal, say, if you're trying to serve a web page – you don't want a site visitor to wait for each image and file to be downloaded, one after the other, if your code and infrastructure can handle simultaneous transmission.

"I think async is the future," said Hettinger. "Threading is so hard to get right."

Lefkowitz, in an email to The Register, said, "Based on my own experience at PyCon, asyncio is really bringing the community together around event-driven concurrency as the main, blessed way to do concurrency at a language level in Python."

Lefkowitz pointed to a blog post he wrote in 2014 and said his predictions have largely come true. "By adopting 'async def' and 'await' into the language, the core team has largely ended the debate about whether it is 'pythonic' or not, and cooperative concurrency has become the dominant coordination paradigm," he said.

The adoption of asyncio has created a library ecosystem boom, Lefkowitz said, but it has not made projects like Twisted or Tornado obsolete. "Relieved of the need to convince people that 'event-driven might be a good idea,' we now have people showing up for the functionality, ready-made ecosystem, and consistent design sensibility that Twisted provides rather than just 'it's the only way to do async,'" he explained.

Lefkowitz believes Twisted helped shape Python community consensus. "There's no one dramatic thing we did," he said. "Asyncio largely adopted Twisted's lowest-level core interfaces (protocols and transports, the reactor) and learned a lot from the higher-level request-response pattern (Deferreds) without adopting it outright. A lot of care was taken in asyncio's design to ensure it could be compatible with existing frameworks and give them a turbo-boost of community interest rather than hitting the reset button."

Over the next decade, Lefkowitz believes the Python community will need to improve packaging and deployment. "JavaScript has a better back-end story than Python has a front-end story right now," he said.

Lefkowitz doesn't see Node's focus on event-driven programming as the primary driver of Python's shift toward the same. "I think that the Python and JavaScript communities definitely learn from each other's' best practices," he said.

"But I also think that advances in Python itself (in the area of generators, coroutines, and evolving best practices around those language features) were more of an impetus to move to an event-driven model than any particular thing happening in JavaScript." ®