Intel builds cloud orchestrator as if it had the chance to start OpenStack from scratch

In 'Ciao' VMs, containers and bare metal live together in perfect harmony

Man with head in the cloud

Interview According to Arjan van de Ven, one of the things that's pretty common at Intel is looking at a technology and asking “what would we change if we were doing it from scratch?” One result of just such a question is Project Ciao, Intel's open source take on what an cloud orchestrator should look like.

Ciao – the Cloud Integrated Advanced Orchestrator – has been quietly unfolding at GitHub for a few months, and Linux engineer van de Ven told The Register, it's an attempt to unify what are usually either-or streams in the world of OpenStack: orchestration managing virtual machines on the one hand, versus orchestration of containers on the other.

“In December last year, we asked 'if we did OpenStack from scratch, what would it look like?”

The biggest requirement to come out of that discussion, he said, was the virtual machines, containers, and – if required – bare metal should all “be treated as equal citizens” by the orchestrator.

The reasoning was simple, he said: nearly any IT shop is going to have a mix of workloads, because nearly nobody has managed to eliminate all their legacy environments.

“If you were building an enterprise from scratch, you might do everything in containers. But a lot of people have legacy infrastructure – the ERP system or the financial system.”

Van de Ven says enterprises will converge to a single model, with everything – including stuff like cloud analytics (think Hadoop or Apache Spark) – on the same infrastructure. However, “as much as you can say containers are the future, hybrid is now.”

To get what they wanted in Ciao meant “replacing big parts of the scheduler”, because looking around existing orchestrator projects, he said, they were strongly one-or-the-other (VMs or containers).

To do that, van de Ven said, the Ciao team “had to define intermediate concepts – at the top level, the workloads – so that almost nothing in the whole stack cares about what's happening somewhere else.

“We had to make an abstraction layer for the workloads that would allow us to reason about them (the workloads), without caring whether it's in a VM or a container.

“If you talk about virtual machines, you're talking about things like memory size or virtual CPUs, which you don't talk about in containers. So we had to abstract those things in a way that makes sense between the two.”

Project Ciao itself is a pretty straightforward architecture, with three primary components:

  • Controller – responsible for tenant workload policy;
  • Scheduler – which implements a push/pull scheduling algorithm managed by the controller. The controller sends an instance to the scheduler, and the scheduler finds the “first fit” among cluster compute nodes requesting work; and
  • Launcher – which abstracts the launch details for the workloads, whether they're containers, VMs or bare metal; and provides per-node stats to the scheduler, and per-instance stats to the scheduler and controller.

There's also a set of networking components that creates a separate Layer 2 network for each tenant; a command line interface; and a Web interface.

Since this is about offering cloud resources to multiple tenants, security's obviously an important part of the design.

Van de Ven told The Register security expectations have changed a lot in the years since OpenStack was first conceived.

“If you started today, there'd be no discussion … everything has to be encrypted over the wire,” he said.

Security and privacy is, after all, the main reason companies give for maintaining private clouds.

So in Ciao, all communications are encrypted, every tenant has its own network by design; and at the administrative level, superusers' virtual network is completely separated from the tenants.

All Ciao communications use the Simple and Secure Node Transfer Protocol (SSNTP), which uses TLS to protect connections between components. ®

Sponsored: Beyond the Data Frontier

SUBSCRIBE TO OUR WEEKLY TECH NEWSLETTER




Biting the hand that feeds IT © 1998–2019