Pivotal takes guts of Google, sells to enterprises
Small team aims to clone Google-stack for enterprises
Pivotal wants to build and sell the systems that run Google, but though its ambition may be bold, its claims are as yet more visions than statements of fact.
At a packed press event in San Francisco on Wednesday Pivotal executives unveiled their plans for the company, which aims to bring the sorts of technologies and techniques pioneered within Google, Amazon, and Facebook, to large enterprises.
"We're building a new platform for a new era, bringing consumer-grade capabilities to the enterprise," Pivotal chief Paul Maritz, said.
Pivotal aims to combine data warehousing and analytics, the Cloud Foundry platform-as-a-service, Spring middleware, a supercharged version of Hadoop (Pivotal HD), messaging, and application frameworks, into a vast platform-as-a-service that can let enterprises not only launch their apps onto standard commodity hardware or clouds, but wire up the apps for live analytics as well.
When put together these technologies allow for applications whose lifecycle is managed within one platform, and whose relationships, generated data, and runtime environments are all dealt with by a fleet of a semi-automatic processes that can be controlled by developers.
It's a bold goal, and one which relies on the organization being able to effectively unify a suite of specialized, technically sophisticated products.
But the payoff is that if it can unify these technologies, then it can offer enterprises the same type of tech available to major consumer giants like Google, Facebook, and Amazon, and the enterprises can run it on their terms in their own data center.
"At the end of the day you can't get the scale, and you can't get to high levels of reliability, if you're not automated," Maritz said.
Though Pivotal was legally formed on April 1st of this year it is no joke, and has received major backing from EMC and VMware, along with a $105 million investment from industrial giant General Electric. The three companies are all betting that large enterprises need the same IT environments as those used within the consumer internet giants, and that they will pay large quantities of money for it.
We understand that to even get a seat at the commercial Cloud Foundry table requires an outlay of $500,000 minimum – that's a lot of cash compared to all the other PaaS's (Heroku, Engine Yard, Appfog) which are priced according to a freemium model.
With a pricing plan like that for just one component of Pivotal, we're sure that the eventual product will be, as they say, reassuringly expensive.
"This is not just a cloud company, it's not just a big data platform company, if we're to provide an enterprise PaaS platform it has to serve a modern application developer," Scott Yara, Pivotal's senior veep of platform and products (essentially, the Organizer in Chief), said.
The tech is cloud-agnostic, and is being designed to work with large pools of commodity hardware for on-premise deployments, or major public clouds such as Amazon Web Services, or systems based on OpenStack, or even VMware vCloud.
"The idea is as a developer you would be able to create a Pivotal account that would instantly give you access to a language and framework runtime hopefully of your choice - whether that's Spring or Java which is the predominant language of the enterprise today - or a more recent entrant like Ruby or node.js or python," Yara said. Developers can then "build applications locally and deploy them to the Pivotal platform. It should be as simple and easy as doing that on a public cloud service today."
These applications will interface with legacy tech through the tooling within Spring, Yara said, and noted that Pivotal ultimately wants to be able to let developers automatically stream log data from applications into an analytics stack for real-time performance analysis.
"The convergence of your globally distributed filesystem with higher level database analytics - that's where it's going," Yara said. "You have one infrastructure that's deployed across tens of thousands of machines. Whether you need a simple object service or transactional database it should be the same."
The company acknowledged that it will take a long time to bring all the disparate services together. It hopes to bring the first version of its platform Pivotal One to general availability by the fourth quarter of this year.
Laboring in the shadow of Google
Google cast a long shadow over the entire Pivotal event.
Much of the technologies Pivotal is either tying together, or planning on creating, are in heavy use within the candy-colored walls of the Mountain View chocolate factory.
The use of one file system – HDFS – is born of the same desires that have seen Google consolidate around the Google File System. Along with this, the integration of analytics with the filesystem via Pivotal HD / HAWQ, bears similarities to Google's recent moves to closely couple its databases with a single file system, such as Spanner and F1.
Google has been building and honing these systems for years and is only now exposing them through select services like Google App Engine, BigQuery, and Compute Engine. The exemplary performance of these technologies is indicative of how advanced trhe underlying platforms are.
So how can Pivotal manage to achieve Google-like performance with only 1,250 staff (700 engineers) at launch?
"The teams and technologies coming into [Pivotal] are not starting from ground zero," Yara told us, pointing out that Pivotal components such as Cloud Foundry, Spring and Pivotal HD (Hadoop) have all been in development for years. However, he did acknowledge that "integration is the challenge".
"Google has the apps and the foundation, we're just trying to do the foundation," Pivotal chief Paul Maritz told us as he walked out of the event to catch a plane whose engines had the potential to generate 30TB of data in a single flight - data which, one day, Maritz hopes, will stream straight into a data center stuffed with hardware running Pivotal tech.