Pivotal takes guts of Google, sells to enterprises

Small team aims to clone Google-stack for enterprises

Securing Web Applications Made Simple and Scalable

Pivotal wants to build and sell the systems that run Google, but though its ambition may be bold, its claims are as yet more visions than statements of fact.

At a packed press event in San Francisco on Wednesday Pivotal executives unveiled their plans for the company, which aims to bring the sorts of technologies and techniques pioneered within Google, Amazon, and Facebook, to large enterprises.

"We're building a new platform for a new era, bringing consumer-grade capabilities to the enterprise," Pivotal chief Paul Maritz, said.

Pivotal aims to combine data warehousing and analytics, the Cloud Foundry platform-as-a-service, Spring middleware, a supercharged version of Hadoop (Pivotal HD), messaging, and application frameworks, into a vast platform-as-a-service that can let enterprises not only launch their apps onto standard commodity hardware or clouds, but wire up the apps for live analytics as well.

When put together these technologies allow for applications whose lifecycle is managed within one platform, and whose relationships, generated data, and runtime environments are all dealt with by a fleet of a semi-automatic processes that can be controlled by developers.

It's a bold goal, and one which relies on the organization being able to effectively unify a suite of specialized, technically sophisticated products.

But the payoff is that if it can unify these technologies, then it can offer enterprises the same type of tech available to major consumer giants like Google, Facebook, and Amazon, and the enterprises can run it on their terms in their own data center.

"At the end of the day you can't get the scale, and you can't get to high levels of reliability, if you're not automated," Maritz said.

Though Pivotal was legally formed on April 1st of this year it is no joke, and has received major backing from EMC and VMware, along with a $105 million investment from industrial giant General Electric. The three companies are all betting that large enterprises need the same IT environments as those used within the consumer internet giants, and that they will pay large quantities of money for it.

We understand that to even get a seat at the commercial Cloud Foundry table requires an outlay of $500,000 minimum – that's a lot of cash compared to all the other PaaS's (Heroku, Engine Yard, Appfog) which are priced according to a freemium model.

With a pricing plan like that for just one component of Pivotal, we're sure that the eventual product will be, as they say, reassuringly expensive.

"This is not just a cloud company, it's not just a big data platform company, if we're to provide an enterprise PaaS platform it has to serve a modern application developer," Scott Yara, Pivotal's senior veep of platform and products (essentially, the Organizer in Chief), said.

The tech is cloud-agnostic, and is being designed to work with large pools of commodity hardware for on-premise deployments, or major public clouds such as Amazon Web Services, or systems based on OpenStack, or even VMware vCloud.

"The idea is as a developer you would be able to create a Pivotal account that would instantly give you access to a language and framework runtime hopefully of your choice - whether that's Spring or Java which is the predominant language of the enterprise today - or a more recent entrant like Ruby or node.js or python," Yara said. Developers can then "build applications locally and deploy them to the Pivotal platform. It should be as simple and easy as doing that on a public cloud service today."

These applications will interface with legacy tech through the tooling within Spring, Yara said, and noted that Pivotal ultimately wants to be able to let developers automatically stream log data from applications into an analytics stack for real-time performance analysis.

"The convergence of your globally distributed filesystem with higher level database analytics - that's where it's going," Yara said. "You have one infrastructure that's deployed across tens of thousands of machines. Whether you need a simple object service or transactional database it should be the same."

The company acknowledged that it will take a long time to bring all the disparate services together. It hopes to bring the first version of its platform Pivotal One to general availability by the fourth quarter of this year.

Laboring in the shadow of Google

Google cast a long shadow over the entire Pivotal event.

Much of the technologies Pivotal is either tying together, or planning on creating, are in heavy use within the candy-colored walls of the Mountain View chocolate factory.

The use of one file system – HDFS – is born of the same desires that have seen Google consolidate around the Google File System. Along with this, the integration of analytics with the filesystem via Pivotal HD / HAWQ, bears similarities to Google's recent moves to closely couple its databases with a single file system, such as Spanner and F1.

Google has been building and honing these systems for years and is only now exposing them through select services like Google App Engine, BigQuery, and Compute Engine. The exemplary performance of these technologies is indicative of how advanced trhe underlying platforms are.

So how can Pivotal manage to achieve Google-like performance with only 1,250 staff (700 engineers) at launch?

"The teams and technologies coming into [Pivotal] are not starting from ground zero," Yara told us, pointing out that Pivotal components such as Cloud Foundry, Spring and Pivotal HD (Hadoop) have all been in development for years. However, he did acknowledge that "integration is the challenge".

"Google has the apps and the foundation, we're just trying to do the foundation," Pivotal chief Paul Maritz told us as he walked out of the event to catch a plane whose engines had the potential to generate 30TB of data in a single flight - data which, one day, Maritz hopes, will stream straight into a data center stuffed with hardware running Pivotal tech.

The Essential Guide to IT Transformation

More from The Register

next story
Manic malware Mayhem spreads through Linux, FreeBSD web servers
And how Google could cripple infection rate in a second
EU's top data cops to meet Google, Microsoft et al over 'right to be forgotten'
Plan to hammer out 'coherent' guidelines. Good luck chaps!
US judge: YES, cops or feds so can slurp an ENTIRE Gmail account
Crooks don't have folders labelled 'drug records', opines NY beak
FLAPE – the next BIG THING in storage
Find cold data with flash, transmit it from tape
Seagate chances ARM with NAS boxes for the SOHO crowd
There's an Atom-powered offering, too
Gartner: To the right, to the right – biz sync firms who've won in a box to the right...
Magic quadrant: Top marks for, er, completeness of vision, EMC
prev story


Top three mobile application threats
Prevent sensitive data leakage over insecure channels or stolen mobile devices.
The Essential Guide to IT Transformation
ServiceNow discusses three IT transformations that can help CIO's automate IT services to transform IT and the enterprise.
Mobile application security vulnerability report
The alarming realities regarding the sheer number of applications vulnerable to attack, and the most common and easily addressable vulnerability errors.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Consolidation: the foundation for IT and business transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.