Nutanix server-storage half-bloods armed with Xeon E5s
Adds KVM alongside ESXi for virty server slicing
Nutanix, which uncloaked last year peddling a virtualisation-driven all-in-one compute-and-storage appliance, is updating its server iron to keep pace with the rest of the tier-one players while also tweaking its underlying systems software.
And the startup hopes to tweak its NX series of converged appliances to cope with a wider variety of workloads.
The company was founded in 2009 and uncloaked from stealth mode back in August 2011. Marketing veep Howard Ting, who was just brought in from hot startup Palo Alto Networks, says business is booming for the Nutanix NX gear.
Nutanix just finished its fiscal fourth quarter, and Ting tells El Reg he reckons the small biz displaced somewhere around $25m in server and SAN storage sales with its appliance clusters. (That is not the revenue that Nutanix generated, which is a guarded secret.)
Ting says that the company is nearly doubling sales quarter-on-quarter sequentially. It is hard to call this early, but Ting says the first quarter of this fiscal year could end up generating somewhere between four and six times the revenues of the company's first full quarter of sales last year. This is the kind of sales rate you expect from a fast-growing startup with a slightly different architecture attacking industry incumbents.
The Nutanix Operating System that underpins the company's hybrid server-storage appliances was not slapped together using open source code by a bunch of amateurs. Dheeraj Pandey, cofounder and CEO, crafted the early Exadata clusters at Oracle and then went to work at Aster Data Systems, the columnar clustered database company that was eaten by Teradata nearly two years ago. Cofounder Mohit Aron was chief architect at Aster Data and before that was the lead designer of the Google File System at the search engine giant that has been mimicked in the Hadoop big data muncher. Ajeet Singh is the third cofounder, who was director of product management at Aster Data. (Singh is no longer with Nutanix, having left in January of this year).
Nutanix raised seed funding from private investors in May 2010 to get rolling, and picked up its Series A injection in April 2011 from Lightspeed Venture Partners and Blumberg Capital.
Nutanix NX-3000 server-storage node
In October 2011, Nutanix raised $25m in its Series B round, led by Khosla Ventures. In August this year, Nutanix raised another $33m in its third round, with everybody kicking in again and Goldman Sachs and Battery Ventures joining in. When that third round came in, the company had 150 clusters in total shipped, with more than 3.3PB of combined disk and flash storage; it had more than 100 employees.
The Nutanix NX appliances are based on tray server designs, and the company is agnostic about what hardware you want. The original NX-2000 was based on quad-node, 2U machines from Super Micro, with a plan to support Dell or Hewlett-Packard equivalents so long as they have native 10 gigabit Ethernet on the system boards. That 10GE networking is important because it underpins the Nutanix Complete Cluster and its Nutanix Operating System. With the NX-3000s, Nutabix has shifted to a quad-node box manufactured by Quanta.
The Nutanix SOCS storage system
The most important part of that NOS software is the Scale-Out Converged Storage – SOCS for short – virtual disk controller. On each server node in the Nutanix cluster, about a third of the cores run this SOCS controller, which pools the Seagate SATA disk drive, Intel SATA SSD flash drive, and Fusion-io PCI-based flash cards into a single pool of storage and exposes it to virtual machine hypervisors as a single pool of block and file storage. In effect, SOCS makes the server cluster itself into a SAN. So compute and storage are on the same cluster. One of the tasks of the NOS software is to try to get compute jobs located as close to the storage they need in the cluster.
With the first-generation NX-2000 series appliances, the Intel 2.5-inch 300GB flash drives are used for local OS booting for each node in the Nutanix appliance, while the 400GB Fusion-io flash cards were used for storing the hottest data in the node. (The Fusion-io flash storage was formatted down to 256GB by the SOCS file system.) The cooler data in each node is pushed out to the five 2.5-inch 1TB SATA drives that spin at a measly 7,200 RPM. The NOS software has automatic data tiering, so you don't have to do anything to get the right data in the right place. Data is spread across the nodes through the 10GE links in the nodes, and virtual machines running on the appliances are accessed through two Gigabit Ethernet ports in each node. The NX-2000 came with 48GB of main memory, expandable to 192GB. The machine came with redundant 1,600-watt power supplies supporting either 110 or 208 volt juice.
Nutanix NX-2000 versus NX-3000 appliances
With the NX-3000 series appliances that start shipping today, Nutanix is moving to the latest "Sandy Bridge-EP" Xeon E5 processors from Intel, specifically the eight-core E5-2660 processors running at 2.2GHz. This is the fastest Xeon E5 with all eight cores fired up that fits in the 95-watt thermal envelope, but the clock speed is a bit lower than the six-core Xeon 5650s uses in the NX-2000 series, which ran at 2.66GHz. If your workloads are clock-speed sensitive, you might be able to talk Nutanix into supporting the 115-watt Xeon E5-2670, which spins at 2.6GHz with eight cores. But the extra 160 watts of peak power could cause problems in a dense tray server.
The NX-3000 machines have a funky new bezel, which doesn't do jack for processing but which does look more stylin' than the plain boxes NX-2000s. This time around, the NX-3000 tucks a 4GB enterprise USB stick containing the OS boot image into the server and puts that 300GB Intel SSD to work doing real stuff. The solid state card (which is not made by Fusion-io, but Nutanix did not specify which one it used) is formatted to make use of its 400GB capacity by the SOCS software, and the server nodes have two 10GE ports for SOCS to have redundant links between nodes for the storage cluster for multipathing, redundancy, and bandwidth. The NX-3000 comes with 128GB of memory as standard, expandable to 256GB per node, and has two 1,400-watt power supplies and needs 208 volts of juice; it has two gigabit Ethernet ports for the hypervisor and its VMs to talk to each other and the outside world.
VDI scalability on Nutanix server-storage appliances is tows the line
Generally speaking, the NX-3000 supports roughly 400 virtual machines per chassis (which Nutanix calls a block and which has four server nodes), compared to 300 VMs for the NX-2000 machines. Obviously, the number of VMs supported depends on what they are doing. The spec sheet for the NX-2000 series says it can support between 200 and 300 virtual desktops running VDI software and somewhere between 80 and 120 "server workloads". Presumably these numbers go up by about 33 percent for the NX-3000 series.
Looking ahead, Ting said that Nutanix was exploring options to increase the number of disk drives attached to server nodes to better chase big data and other analytics workloads. This could happen by sliding storage trays into the existing enclosures or building a different form factor. Nutanix has not made any commitments yet as to how it will do it. But you can expect an announcement sometime in the spring, says Ting.
Compression added to NOS
The Nutanix Operating System gets some important tweaks that go along with the new hardware and still run on the old hardware, too. (That is one of the benefits of implementing a storage controller in software that runs on x86 iron instead of tying it to a specific hardware-based controller.)
The first big change is that the KVM hypervisor championed by Red Hat, Canonical, and others, is now supported on the NX series appliances. The machines originally shipped with support for VMware's ESXi 5.0 hypervisor last year, and have been updated to run the new ESXi 5.1 hypervisor as well. The expectation last year was that Hyper-V would come out ahead of KVM, but Nutanix is implementing support in the order that customers request and when a sufficient number of them ask for it. Like most of us, Nutanix is not sure what to do with the well-respected but increasingly marginalized Xen hypervisor and its XenServer commercial derivative from Citrix Systems.
The machines ship with KVM or a trial version of ESXi, but Nutanix expects customers to buy commercially supported licences to get tech support from Red Hat or VMware. (It is not required, of course.) You can layer VMware's vCloud Director cloud control freak atop of ESXi and the OpenStack control freak atop KVM to orchestrate the virtual machines atop the Nutanix appliances.
The NOS 3.0 software also includes data compression that can crunch data inline, when data is being written to either the flash or disk storage in the clustered storage array, or in a post-processing mode, when data has become cold and is going to be moved out to the SATA disks by the hierarchical storage manager inside the SOCS controller.
You can set up the inline or post-processed data compression based on policies and on each individual VM running on the Nutanix clusters. Like the disk controller itself, this data compression software runs on the CPUs in the machines and does not require any supplemental hardware. Nutanix has opted to use Google's Snappy compression algorithm.
The updated code for the Nutanix appliances also has VM-level disaster recovery (DR) replication, which Ting says is better than the traditional DR programs used with systems, which replicate data at the LUN or file system level. The DR algorithm that Nutanix has created does active-passive clustering, and the DR can run on a VM of any size and is not restricted in terms of core count for the VM. You can also do one-to-many replication if you are really paranoid.
NOS 3.0 will also make use of the Avahi network discovery software to discover new Nutanix appliances as they are added to the cluster. Avahi is an open-source clone of Apple's Bonjour network discovery protocol, and you can add a new machine to a cluster in two mouse clicks and without having to take the cluster offline. When you fire up the cluster, workloads can be assigned to it, and if data needs to move across nodes, it gets done automagically by the SOCS storage controller. Finally, the NOS 3.0 software allows for rolling upgrades of NOS software in the Nutanix appliances without having to take the cluster offline.
The Complete Cluster NX-3000 appliances are available now. A four-node NX-3400 loaded up costs $144,000, which is 20 per cent more expensive than the NX-2400 it replaces. That seems a fair price, given that the newer machine has about 33 per cent more oomph. ®
Re: Really? Wow!.... so what?
let me help you ..
they don't claim there is anything special about the hardware - just the opposite, that it's standard commodity x86 gear... however....
assuming you are talking about their claim of 400 VM's per 2U cluster ( which contains 4 nodes )
4 nodes, each with 16 cores is 64 cores in total plus hyperthreading is 128 threads - so 400/128 = 3.125 - slightly heavy maybe but for knowledge worker desktops not unreasonable
256GB of memory in each node - which is 1TB of memory - divide this by 400 and you get 2.5GB of memory per VM - seems ok to me
Storage again 4 nodes each of 5TB raw gives 20TB RAW - lets assume mirroring so 10TB protected storage - divide that by 400 ( assuming there is no de-dupe of templates etc .. which there is ) would be 25GB per VM - again not unreasonable. With de-dupe of the base images this number would rise dramatically.
The clever part being they take jbod in each node - and form a fully protected clustered filesystem across all nodes - with data replicated across the cluster for protection
The innovation is a cluster that scales linearly in small increments, that provides a single global namespace that looks just like a SAN ( data protection/snapshots/replication/compression all built in ) from local storage - whilst providing the performance of flash storage through the use of automatically tiered data.
If you know a junior sys admin who could build this - he/she shouldn't be a junior sys admin, but running their own company ....
Really? Wow!.... so what?
it's an x86 server. I fail to see any innovation worth mentioning. They package a hypervisor with some flash and claim massive over-committed VMs, which I'm calling Raspberries on.
With hyper-threading turned on that's 32 HW threads - or simple arithmetic shows 1 VM: 0.08 HW threads (or 0.04 without).
Memory: (256GB - ~24 Hypervisor overhead) / 400 = 580MB avg.. Tell me what modern OS or app can run in that?
Storage? Presuming 100% raw space available (5.7TB) which would be absolute lunacy to run JBOD, that's ~14GB per VM on average.
So what's the innovation here, and why should we care? You could get a jr. admin to put this together.