Original URL: http://www.theregister.co.uk/2013/08/26/vmware_esxi_vsphere_nsx_vsan_sddc/
VMware goes after biz critical apps with vSphere 5.5
Virtual SAN and networks complete grand unified theory of virtualisation - eventually
Posted in Virtualization, 26th August 2013 16:02 GMT
VMworld 2013 Server virtualization juggernaut VMware has been talking about the software-defined data center (SSDC) for so long that you have to stop and remember that all of the software that enables this strategy is not actually available. But at VMworld 2013 this week in San Francisco, Virtzilla will be rolling out some major pieces of its SDDC stack and putting other pieces into beta and positioning itself as best as can be hoped to capitalize on its vast ESXi hypervisor installed base and defend against the encroachment of Microsoft's System Center/Hyper-V combo and the rapidly rising OpenStack/KVM alternative.
Ahead of the keynotes rolling out the updated ESXi hypervisor, the vSphere editions based on it, the vCloud management tools that turn virtual servers into private clouds, and the new NSX virtual networking and vSAN virtual storage components, VMware graciously gave El Reg a prebriefing on the major new features in these updated and new projects. Team Reg has lots of feet on the street at VMworld and will be digging into the details for each product for a more thorough analysis.
The time is right for virtualizing storage and networks, says John Gilmartin, who is vice president of cloud infrastructure products at VMware. He cites statistics from VMware's own ESXi/vSphere customer base that shows that 77 per cent of customers polled by Virtzilla want to expand virtualization from compute out to networks and storage.
We surmise that many customers would have liked to have been able to virtualize storage and networks years ago as VMware's server virtualization products had matured. And, in fact, many will be impatient to wait longer - as they must - for some of the network and storage virtualization products being announced today will not ship for a while yet.
But VMware has to walk a fine line, trying to not alienate storage and network hardware providers (including parent company EMC) and countering various open source alternatives to virtualizing all components of the data center and orchestrating the movement of compute, data, and connectivity between the two and end users. This SDDC concept is a massive software development effort, and even with acquisitions like Nicira helping to speed up the effort (in this case for network virtualization), there is still a lot of code that has to be developed and thoroughly tested so it is enterprise-ready. This takes time. And money. And the good news is that VMware's competition has to marshal huge resources to take on a similar SDDC effort, even if they are not trying to do all parts of the stack alone. This doesn't make VMware late to the party by any stretch, but the heat is on and it will never relent. There is little margin for error, and everyone in IT can't help but root for the underdog, which VMware most certainly is not when it comes to virtualized servers.
The ESXi foundation updated to release 5.5
With VMware, everything starts and ends with the server virtualization hypervisor, and that is never going to change. For whatever reason, VMware is jumping from 5.1 this time last year to 5.5 now with the ESXi hypervisor and the vSphere editions that progressively turn on more features in that hypervisor as you pay more money. The hypervisor has not had a major overhaul, so a 5.5 designation makes sense. It is able to support the impending "Ivy Bridge-EP" Xeon E5 v2 processors from Intel as well as the existing Opteron 3300, 4300, and 6300 processors that came out from Advanced Micro Devices last fall.
With the 5.5 release, VMware is doubling up the capability of the hypervisor, but only tweaking some aspects of the virtual machines that ride on top of the hypervisor. It is unclear why VMware doesn't re-architect ESXi so a virtual machine can scale from tiny slice of a processor out to encompass an entire host if a customer wants to do that - this is how IBM architects its PowerVM hypervisor for its Power Systems servers as well as z/VM for its System x mainframes - but ESXi has always had separate maximum capacities for the hypervisor and the VMs that frolic upon it. (It no doubt has to do with the architectural decisions in the guts of the hypervisor, which are tough to change.) The important part for VMware is that Microsoft's Hyper-V, Citrix Systems XenServer, and Red Hat's KVM have similar architectures, so x86 server customers are used to things being this way.
With ESXi 5.5, the hypervisor can scale across a maximum of 320 logical cores and address up to 4TB of main memory on a host server; it can have up to 4,096 virtual CPUs (or vCPUs) carved up on that host. ESXi's previous incarnation could ride 160 logical cores and 2TB of memory, and have as many as 2,048 vCPUs on that host.
At the moment, machines using Opteron servers top out at 64 cores and do not have multiple threads per core, so this is plenty of room. A two-socket Ivy Bridge Xeon E5-2600 server is expected to have 24 cores (a dozen per socket), and with HyperThreading turned on that doubles up to 48 logical cores in a single system image. If Intel gets an E5-4600 server into the field sometime soon for four-socket servers, that would double up again to 96 cores and 192 threads for ESXi to stretch across. With HyperThreading turned on that would have been beyond the 160 logical core maximum supported by last year's ESXi 5.1 release. And for customers using Xeon E7 processors, the update allows them to scale beyond the current iron and, presumably, has enough scalability to allow ESXi 5.5 to span "Ivy Bridge-EX" processors that are expected to start shipping for revenue before the end of the year and appear in products early next year. The current "Westmere-EX" Xeon E7 processors have ten cores and twenty threads per socket, so ESXi 5.1 could span an eight-socket machine with HyperThreading turned on, but that was it. Intel will no doubt be adding more cores with the Ivy Bridge versions of the Xeon E7 - rumors have the core counts at anywhere from 12 to 16 cores per socket - so ESXi has to be able to span those cores and threads. If the future Xeon E7s have sixteen cores and 32 threads per socket, that's only 256 logical cores, and ESXi 5.5 can more than handle that. This is no coincidence, of course.
If ESXi 5.5 is constrained anywhere, it could turn out to be in memory addressing. ESXi 5.1 could only span 2TB of main memory, which doesn't seem like a lot these days, and it doesn't look like it will be enough for the fattest Ivy Bridge Xeon E7 v2 configurations, which Intel has said will top out at 12TB, three times the current top-end Xeon E7 v1 chips in an eight-socket setup. Presumably VMware will be able to increase physical main memory addressing to match the iron in quick order. This will be important to give customers some ceiling room, even if they don't use it. The mantra is for vSphere to virtualize more business critical applications, or BCAs in the VMware lingo, and that should mean always matching the capabilities of the iron.
The virtual machines that run atop the ESXi 5.5 hypervisor did get one big capacity change with the update, and that is for the virtual disk locally attached to a VM to be extended from 2TB with ESXi 2.1 to 64TB for ESXi 5.5.
"This is what customers talk about the most," says Gilmartin.
The amount of virtual memory that a VM can address with ESXi 5.5 stays the same at 1TB and the number of vCPUs it can span stays at 64; other aspects of the VMs also remain for the most part the same, Gilmartin says. (The capacity maximum documents for vSphere 5.5 are not yet available as we go to press, but we will bootnote a link to them as soon as they are.)
In addition to the scalability updates above for the hypervisor and VMs, vSphere 5.5 has some other goodies tossed in.
The most interesting one is called vSphere Flash Read Cache, and as the name suggests, it is a way of using local solid state drives running off disk controllers or PCI-Express flash cards as a read cache for the hypervisor and its VMs. This is a read-only virtual cache to speed up access to hot data and does writes through to disk drives that store this data persistently. This read cache feature for vSphere allows you to size and allocate a slice of virtual flash to each VM individually (or not, as you see fit), and if you do a VMotion from on physical server host to another one, provided that both servers have flash storage, those caches will teleport along with the VM. (It is not clear what happens if you VMotion from a server with cache to one without it, but El Reg did ask. We assume that the VMotion will work and the VM will continue running, albeit with slower reads coming off disks instead of out of flash memory.) The performance benefit of this read cache feature is obviously dependent on how read-intensive workloads running on top of vSphere are. The performance gains are akin to the delta in I/O operations per second between disk drives and flash in the system, says Gilmartin, which makes perfect sense. Your mileage will vary, and presumably VMware will soon be showing off some benchmarks.
vSphere 5.5 also includes a tweak to the vSphere HA high availability cluster tool, called vSphere App HA, that can reach into VMs and see when an app has failed. Sometimes, apps fail and need ro be remotely recovered even if the underlying operating systems are running just fine. The HA tool needs to know how to distinguish between the two, and hence the new feature. The vSphere HA feature is still, as far as we know, limited to VMs that span only a single core.
Finally, vSphere 5.5 now includes the "Project Serengeti" tools for implementing Hadoop atop virtual servers, which is called the vSphere Big Data Extensions. El Reg has covered these extensions when they went into beta in June. The only new thing here is that they have been rolled into the vSphere Enterprise and Enterprise Plus editions and therefore formally have technical support from Virtzilla for the first time.
Virtual networks and virtual storage
After inhaling virtual networker Nicira for $1.26bn in July 2012, VMware has been hard at work integrating the virtual networking and security features of the ESXi hypervisor that were subsequently busted out and included in the vCloud Suite cloud orchestration product, with the virtual switching and network controller it got from Nicira. These hypervisor features, called vCloud Networking and Security, or vCNS, hook into the Nicira products: the Open vSwitch virtual switch (which tucks inside the ESXi hypervisor) and the NVP OpenFlow controller, which sits outside of switches and routers and controls the forwarding and routing planes in those physical devices. Mash all this functionality up and you get NSX.
Block diagram of the NSX virtual networking stack from VMware
With NSX, VMware is maintaining the open stance of Nicira even if a lot of the code behind it remains closed. (Exactly what bits are open and closed was not available at press time.) The idea behind NSX, says Gilmartin, is to virtualize the network from Layer 2 all the way up to Layer 7, and to do so atop any existing Ethernet network infrastructure in the data center and without disrupting that network.
"We run Layer 2 through 7 services in software, and now you can deploy network services in seconds instead of days or weeks," he says.
This is the same exact playbook for server virtualization a decade ago, of course. VMware is committed that NSX will hook into any server virtualization hypervisor, not just its own ESXi, will run on any network hardware, will interface with any cloud management tool, and will work in conjunction with applications without modifications. And, as El Reg pointed out back in March when VMware previewed its plans for NSX, there is another parallel with ESXi. As network virtualization controllers are commoditized and possibly open sourced, VMware will eventually be able to give it away if necessary to compete but still charge a licensing fee for the management console - in this case, NSX Manager - to unlock, in golden screwdriver fashion, the features of the network controller.
The materials that VMware has put together ahead of the NSX launch are a bit weak on details, and our coverage from March, while having its share of guesses, has more information than the backgrounder that VMware has put together. A recap is certainly in order, with the addition of whatever other information we can get our hands on thus far ahead of the launch.
NSX rides on top of a hypervisor itself and integrates with other hypervisors in the data center to provide logical L2 switches and logical L3 routers, whose forwarding and routing tables are centralized in the NSX controller and which can be changed programatically as network conditions and applications dictate rather than manually as is required today.
The core NSX controller provides integration with other cloud management tools - VMware named OpenStack and its own vCloud Director by name in March and has been less specific in its backgrounder - is accomplished through a set of RESTful APIs. The add-ons to the core NSX include logical firewalls, load balancers, and virtual private network servers. And VMware expects a partner ecosystem to evolve to plug alternative network applications into the NSX framework, thus giving customers choice to use the logical switches, routers, load balancers, firewalls, and VPN servers.
The NSX stack is expected to be available in the fourth quarter, and thus VMware is not yet talking about the packaging and pricing for it. If history is any guide, there will be Standard, Advanced, and Enterprise Editions of the network virtualization stack, and it would not be surprising to see the software have extra features or cost less when used in environments based solely on the ESXi-vCenter-vCloud. It is not clear what virtual switches will be supported with NSX, but Open vSwitch supported Hyper-V, KVM, and XenServer and VMware said in March it would be integrating it with ESXi. It is not clear what other virtual switches will be able to work along with NSX, but the spirit of openness would suggest that the ones from Microsoft, Cisco Systems, IBM, and NEC should be part of the compatibility matrix at some point.
What CEO Pat Gelsinger will focus on in his keynote announcing NSX is that Citigroup, eBay, and GE Appliances, and WestJet are early adopters of the technology and thus far more than 20 partners have signed up to snap their products into the NSX controller.
Like SANs through the hourglass. . . .
On the storage front, VMware is moving the vSAN virtual storage area network software that has been available in a private beta since last year to an open public beta. The company is also providing more details about how vSAN will be packaged, what workloads it is aimed at, and how it will work alongside real SANs in the data center.
The vSAN software bears some resemblance to the Virtual Storage Array (VSA) software that VMware created for baby server clusters generally stored in remote offices. That VSA software, which debuted in two years ago, is currently limited to a three-node server cluster and was really designed for customers who would never buy a baby SAN for those remote offices. With Citrix Systems, Nutanix, and others offering virtual SANs aimed at virtual desktop infrastructure and other workloads, VMware had to do something to compete. vSAN will not be a toy, but the real deal, with more scalability and performance and intended for data center workloads.
vSAN is that converged storage layer in the storage scheme VMware has cooked up
vSAN's design point, says Gilmartin, is to span a vSphere cluster, which tops out at 32 servers and 4,000 virtual machines using ESXi 5.1 in a single management domain. (This is presumably the same with ESXi 5.5.) The vSAN code takes all of the disk and flash storage in the servers in the cluster and creates pools that all of the VMs in the cluster can see and use. vSAN uses flash for both read and write caching, and disk drives for persistent storage.
"vSAN is really simple to turn on, just one click, and we are really pointing this at clusters supporting virtual desktops and lower-tier test and development environments," says Gilmartin.
Just in case you think VMware is going to take on parent EMC with its physical SANs, that is not the intent. But of course, if the pricing on vSAN software plus local disk and flash storage is compelling enough in combination, and the performance is respectable, then there is no question that vSANs will compete against real SANs as data stores for vSphere workloads. Customers will be able to point VMs at either vSAN or real SAN storage, so it is not an either-or proposition.
While NSX virtual networking is open (in terms of supporting multiple hypervisors, cloud controllers, switches, and routers), vSAN currently only works with the ESXi hypervisor, and specifically only with the new 5.5 release. And that is because it needs to be tightly integrated into the ESXi kernel to work properly.
vSAN uses solid state disks and disk drives on the servers, and has distributed RAID and cache mirroring data protection to guard against data loss if there is a drive, host, or network failure. If you want to add capacity to the vSAN, you add drives to existing hosts its or new ESXi hosts and their capacity is available to be added to the data store pool. Perhaps most importantly, each VM in a cluster has its own storage policy attached to it and controlling vSAN slice so its capacity and performance needs are always met, and the capacities for flash and disk slices are automagically load balanced across the cluster. vSAN is aware of VMotion and Storage VMotion (which respectively allow for the live migration of VMs and their underlying storage), and integrates with vSphere Distributed Resource Scheduler to keep compute and storage slices working in harmony. vSAN also has hooks into the Horizon View VDI broker and vCenter Site Recovery Manager, and is controlled from inside the vCenter management console.
vSAN has no particular hardware dependencies and works with any server that supports the new ESXi 5.5 hypervisor. vSAN will bear the same 5.5 release number as the new ESXi and vSphere, and will come in two editions when it becomes generally available in the first half of 2014. vSAN Standard Edition will have a cap of 300GB of SSD capacity per host, while vSAN Advanced Edition will have an unlimited amount of SSD storage available for each server node in the cluster and will also sport higher performance. (It is not clear if that means SSDs are driving that higher performance, or if VMware is somehow otherwise goosing the Advanced Edition.)
vSAN will work on server nodes with Gigabit Ethernet links between them, but VMware recommends that customers use 10Gb/sec Ethernet connectivity to get decent performance. Each node needs a SATA or SAS host bus adapter or a RAID controller with pass-through or HBA mode, and there has to be one SSD and one disk per node, and VMware recommends that SSDs make up at least 10 percent of total capacity for respectable performance. VMware requires vSAN to run on at least three nodes and during the beta testing, the code will top out at a mere eight nodes, but again the design scales a lot further than that.
You need vSphere Standard, Enterprise, or Enterprise Plus at the 5.5 release level as well as vCenter 5.5 to use vSAN. You can also the vCloud Suite, which includes vSphere 5.5 in its latest incarnation. You can get the beta of vSAN here. Pricing has not been set for vSAN yet, and VMware is not giving any hints, either. ®