Original URL: http://www.theregister.co.uk/2010/04/21/redhat_rhel6_beta_1/

RHEL 6 gets its first beta

Massive scale, less virty overhead

By Timothy Prickett Morgan

Posted in Operating Systems, 21st April 2010 15:20 GMT

The bit twiddlers at Red Hat have been hard at work for the past four years, laying the groundwork in the upstream Linux kernel for what will become Enterprise Linux 6, says Tim Burke, vice president of Linux engineering development at the company. And with RHEL 6, which just entered its first beta today, the company wants to make it cheaper and easier for companies to deploy Linux in their data centers.

With the RHEL 5.5 update three weeks ago, Burke says that the focus was to make that Linux distro compatible with all the new processors coming out this year, including the Xeon 5600 and 7500 from Intel, the Opteron 4100 and 6100 from Advanced Micro Devices, and the Power7 from IBM.

RHEL 5 came out in March 2007 after a six-month beta and release candidate cycle. It started out with the Linux 2.6.18 kernel and has had lots of more current features backported to it without breaking application compatibility as each successive release came out over the past three years. At some point, however, the changes that need to be made to the kernel and related systems software stack are so great that a new version with a new architecture is called for, and such is the case with RHEL 6. Although the two releases will, of course, share some common attributes in terms of scalability and hardware support.

To cook up RHEL 6, Red Hat grabbed a Fedora development release based on the Linux 2.6.32 kernel and hardened it, and then grabbed some of the features that the RHEL engineers deemed were ready for primetime and backported them to that kernel. The main concern with RHEL 6, aside from building in lots of scalability, is to tweak Linux and its management tools so virtualized servers are no more difficult to manage than physical ones, and to remove some of the performance penalties that virtualized servers have to pay.

One big change in the guts of the kernel at the heart of RHEL 6 is that the CPU structures that determine how many physical processors the kernel can span have been made more flexible, thus making it easier for Red Hat to dial up support for increasing numbers of processor cores as chip makers ride Moore's Law. It looks like we'll be getting 50 per cent more cores per year from here on out, on average, because we can’t increase clock speeds any more because of thermal issues, and every operating system has to be ready to absorb that number of cores.

RHEL 6 will have no trouble, laughs Burke, since in theory it has been designed to support up to 64,000 cores in a single system image. And because the way that CPU counts are assigned in the kernel - not with a fixed array that mapped CPU structures to the kernel like in RHEL 5 and earlier releases - it will be relatively easy to make changes as new chips come out.

Ditto for main memory addressability. While RHEL 4, 5, and 6 offer both 32-bit and 64-bit memory addressing, depending on the iron you run it on, the address bits tend to be set a bit lower than 64-bit for physical addressing. With RHEL 5, the theoretical memory limit in the kernel was 46 bits, which yields a maximum of 64 TB of addressable memory. With the RHEL 6 kernel, the theoretical maximum physical memory has been boosted to 48 bits, which yields 128 TB of memory for the kernel and 128 TB for the userspace where Linux applications frolic.

Of course, RHEL 6 has not been tuned to run on a 64,000-core server with 256 TB of memory, not the least of which because it will be at least a few years before such a system is available from Lenovo or Tata. (If you assume 64 sockets maximum and a 50 per cent increase in cores per year, you get to 1,000 cores per chip and 64,000 cores per SMP server in 2022, assuming a linear ramp of Moore's Law.)

"At this point, we have plenty of headroom for CPUs and memory," Burke chuckles. RHEL 5 had a practical memory limit of 1 TB, and RHEL 6 is ready to support the 2 TB main memory being offered on Xeon 7500 and Power7 systems and will scale as the hardware scales.

The KVM hypervisor embedded in RHEL 6 is also getting scalability improvements, which will have larger number of virtualized cores than was possible with RHEL 5 and be able to allocate more CPUs to guest virtual partitions running atop RHEL 6. With RHEL 5, KVM topped out at allowing a guest to address 16 physical processors, and with the initial RHEL 6, this has been quadrupled to 64 physical processors. (When Red Hat says "processor," it means cores, not sockets containing cores or virtual threads inside cores.)

The amount of memory that a KVM guest can address was not at Burke's fingertips when we interviewed him and the release notes that are supposed to be live dead end in a 404 at the moment, so we can't tell you. The original Red Hat Enterprise Virtualization 2.2 hypervisor that is based on RHEL 5 was able have a guest address up to 16 processors and 256 GB of memory, and with RHEL 5.5 that was boosted to 32 processors and 512 GB of memory. It stands to reason that the KVM embedded in RHEL 6 can address 1 TB of memory to match its 64 processors.

RHEL 6 is available on 32-bit x86 and 64-bit x64 processors from Intel and AMD and Power7 and mainframe processors from IBM. As The Register divulged in December 2009, Red Hat is pulling the plug on Intel's Itanium processor support with RHEL 6, much as Microsoft has done with Windows. RHEL 5 will be the last Red Hat Linux to support Itanium (out to 2014 with security patches and updates to support the future "Kittson" and "Poulson" Itanium chips). Microsoft's Windows Server 2008 R2 edition is the last one to support Itanium and will be similarly tweaked to support the two future Itanium chips on the Intel roadmap.

Interestingly, the tickless kernel that made its debut in Red Hat Enterprise MRG - the messaging, real-time, grid variant of RHEL aimed at high performance computing customers - is mainstreamed with RHEL 6. So companies won't have to buy the MRG variant to get this feature. However, Burke says that MRG will continue to be a distinct distro from Red Hat, since there will always be some customers where network latency or interrupt handlers are an issue. Exactly what might be in this RHEL 6 MRG product, Burke would not say.

There are a number of interesting features to boost the performance of applications running in virtualized mode. For instance, RHEL 6 includes a feature called transparent huge tables, which takes the concept of huge page memory support that has been around forever on physical servers to boost the performance of databases. In virtualized environments, huge pages were not supported in guests at first, and then when it was brought in, they had to be configured at boot time to preserve memory for the huge pages. Now, the KVM hypervisor can transparently configure huge as well as small or large memory pages as applications running in its guests need. This removes a barrier to the performance of virtualized databases.

Ditto for the networking stack. According to Burke, the way the virtualized networking stack was implemented in RHEL 5, the QEMU emulation layer of KVM hosted the networking stack, but with RHEL 6, a lot more of the networking code has been pushed down into the kernel, so physical and virtual machines have equal access to the features and, more importantly, equal performance. Similarly, the Asynchronous I/O support that has been in Linux for years has been reworked and has a more complete set of features.

"The point is, you no longer pay the virtualization overhead," says Burke.

RHEL 6 includes the ext4 file system as its default, and with the new version it will be able to scale to 16 TB file systems. The important new change with this implementation of ext4, says Burke, is that there is an improvement by many orders of magnitude in the file system recovery time in the event of a crash. It used to take hours to recover a file system on ext4, but now a 16 TB file system can recover itself in a matter of minutes. Red Hat is also making the XFS file system part of the RHEL 6 stack for customers who need to support very large file and directory sizes. NFSv4 is also tossed in, which supports the IPv6 network protocol.

Network block storage is also supported through the FCoE and iSCSI protocols, and LVM/DM volume managers now can resize mirrored and multipathed volumes on the fly. Intel, Cisco Systems, and IBM have done a lot of work on the FCoE drivers along with Red Hat, and they now include topology awareness commands that allow RHEL 6 to ask storage arrays what their optimum chunk sizes and file system specs should be and then pass this information up to virtualized guests and databases to optimize the performance of the software against the specific storage software.

On the RAS front, RHEL 6 has been tweaked to add support for hot-adding of CPUs and memory where the new hardware supports it, and adds hardware checksums for data moving from an application through the system out to disk storage using a feature called DIF/DIX.

RHEL 6 will go through a number of betas and release candidates before being officially launched. How many depending on tester feedback. But if history is any guide, it should take five or six months for RHEL 6 to get fully baked.

RHEL 6 will be available in both server and desktop variants, and you can take the first beta for a spin here. ®