Onwards to Valhalla: Java ain't dead yet and it's only getting bigger

Please, contain yourself

In JDK 8, JVMs were not container aware. That has an impact, because it made it more difficult for a JVM to evaluate the computing resource its container had. "There were situations where it could effectively try and use too many resources and it would cause you some problems. That has been resolved to a certain extent in JDK 9," said Simon Ritter, deputy CTO at Java runtime firm Azul Systems, who represents it on the Executive Committee.

A big factor in program performance is garbage collection. Unlike C++, where developers have to keep track of their own objects and reclaim the memory they were using when they are no longer needed, Java is a managed memory language, in which the runtime takes care of the problem for you using garbage collection algorithms. Garbage collection is a devilishly difficult thing to do.

"The big thing that everyone worries about are these 'stop the world' events," said Kirk Pepperdine, principal consultant at Java performance company Kodewerk. In this scenario, a JVM dealing with a large heap space must sometimes stop all threads to clean up all the objects that are no longer referenced, which in turn stops the application running. That can occasionally take minutes at a time, with obvious performance implications, and they are also nondeterministic events – you don't know when they are going to happen.

Garbage collection can be especially problematic in web-style environments, where lots of objects are created in short periods of time, and are often short-lived. This leads to a challenging problem for garbage collection algorithms.

Taking out the trash

Java 9 switched its garbage collector to one that had been around as an option since version 7: G1.

"G1 is Oracle's attempt to maintain pace with this lead to hyper scale," Pepperdine said, adding that "it sucked" when it first came out, but now, as the new default in Java 9, it's better.

Chris Engelbert, managing of developer relations for in-memory computing platform vendor and Java Community Process Executive Committee member Hazelcast, said G1 is still a generational garbage collector, which divides the processing of young objects and longer-persisting ones. It expects most objects to die extremely young, and it only copies living objects to the next stage, simply disregarding those that are not longer used.

"The problem is that for larger applications you normally have a database and you have to add some kind of caching, and that means long-living objects," he said. "They move to later generations, and those generations have to be collected differently."

Engelbert says that while G1 is a low-latency collector, it still has problems.

"If it can't keep up with the rate of allocation or the rate of fragmentation of the heap space, then it will still resort to having to pause all the application threads and then do a full compaction on the old generation."

Hazelcast makes its money from providing high-performance alternatives to Java's managed memory. It takes all the memory management off-heap and handles it for you itself, so it naturally sees the flaws in the tech. Nevertheless, Pepperdine has also identified shortcomings in the system.

There's nothing that says you have to use G1 if you don't want to. Admins can specify alternative garbage collectors at runtime, and there are several available, often embedded in alternative versions of the vanilla OpenJDK Java SE reference implementation.

For example, Azul has its own JVM with what Ritter calls a "soft real-time JVM" with others working on their own. Eclipse now has OpenJ9, a full implementation of IBM's J9 Bluemix-based JVM that is optimised for large server environments. Red Hat's Shenandoah offers what employees have called a "pauseless" garbage collection algorithm.

One intriguing idea Pepperdine points to is that Java may have the option of no garbage collection at all. When you run out of memory on the heap, that's it. Game over. Wait – what?

It's not as crazy as it sounds, he claims. "If you get into microservices container environments, then if you understand how much memory you're using in a container, then when you get close to the limit you can just recycle the VM." After all, containers are supposed to be ephemeral and short-lived.

Then, he suggests, you'll just run stateless or manage state by replicating containers, which any self-respecting microservices deployment should be doing anyway for resiliency’s sake.

10 more years!

There are some future Java enhancements that could have some implications for large-scale web deployments. One of them is here now. Sort of. Ahead-of-time compilation is a feature in Java 9, but it's still marked as experimental and unsupported.

"The normal operation of the JVM is to load classes and optimise their execution during runtime," said Donald Smith, Oracle's senior director of product management. "This allows Java applications to often run faster than native compiled applications because optimisations are made continuously and on-the-fly." Java's just in time (JIT) compilation needs some time to optimise its compilation, resulting in an initial "warm-up time" as it becomes more efficient.

"AOT allows for some of this optimisation work to take place in advance, in most cases resulting in faster initial performance," Smith added. "It achieves this by looking for AOT-compiled methods created in advance when a JVM loads a class." In short, AOT compilation streamlines JIT warm-up time, which can only be a good thing for short-lived, container-based JVMs.

Engelbert points to the curiously named Project Valhalla, which introduces an alternative to objects.

"Valhalla gives you a new type in Java which is not an object any more, but a complex value." The promise of this kind of implementation is that chains of methods can be passed and handled on the stack, rather than the heap. "It doesn't need to be an object and it doesn't need to live on the heap. If it doesn't have to live on the heap, then obviously there is no garbage collection," he said.

Valhalla was the place in Norse mythology where the worthy dead resided with Odin.

Java is not dead. Far from it – despite being nearly smothered during the log-jam of the latter Sun-Microsystems era. Java is already good at web-scale deployments but even good can get better, it seems. ®