It's a Flink-off: Hazelcast launches own big data munching distributed processing engine

Brilliant, it's about jolly time someone invented one of those

By Alexander J Martin


In-memory computing hopeful Hazelcast is launching Hazelcast Jet, a distributed processing engine for the streams of big data which the business expects will be pouring into enterprises soon.

Hazelcast's eponymous in-memory data grid is based on commodity hardware rather than expensive proprietary Oracle Exadata boxes, and will be providing storage functionality for Jet. With both computation and storage kept in-memory, Hazelcast hopes to achieve high speed and low latency.

The company's CEO, Greg Luck, said the business had recognised an “opportunity in the market for Hazelcast as a distributed computing platform to provide something very fast and simple, not just for programming but also for operations.”

Apache Flink-competitor Jet, which the company has licensed as per Apache 2.0, aims to perform parallel execution to enable data-intensive applications to operate in "near real-time".

Unlike Hadoop and Spark, which run from OS processes, Hazelcast Jet doesn't demand operations maintain a whole separate infrastructure. Jet can execute both batch and stream-based data processing applications, which the company believes there to be much market hunger for, especially with applications that require near real-time workloads from sensor updates in IoT architectures (house thermostats, lighting systems), through to in-store e-commerce systems as well as social media platforms.

Luck said: “Hazelcast Jet is a super fast, low latency, next generation directed acyclic graph engine for big data processing. We believe that the Hadoop and Spark ecosystems are too complex to program and to deploy and have set out to bring Hazelcast’s legendary simplicity to big data.”

“We have designed it as a general purpose engine for the intersect of big data programmers and Java programmers,” with a specific focus for existing Hazelcast users.

Hazelcast Jet is built on top of a one-record-per-time architecture (sometimes known as continuous operators), processesing incoming records as soon as possible, as opposed to accumulating records into micro-batches, consequently lowering latency for applications. Jet ingests data at high-velocity (via socket, file, HDFS or Kafka interfaces), and processes the business logic or complex computation on incoming data. ®

Sign up to our NewsletterGet IT in your inbox daily

Post a comment

More from The Register

Storage bunfight: Hazelcast says Gridgain faked benchmark result

In-memory startup denies Hazelcast's alleged porkies

Hazelcast adds amped-up caching to its in-memory tech

Company goes NUTS with enterprise add-on to open-source core

Hazelcast and C24 waste no time, emit Hypercast after conjoining

Market listens for the tell-tale pitter-patter of tiny little PREON files

Hazelcast trousers cash, poises open-source cutlass against Exadata belly

Do ya feel Luck-y, punk?

Hazelcast signs Java speed king to its in-memory data-grid crew

Ex-Ehcache man takes CTO crown as VC cash pours in

You've feasted for days but the courses keep on coming: Let's talk storage turkey

And finally sir, a wafer-thin mint... it is but wafer-thin

Get that trash out of your cache

Striim shim prevents stale data Hazelcast in-memory cache corruption

Curiouser and curiouser: Quasi NVDIMMs, GDPR-tastic SaaS and more

Behind the looking glass in Storage-land

Four hybrid clouds, three server SANs, two snapshotters... and a partridge in a pear tree

Yes, it's the week in storage

Security patches done, machines upgraded, the coffee's hot – just in time for storage roundup

Could Nutanix be gearing up for an acquisition?