IO, IO, it's off to Weka.io we go: Let's take a look at a file system upstart
Holds up versus IBM, Oracle in benchmarks... whether it's worth the cost is another thing
Weka.io claims its file storage device can scale out to billions of documents. Rather than rely on object storage to cope with this kind of data, you just need to reinvent the filer, which it reckons it has done.
The WekaIO Matrix is said reinvented filer: it's software that runs on any SSD-packed commodity server or cloud compute instance, and forms a single cluster. It presents a single high-performance scalable global namespace for this storage pool to applications running in the cluster.
Weka says the software works in parallel, is designed for flash, and is POSIX compliant. We're told it is the world’s fastest distributed file system, processing twice the workload of IBM’s Spectrum Scale in a SPEC SFS 2014 software build showdown. That benchmark showed it running at 500 builds. Weka hopes a second SFS2014-swbuild attempt, one using 120 cloud compute instances instead of the 60 used in the 500 build result test, will hit the 1,000 build mark, its system's performance scaling linearly as you add compute instances.
A look at the SPEC SFS2014-swbuild summary shows Weka Matrix used some 14TB of memory compared to Spectrum Scale's 1TB, and the Oracle ZFS Storage's 640GB. So even though Weka won the benchmark, whether or not it offers the best price/performance figure is another matter. The tests don't reveal that.
"A single WekaIO Matrix file system was created and distributed evenly across all 60 [EC2] instances in the Amazon cluster," the benchmark notes read. "Data was protected to an 15+2 failure level. All data and metadata is distributed evenly across the cluster.
"The deployment model is as a hyperconverged architecture where both storage and application services are run on the same infrastructure. Each EC2 instance has 32 vCPU, 244GiB of memory and 2x320GB (total 640GB) of SSD local to the instance. Two cores of the available 32 (6.25 per cent) were provisioned with MatrixFS, the remaining 30 cores were available for application usage. Each instance had 13GB (5 per cent) of memory assigned to WekaIO."
Screenshot of the Matrix user interface
The compute instances used Intel Xeon E5-2670 v2 CPUs clocked at 2.50GHz, and Weka used two cores and 13GB of memory per instance. Local SSDs inside the servers are aggregated into one logical pool, which is then presented as a single namespace to the host applications. A transparent tiering layer offloads cold data to any S3 or Swift cloud object store for unlimited capacity scaling, under the same single namespace.
Liran Zvibel, Omri Palmon, and Maor Ben-Dayan founded Silicon Valley-based Weka in 2013, as their second venture following XIV, which was acquired by IBM in 2008. The upstart is accepting purchase orders for its software effective immediately. ®
PS: Yes, we had noticed that there's something awfully familiar with the Weka homepage. We can't quite put our finger on it.
Sponsored: Becoming a Pragmatic Security Leader