Gluster techie shows off 'MySQL of object storage' Minio projects
Github star count has it beating Ceph and Swift
Backgrounder Minio and its µServer were first described by El Reg in December a year ago. Now we have had a closer look, courtesy of a press tour to Silicon Valley earlier this month.
The basic product comes as three software items:
- Open source µServer
- Open Source Client
- Software development kit
The µServer is a lightweight piece of code, written in Go, that runs as VM or in a container on on-premises COTS servers and storage, and provisions object storage as containers with an S3 access protocol; S3 being the near-enough de facto object storage access protocol these days. It is hardware-agnostic and positioned as a sort-of MySQL of object storage, with S3 being the analogous SQL of object storage. A distributed version enables the pooling of multiple nodes.
Minio µServer supports various types of drive media and configurations; NVMe/SSD, SAS/SATA HDD, HGST Open Ethernet drive, RAID and JBOD configurations. It has reference hardware arrangements with Cisco, Intel, Quanta Grid and Supermicro.
For example, an OCP-compliant D512B-2U from Quanta Grid involves a 2U chassis from Quanta with dual E5-2600v3 CPUs and 12 x 3.5-inch drive bays. The SMC 1U 5018A-AR12L is an Atom-powered system with 12 x 3.5-inch drive bays.
As described by Minio the Client “provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff etc. It supports filesystems and Amazon S3 compatible cloud storage service (AWS Signature v2 and v4).” It runs on GNU/Linux, Windows, OS X, Solaris/Illumos and FreeBSD.
Minio was co-founded by Anand Babu (AB) Periasamy and Garima Kapoor in 2014 and has a recorded $3.3m in seed funding, though it might be more. This comes from three VCs: General Catalyst Partners, Nexus Venture Partners and Index Ventures, and some nine angel funders, who include Mark Leslie and Jerry Yang.
Before Minio, Periasamy was the co-founder and CTO at Gluster (June 2005 - October 2011) and joined Red Hat in the CTO office, when Red Hat bought Gluster for $136m in October 2011, staying until November 2014.
There is a team of 16 engineers with no managers, and Periasamy running the show. He talks extremely fast. Listening to him pitch his company and its software, you realise the old corporate IT development type certainties just don't exist. DevOps, S3, Docker and GitHub is the world he knows.
Minio founder and CEO AB Periasamy
You could run Minio on Amazon’s EC2 or other public clouds. Minio tracks its downloads and Periasamy said: “More than 90 per cent of our downloads are happening on Docker. We provision object storage as containers. Minio is involved as a service.”
Minio and Github
He claims the old models of block storage and file storage are all going away. MongoDB andMySQL use simple key:value storage. Developers are familiar with it and looking for S3 storage, not object storage per se. Periasamy says that there are no knobs to turn and everything is dead simple. The object storage becomes part of an application’s code stack.
Periasamy says: “Where does Minio fit? We need a scale where our data footprint is comparable to Amazon. That's where we'll survive or get crushed. … We have to be critical part of our customer's structure and then they'll pay us. … I don't want to be in the infrastructure business; that's a race to zero.”
There is a Minio community centred around its Github presence, with 113 committers, 643 community members, and 6,000+ Github stars; “star" is a "Like" in the Github developer world.
Minio Github star count versus Ceph and Swift.
The download trends show exponential growth, and Periasamy talked of 300,000 Docker pulls. He says: ”We are now as big as Amazon when it comes to S3.” The team appear to be in tune with developers and push community values quite strongly.
We asked Periasamy about Minio getting income from customers. He blithely replied: “Getting paid? At million dollar checks it starts to become reasonable for us. We don't want small business. It's a distraction. Let it go. We want growth.” He said he still has half his seed funding in the bank.
Minio has a phase 1 and phase 2 development plan. Phase 1’s goal is to lead in open source object storage. It involves providing Minio µServer and Client software and an SDK. Major software features include:
- Erasure coding over 16 drives (8+8)
- Bitrot protection through checksums
- Lambda functions triggered by events
Minio Reed-Solomon-based erasure code runs in-box or across boxes. Over 16 drives there are 8 for data and 8 for parity. Data and parity blocks are sharded across the drives. Half the 16 drives could die and you're okay.
Users can replicate from Minio to S3 Amazon or Google Cloud Storage (via S3) targets. You can continuously mirror between any S3-compatible storage service. Minio Mirror does a full scan and transfer first and then sends updates.
Phase 1 started in 2015 and ends at the end of 2016. Phase 2 runs from 2017 to 2018 and its overall aim is to accelerate object storage adoption. It includes a Minio X product and a deep learning engine.
Periasamy says: “The value of the data is many times more than the storage.”
He thinks a killer app is to be able to search within unstructured data, like images, using deep learning-generated metadata. The idea is for the system to respond to queries such as: ”Find a blue coffee mug.” The metadata for a blue coffee mug could be viewed as a kind of semantic hash.
This by the way, as we understand it, is currently impossible.
Machines could then recognise coffee mugs in photos with no humans in the loop. In other words Minio X will generate and provide object content metadata, such as descriptions of elements in an image.
Hang on, are they looking to kill image CAPTCHAs?
At this point we began to find it hard to be on the same planet. This is a guy who takes in $10m or so seed funding to build an S3-compatible object storage server, which he then open-sources. He’ll take in million dollar checks from large enterprises who like Minio for some reason, and then go off and build a deep learning facility to search inside objects.
He's a nice guy and talks at lightning speed, and we’re left thinking he’s either a genius or or a speed-talking fantasist.
Given the Gluster and Red Hat background, storage desk tends to think its the former. Put Minio on your S3/object storage/deep learning radar. ®