Feeds

Sumo Logic slings log file ops into the clouds

Meet me at the Chatsubo bar for a big data bash

Next gen security for virtualised datacentres

Sumo Logic CTO and co-founder Christian Beedgen sports a Chatsubo Bar banner on his t-shirt. When you meet a startup guy wearing a Neuromancer t-shirt you know the product is going to be a dreadful self-indulgent failure or an extremely cool resource that delivers the goods. Sumo Logic's Log Management and Analytics Service is the latter.

Christian Beedgen

Sumo Logic co-founder and CTO Christian Beedgen (IT Press Tour)

But log files? Log files are not cool. However, they are vital when apps degrade or crash. The log files – the record of the events that occur when the app executes – are the only detailed info into the application's run-time history that IT administrators have. The collected log file records are like a data centre flight recorder, a black box. When there is an airplane crash or failure the black box is the first port of call.

Every app has its own log file format, though they are often ASCII files. There can be tens of thousands of these records, terabytes of the stuff. And the problem of checking log files grows because we have application suites with many components running on different servers linked by network devices. Soon there are millions of log file records. The problem expands to finding the ones of interest, the needles in the vast haystack, and finding them quickly when something goes wrong in an application suite that is used by hundreds if not thousands of users and is critical to the running of your business.

Kumar Saurabh

Sumar Logic co-founder and acting CEO Kumar Saurabh

There is a second aspect to this, and that is the storing of the log files and ensuring you have enough storage capacity for that and then enough server capacity to run the analytic queries. Sumo Logic aims to deal with both problems.

Log file library operations

What co-founder and acting CEO Kumar Saurabh and Beedgen have done is to provide a cloud-based log file recording and storing service together with analytics software to help admin staff search and identify log file records of interest. We could think of Sumo Logic's Log Management and Analytics Service as working like a librarian. Books are stored in a library and classified, by the Dewey Decimal scheme for example.

Researchers need to find books but don't understand book type classification. They tell the librarians what they want and the library staff find it, without themselves knowing about the detailed contents of books. The analytics service provides algorithms for finding log files of interest. It has a specific function called Log reduce, a tongue-in-cheek reference to Hadoop's Map Reduce. Although it works in a different way the net effect is the same in that it produces a subset of records of interest out of the thousands of possible hits.

Operators might be looking for events connected with a particular user, IP address or business code of some sort and Log Reduce will quickly sort through a large population of log files and produce a manageable subset that a human can look through and assess and use to decide what to investigate next.

Patterns and collectors

The analytics software looks for patterns and can learn what patterns to look out for, further helping operators to refine subsets of interest.

The log files are collected by collectors, software agents that can operate in the servers where the events are occurring or remote from them, and thus not using any CPU cycles in them. They are collected and then sent up to the cloud for subsequent access. Although there can be many of them they are individually small and don't consume much network bandwidth.

Whatever cost is involved here is inconsequential when compared to the increase in productivity of the operators investigating log files for the cause of a failure, the founders say. Sumo Logic uses its own software to manage the operations of its own service and so has direct experience of its utility and a direct connection to its users' experience of the service.

CTO Beedgen said that the days of developing software and shipping copies to customers all over the world are gone ... and as far as he is concerned, they are not lamented. It is far easier for developers to look after software in one place which provides a service to thousands of users than thousands of copies of software in thousands of remote places needing a substantial support and software distribution operation.

The cloud is real

You listen to people like Beedgen and Saurabh explaining how their cloud service works, how it improves the necessary grunt work involved in log file collection, storage and analysis, and you realise that for applications like this the cloud is just a natural and logical fit. The pair are providing log monitoring and analytics to cloud computing environments via a Software-as-a-Service model.

Sumo Logic is doing its bit to transform the world of log file collection, storage and analysis into a cloud service. For its users, once they have experienced log file operations in the cloud, there is pretty much no going back. ®

5 things you didn’t know about cloud backup

More from The Register

next story
The Return of BSOD: Does ANYONE trust Microsoft patches?
Sysadmins, you're either fighting fires or seen as incompetents now
Oracle reveals 32-core, 10 BEEELLION-transistor SPARC M7
New chip scales to 1024 cores, 8192 threads 64 TB RAM, at speeds over 3.6GHz
Microsoft: Azure isn't ready for biz-critical apps … yet
Microsoft will move its own IT to the cloud to avoid $200m server bill
Docker kicks KVM's butt in IBM tests
Big Blue finds containers are speedy, but may not have much room to improve
US regulators OK sale of IBM's x86 server biz to Lenovo
Now all that remains is for gov't offices to ban the boxes
Gartner's Special Report: Should you believe the hype?
Enough hot air to carry a balloon to the Moon
Flash could be CHEAPER than SAS DISK? Come off it, NetApp
Stats analysis reckons we'll hit that point in just three years
Dell The Man shrieks: 'We've got a Bitcoin order, we've got a Bitcoin order'
$50k of PowerEdge servers? That'll be 85 coins in digi-dosh
prev story

Whitepapers

Endpoint data privacy in the cloud is easier than you think
Innovations in encryption and storage resolve issues of data privacy and key requirements for companies to look for in a solution.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Top 8 considerations to enable and simplify mobility
In this whitepaper learn how to successfully add mobile capabilities simply and cost effectively.
Solving today's distributed Big Data backup challenges
Enable IT efficiency and allow a firm to access and reuse corporate information for competitive advantage, ultimately changing business outcomes.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.