Start-up promises era of grand log collection
Honest Abe would be proud
Talking about logs is like, well, talking about logs. Your log handling expertise isn't usually the big kicker item you want to tout to your CIO or CEO for career development. But that might be changing.
US start-up LogLogic has charged itself with the task of making logs sexy. (No that isn't some feeble scat pun.) Using a combination of proprietary software, compression techniques and appliance-like hardware, LogLogic has made it possible for companies to store 100 per cent of their log data. Customers can then use more sophisticated, custom LogLogic software to pore through all this information.
Why would any company want to do this?
Like many of the predators circling the storage industry, LogLogic has turned to fear as a primary sales pitch. You want to save all your log files because a regulator or lawyer might come looking for a specific bit of compliance information around who accessed certain servers, where employee X sent a confidential file or who saw person X's health care information. In addition, companies may want to make sure old staffers don't still have access to protected servers and that customers aren't making their way to supply chain data meant to be off limits.
"We are kind of like the Google or Yahoo! for the world of machine data," said Andy Lark, chief marketing officer at LogLogic.
In the past, companies could not afford to keep all of their log data from a storage cost or time point of view. But the ever-falling price of disk coupled with high-performing chips and compression software has made it possible for LogLogic to store 24TB of log data on a single appliance and churn through up to 50,000 messages per second.
(As a point of reference, LogLogic said a typical security event recording appliance would only track about 245m message per day.)
Most existing products aimed at collecting log data tend to fall in a niche area such as security or web serving. LogLogic cheers such efforts but, of course, says they're not enough.
"This has gone from using log management to gain some intelligence in select areas to being a best practices kind of thing for any organization," Lark said.
LogLogic can boast customers such as Yahoo!, Fidelity, Harley-Davidson and JPMorgan Chase. These firms use the log management tools for a wide variety of tasks such as seeing what customers do on a given web site and checking that IT and HR personnel are using the correct procedures for canceling an ex-employee's access to a network.
The company, which has been in business for two years and just opened a new office in the UK, has released this week Version 3.1 of its appropriately named LogLogic software.
Customers will find that the new code runs faster than before and provides a host of new options for customizing log searches and analysis. For starters, companies can now collect log data from their own in-house, custom applications. LogLogic spent a lot of time making sure its software could recognize the unique data outputs and fields of various organizations.
In addition, the new software includes 13,000 report templates that can be used to produce information relevant to Sarbanes-Oxley, HIPAA, COSO and other regulatory frameworks. The templates help companies produce and analyze the right types of data and can be customized for particular jobs. To complement these additions, LogLogic 3.1 boasts speedier indexing and search technology, a type of artificial intelligence technology for alerting administrators to odd patterns or behavior and tie-ins to storage management systems from EMC, OnStor and NetworkAppliance.
A low-end LX appliance from LogLogic will start at close to $20,000, while a high-end ST 3000 box can cost close to $200,000. LogLogic has also just released a midrange box starting at around $75,000. (At the moment, the boxes primarily run on Xeon chips from Intel, have large amounts of internal storage and memory and take up between 1U and 3U of rack space. In the future, LogLogic may switch to AMD's Opteron processor in the hopes of improving price/performance.)
LogLogic has a unique background for a company touching the storage market in that it is filled with ex-Visa staffers - the folks who write software to detect weird purchasing trends and other patterns. Now this group have turned their focus to enterprise-wide logs. (Oh, come on, grow up. Stop with the jokes already - Ed.)