Analyse this: IBM moves Watson machine learning to mainframes

IBM is adding the machine learning technology from Watson to its z/OS mainframe for smarter, faster analytics of transaction data.

Big Data analytics is typically applied to unstructured data in the Hadoop world. By contrast, older data warehousing and business intelligence (DW/BI) products are applied to structured data in databases, which are extracted, transformed and loaded (ETL) into data warehouses for analysis.

DW/BI facilities have long been available for mainframes but are not used that much. This 2014 Wikibon report explains why: "Sorting and transforming mainframe data in order to perform complex data analytics is difficult, expensive and requires trade-offs relative to CPU cycles. As a result, most enterprises are not leveraging mainframe data to support data-driven decision-making as much as they should.

"Considering the type of data stored in mainframes - namely customer financial transaction data - enterprises are leaving an awful lot of value on the table."

One option is "to offload mainframe JCL batch workloads to Hadoop. This approach results in significant cost savings over processing data in the mainframe, with the added benefit of having the resulting data available for analysis." A problem is the lack of native mainframe-Hadoop connectivity.

Real-time club

The latest IBM initiative is aimed at analysing mainframe data in place with Watsonian help: you get to use up mainframe CPU cycles but the trade off is faster analysis. According to IBM, in-place analytics help to minimize latency, costly processing and security risks associated with traditional ETL processes.

Of existing processes the company says: "Data scientists - in shortest supply among today’s IT skills - might spend days or weeks developing, testing and retooling even a single analytic model one step at a time."

Big Blue says: "IBM Machine Learning for z/OS ... helps organizations quickly ingest and transform data to create, deploy and manage high quality self-learning behavioral models using IBM z Systems data, securely in place and in real time."

IBM Machine Learning, using IBM Research-produced Cognitive Assist for Data Science, helps automate the creation, training and deployment of operational analytic models that will support:

• Any language (eg. Scala, Java, Python),
• Any popular Machine Learning framework like (eg. Apache SparkML, TensorFlow, H2O),
• Any transactional data type.

Gee, Brain. What are we going to do tonight?

IBM says it will simplify the work of data scientists in analytic model creation, deployment and management and help ensure model accuracy because it "continuously analyzes the data and models to provide better predictions and optimisation of behavioral models, speeding time to insights".

The Cognitive Assist for Data Science part will "assist data scientists in choosing the right algorithm for the data by scoring their data against the available algorithms and providing the best match for their needs. The service also considers various circumstances - such as what the algorithm is needed to do and how fast it needs to produce results."

By bringing recent and older transactional data into the analytics arena, data scientists can get real time results from mainframe data analytics, and so enable an organisation to react in a better informed way, and pretty much instantly, to events affecting its customers or operations, IBM says.

Its marketing blather about mainframe machine learning is creamy and thick: "The industrial revolution was a major turning point in the history of humanity. It enabled businesses to be more productive, create more jobs, and raise the overall standard of living. Today, we are on the precipice of another revolution. With machine learning done right, organizations can develop insights instantly and dramatically grow their business."

We might agree with "develop insights instantly" but not necessarily with "dramatically grow their business." What if the insight is that a product or service is crap?

We also agree that, with billions of daily transactions processed by banks, retailers, insurers, transportation firms and governments on mainframes, there is surely scope for analysing that data better.

IBM Machine Learning is available first on z/OS and then on POWER servers and, IBM teasingly says, "other platforms." We assume it means x86 servers. Go to this website to dig deeper into IBM's mainframe machine learning activities. ®