Watson can't cure cancer ... or all the stuff that breaks IT projects
University spends $62m on AI trial, gets the usual trials that come with failure
A University of Texas audit report last week tipped a bucket on the conduct of a high-profile “Watson to cure cancer” project.
The University has criticised the MD Anderson Cancer Center's “Oncology Expert Advisor (OEA) Project”, which since 2014 has poured tens of millions into seeing if IBM's Watson was smart enough to beat cancer.
While staying out of the scientific merits of the project, the report [PDF] has plenty to say about project governance.
The Houston-based MD Anderson Cancer Center put its Watson experiment in place without going through its normal IT procurement processes, and ultimately sent US$62 million in the general direction of Big Blue and PwC.
The project never went live before it was terminated in September 2016, and the report also questions the role of MD Anderson scientist Lynda Chin, wife of the institution's president Ronald DePinho.
Other criticisms in the audit include:
- ”Work performed under an amended scope of work extended beyond the OEA project and intent of funding as approved by the Board of Regents of the UT System (Board)”; and
- ”Invoices were paid in full regardless of whether contracted services were delivered as agreed upon, and invoice review and approval to process payment to the vendor was not consistently documented as evidence that invoiced services and deliverables were both received and acceptable.”
The audit also finds much of the money was spent without a competitive tendering process, and that fees “were consistently set just below the amount that would have required Board approval.”
Most of the money came from donations tied to the project, but there's a problem there as well: it's in deficit, because it spent $11.59 million of gifts it had not yet received.
As the Houston Chronicle notes, Chin has defended herself, complaining that the auditor dismissed her expertise, but noted that “discussions and collective decisions could have been better documented”.
The project looks so badly stalled it's hard to imagine it being resumed. Not only has the system never been piloted outside MD Anderson, IBM ended support for the project in 2016. It may not be used for treatment except in test and evaluation cases. To restart the project, the centre would also have to integrate it with a new medical records system MD Anderson implemented.
There's another illustration of how hard it is for IT timelines to cope with the outside world: “staff told us that drug protocol and clinical trial data in OEA is now outdated and must be updated before OEA can be piloted again within MD Anderson, before conducting pilot testing with a network partner.” ®