Feeds

ADVISE data-mining program cut by Homeland Security

We advise skepticism

5 things you didn’t know about cloud backup

The Department of Homeland Security (DHS) is ditching a controversial data-mining program, the Associated Press revealed today.

The Analysis, Dissemination, Visualization, Insight and Semantic Enhancement program (ADVISE), a massive data-mining system under development at Lawrence Livermore Laboratory since 2003, was capable of analyzing one billion pieces per hour of "structured" information, such as databases, and one million pieces per hour of "unstructured" information, such as intelligence reports, emails or news articles.

The program had been quietly suspended in the spring due to privacy concerns, after reports surfaced that real personal data, rather than dummy information, had been used in the testing process. The Government Accountability Office (GAO) noted at the time that "the ADVISE tool could misidentify or erroneously associate an individual with undesirable activity such as fraud, crime or terrorism."

The program would have provided information to a slew of DHS agencies, including immigration, customs, border protection, biological defense and its intelligence office. The program had been temporarily halted to allow the DHS to develop stronger privacy protections for controversial program. DHS has apparently thrown in the towel on this one.

According to DHS spokesman Russ Knocke, "ADVISE is not expected to be restarted."

The report on the program completed in June by the DHS Office of Inspector General (OIG) seems to have been the nail in the coffin for this particular domestic intelligence operation. The report criticized the operation for failing to provide required privacy protections, as well as for failing to take into account the needs of the DHS "components" that ultimately would be expected to pay for the program. The DHS's own Office of Intelligence and Analysis (OIA) rejected the program in its entirety.

The DHS was cobbled together out of a variety of occasionally competing government agencies back in 2002, and the OIG report says as much about bureaucratic turf war and chaotic, failed attempts to run the government like a business as it does about the privacy implications of government data-mining. About $42m has been invested on a program that, through a combination of communication weakness and bureaucratic infighting, at times failed to obtain necessary information from the agencies or DHS "components" that were supposed to be the ultimate "customers".

The establishment of the DHS was intended to eliminate this kind of infighting, but old habits die hard. The appropriation of business terminology provides sadly comical insight into more general failings at the DHS, as its vast and ill-defined mandate led to generally confused interactions between its myriad agencies,"components" and leadership. The DHS Science and Technology Directorate (S&T), which had the reins on this project, never developed the required "business plan," and repeatedly failed to consult with "stakeholders" and "customers" - i.e., Immigration and Customs Enforcement (ICE), the OIA, and the Customs and Border Protection Office of Strategic Trade.

Most people, of course, just want their government to work, and don't concern themselves with the minutiae of Washington politics. Mass data-mining of the type that derailed the Total Information Awareness (TIA) initiative is more unnerving than petty infighting - the behavioral algorithms pioneered by credit rating agencies, based as they are on probabilities, lead to false positives on a regular basis and touch on constitutionally protected liberties.

The root of the dilemma is that effective data-mining requires the kind of mass data aggregation that ultimately and inevitably erodes human dignity and privacy.

Although the software program at the lab has been shelved, the Knocke noted that S&T had "determined that new commercial products now offer similar functionality while costing significantly less to maintain than ADVISE." Parts of the TIA continued, of course, under other names.

Mass government data-mining is most assuredly here to stay.®

Burke Hansen, attorney at large, heads a San Francisco law office

The essential guide to IT transformation

More from The Register

next story
GCHQ protesters stick it to British spooks ... by drinking urine
Activists told NOT to snap pics of staff at the concrete doughnut
Britain's housing crisis: What are we going to do about it?
Rent control: Better than bombs at destroying housing
Top beak: UK privacy law may be reconsidered because of social media
Rise of Twitter etc creates 'enormous challenges'
What do you mean, I have to POST a PHYSICAL CHEQUE to get my gun licence?
Stop bitching about firearms fees - we need computerisation
Ex US cybersecurity czar guilty in child sex abuse website case
Health and Human Services IT security chief headed online to share vile images
We need less U.S. in our WWW – Euro digital chief Steelie Neelie
EC moves to shift status quo at Internet Governance Forum
Oz biz regulator discovers shared servers in EPIC FACEPALM
'Not aware' that one IP can hold more than one Website
prev story

Whitepapers

Endpoint data privacy in the cloud is easier than you think
Innovations in encryption and storage resolve issues of data privacy and key requirements for companies to look for in a solution.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Advanced data protection for your virtualized environments
Find a natural fit for optimizing protection for the often resource-constrained data protection process found in virtual environments.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Next gen security for virtualised datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.