Feeds

ADVISE data-mining program cut by Homeland Security

We advise skepticism

Boost IT visibility and business value

The Department of Homeland Security (DHS) is ditching a controversial data-mining program, the Associated Press revealed today.

The Analysis, Dissemination, Visualization, Insight and Semantic Enhancement program (ADVISE), a massive data-mining system under development at Lawrence Livermore Laboratory since 2003, was capable of analyzing one billion pieces per hour of "structured" information, such as databases, and one million pieces per hour of "unstructured" information, such as intelligence reports, emails or news articles.

The program had been quietly suspended in the spring due to privacy concerns, after reports surfaced that real personal data, rather than dummy information, had been used in the testing process. The Government Accountability Office (GAO) noted at the time that "the ADVISE tool could misidentify or erroneously associate an individual with undesirable activity such as fraud, crime or terrorism."

The program would have provided information to a slew of DHS agencies, including immigration, customs, border protection, biological defense and its intelligence office. The program had been temporarily halted to allow the DHS to develop stronger privacy protections for controversial program. DHS has apparently thrown in the towel on this one.

According to DHS spokesman Russ Knocke, "ADVISE is not expected to be restarted."

The report on the program completed in June by the DHS Office of Inspector General (OIG) seems to have been the nail in the coffin for this particular domestic intelligence operation. The report criticized the operation for failing to provide required privacy protections, as well as for failing to take into account the needs of the DHS "components" that ultimately would be expected to pay for the program. The DHS's own Office of Intelligence and Analysis (OIA) rejected the program in its entirety.

The DHS was cobbled together out of a variety of occasionally competing government agencies back in 2002, and the OIG report says as much about bureaucratic turf war and chaotic, failed attempts to run the government like a business as it does about the privacy implications of government data-mining. About $42m has been invested on a program that, through a combination of communication weakness and bureaucratic infighting, at times failed to obtain necessary information from the agencies or DHS "components" that were supposed to be the ultimate "customers".

The establishment of the DHS was intended to eliminate this kind of infighting, but old habits die hard. The appropriation of business terminology provides sadly comical insight into more general failings at the DHS, as its vast and ill-defined mandate led to generally confused interactions between its myriad agencies,"components" and leadership. The DHS Science and Technology Directorate (S&T), which had the reins on this project, never developed the required "business plan," and repeatedly failed to consult with "stakeholders" and "customers" - i.e., Immigration and Customs Enforcement (ICE), the OIA, and the Customs and Border Protection Office of Strategic Trade.

Most people, of course, just want their government to work, and don't concern themselves with the minutiae of Washington politics. Mass data-mining of the type that derailed the Total Information Awareness (TIA) initiative is more unnerving than petty infighting - the behavioral algorithms pioneered by credit rating agencies, based as they are on probabilities, lead to false positives on a regular basis and touch on constitutionally protected liberties.

The root of the dilemma is that effective data-mining requires the kind of mass data aggregation that ultimately and inevitably erodes human dignity and privacy.

Although the software program at the lab has been shelved, the Knocke noted that S&T had "determined that new commercial products now offer similar functionality while costing significantly less to maintain than ADVISE." Parts of the TIA continued, of course, under other names.

Mass government data-mining is most assuredly here to stay.®

Burke Hansen, attorney at large, heads a San Francisco law office

Build a business case: developing custom apps

More from The Register

next story
Munich considers dumping Linux for ... GULP ... Windows!
Give a penguinista a hug, the Outlook's not good for open source's poster child
UK fuzz want PINCODES on ALL mobile phones
Met Police calls for mandatory passwords on all new mobes
e-Borders fiasco: Brits stung for £224m after US IT giant sues UK govt
Defeat to Raytheon branded 'catastrophic result'
Yes, but what are your plans if a DRAGON attacks?
Local UK gov outs most ridiculous FoI requests...
Detroit losing MILLIONS because it buys CHEAP BATTERIES – report
Man at hardware store was right: name brands DO last longer
Snowden on NSA's MonsterMind TERROR: It may trigger cyberwar
Plus: Syria's internet going down? That was a US cock-up
UK government accused of hiding TRUTH about Universal Credit fiasco
'Reset rating keeps secrets on one-dole-to-rule-them-all plan', say MPs
Caught red-handed: UK cops, PCSOs, specials behaving badly… on social media
No Mr Fuzz, don't ask a crime victim to be your pal on Facebook
EU justice chief blasts Google on 'right to be forgotten'
Don't pretend it's a freedom of speech issue – interim commish
prev story

Whitepapers

Endpoint data privacy in the cloud is easier than you think
Innovations in encryption and storage resolve issues of data privacy and key requirements for companies to look for in a solution.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Top 8 considerations to enable and simplify mobility
In this whitepaper learn how to successfully add mobile capabilities simply and cost effectively.
Solving today's distributed Big Data backup challenges
Enable IT efficiency and allow a firm to access and reuse corporate information for competitive advantage, ultimately changing business outcomes.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.