The recent disclosures of decades long data gathering of individualized communications behaviors by the US government intelligence community, have confirmed that the government considers the gathering of massive amounts of personal data, as essential for what they claim to be doing. So motivated are these official actors that many citizens and organizations believe that they have moved beyond the traditional limits that our bill of rights placed on this type of government behavior.
Is this ever increasing movement toward greater surveillance and information gathering, just the result of willful indifference to our legal safeguards ? Sometimes it is. But one might also ask whether the very logic of data mining methods, engenders ever greater data acquisition for its projects ?
"The analytical techniques used in data mining are often well-known mathematical algorithms and techniques. What is new is the application of those techniques to general business problems [ and by government ] made possible by the increased availability of data and inexpensive storage and processing power." The current data mining approach to intelligence production is based on the need to develop huge digital data warehouses to successfully apply sophisticated computer technology to develop actionable intelligence.
The key aim of data mining is to uncover hidden patterns in large data sets. To detect something new that may not be apparent to experts because it lies outside their experience. This process requires advanced statistical methods and complex software algorithms and expert human input, to be able to surface these hidden patterns that may exist in the data provided.
"Data mining analysis tends to work from the data up and the best techniques are those developed with an orientation towards large volumes of data, making use of as much of the collected data as possible to arrive at reliable conclusions and decisions..."
( CHAPTER 1 , Data Mining And Warehousing Concepts )
The greater the volume and diversity of data that can be assembled for this process, the greater the probability for uncovering new connections and patterns that can be turned into actionable knowledge." In the business world, digging through and analyzing enormous sets of data with data mining methods has made it possible to predict behaviors and future trends and answer business questions that traditionally were too time consuming to resolve. Automated data processing, aided by other discoveries in computer science have made it possible to work with data sets that have grown in size and complexity."
It should be no surprise that the government would eagerly turn to using these same methods in a post-9/11 world, to enhanced the traditional means of intelligence gathering against the new non-state enemies. "NSA is reported to use its computing capability to analyze "transactional" data that it regularly acquires from other government agencies, which gather it under their own jurisdictional authorities. As part of this effort, NSA now monitors huge volumes of records of domestic emails and Internet searches as well as bank transfers, credit-card transactions and travel and telephone records, according to current and former intelligence officials interviewed by the Wall Street Journal."
A current news story reports on the building of two huge data storage centers that the NSA "...will need to intercept, copy and mine the metadata from 1.7 billion daily phone calls, texts, emails and other electronic communications the agency reportedly tracks..." The article states that about a quarter of the new space will be dedicated "... to storing numbers that replicate the way people around the world live in human networks. The NSA is looking for networks that connect occasionally to suspected terrorists."
The 4th amendment is about particularizing government "searches and seizures" against individuals based on specific "probable cause". It is based on narrowing and controlling in what manner the government could be allowed to gather information from its citizens.
However, the congressional vote on the FISA Amendments in late 2012 and subsequent court decisions appear to have interpreted this dragnet type of data gathering and storage to be legal and constitutional. Thus removing the legal restraints on the use of a technological application, that is partially driven by an imperative to continue expanding as a basic feature of it's design.
As the Electronic Frontier Foundation observes, "National and international laws have yet to catch up with the evolving need for privacy that comes with new technology. Several governments have also chosen to use malware to engage in extra-legal spying or system sabotage for dissidents or non-citizens, all in the name of “national security.” "...National governments must put legal checks in place to prevent abuse of state powers, and international bodies need to consider how a changing technological environment shapes security agencies’ best practices."