The Temptations of Predictive Policing [Second Draft]

-- By MathewKenneally - 4 May 2015

The aspiration of online advertising is to offer something to a customer at the moment they are most likely to purchase it. A similar aspiration exists for law enforcement, to identify a criminal before they commit a crime: "predictive policing". We should be wary of granting law enforcement access to personal metadata. The trade-off may seem reasonable in the short term, but in the long term surveillance to identify “future criminals” could compound discriminatory policing, undermine privacy, and stifle political dissent.

For instance, Australia recently passed legislation requiring ISP’s to hold metadata of all users for two years. Prime Minister Abbott advocated for the legislation arguing that to not pass the bill would be requiring law enforcement to unilaterally disarm in the fight against terrorists and pedophiles. Warrants to access the data have been deemed unnecessary, because accessing metadata is not sufficiently intrusive. The opposition gained a concession to exclude browsing history from the data collected. Unsurprisingly, the law passed easily. The short-term trade off, an insignificant amount of privacy for security seems pretty good.

Contrary to the claims by government metadata is extraordinarily intrusive. Metadata is usually defined as data about a message, as opposed to the content. For example, it is the date, time, sender, location of the sender, and recipient of an e-mail, but not the text of the e-mail. Metadata can be used to track a person’s movements, reading, and contacts. As Justice Sotomayar observed merely tracking a person’s movement enables government to ascertain “their political and religious beliefs, sexual habits…”. Metadata can identify patterns in an individual’s behavior not apparent to the individual, revealing unconscious thoughts. This is why Edward Snowden said, "I would prefer to be looking at metadata than looking at content, because it's quicker and easier, and it doesn't lie".

Predictive policing is already a reality. The Chicago Police Department used recent arrest records and historic crime data to create a “heat list” of the 400 people in an area most likely to commit a violent crime. The police visited some on the list, to warn them that crime does not pay. The program relied on research that shows a correlation between an individual’s social network, and likelihood she will commit a violent crime.

Data analysis makes it easier to identify networks of likely criminals. Arrest someone for a violent offense; check her metadata; find her closest ties; you have your next offender. Using metadata a “heat list” can be assembled from the entire population without any arrest. It allows authorities to map “types of people”. Police could search for “groups” they believe are likely to commit an offense. For example, police may wish to identify all young Muslim men connected to a radical preacher.

Individuals, who have committed no offense, may find themselves subject to police scrutiny merely because they associate with or have similar interests, as offenders.

It could be argued that policing by data may avoid discrimination. Algorithms cannot be racists. Data generated from previous arrests and convictions may reflect previous biases. The interpretation of data can itself be biased. What we learn from data depends on what we ask in our analysis. The algorithm may provide a statistical justification for increasingly discriminatory enforcement.

There is a risk of criminalizing the “types of persons” put on the “heat list”. If these people are subject to more police surveillance, more offenses are likely to be detected. The data’s prophecy becomes self-fulfilling. Harcourt made this argument in his critique of the data supporting the effectiveness of broken windows policing. He suggests that misdemeanor arrests rose in “high crime areas”, not because there was a high level of disorder in those areas, but because there were more police placed in those areas on the look out for misdemeanor offenses.

Data can also aid predictive policing, by helping to identify how “criminals” think. Suppose a security agency compares the online behavior of those who perpetrated lone wolf terrorist attacks and identifies a pattern of sub-conscious behavior. A search of held metadata could identify all the “likely terrorists”.

This would allow police to treat thoughts as suspicious. It is a system that could lead an individual to be the subject of surveillance because she unconsciously thinks like a person who might do something wrong. Put another way “she googles like a criminal”.

A system established to identify terrorists and criminals could be used to suppress political dissent. Terrorists are, after all, political dissidents that turn violent. People likely to adopt politically subversive views could be identified through their reading and social networks before they even develop their beliefs. The police could investigate, or pay a visit to warn them that unorthodox ideas do not pay. Recently, in Baltimore, the media and law enforcement have sought to blame political activists for violence. Why would police, armed with tools to identify potential activists in advance, not seek to stop such protests before they occur? We should not expect them to "unilaterally disarm".

The response to these concerns is that current metadata collection systems contain safeguards. The Australian program excludes browsing history; the USA restricts metadata use to national security and foreign intelligence matters, not domestic law enforcement.

These limitations might be hard to maintain. The incremental requests to surrender “just a little more” privacy to prevent frightening crimes are appealing. Politicians are short-term creatures. After a terrorist attack, they want to ensure they cannot be blamed for the next event. Communities are easily frightened, and promises of security, however speculative, are hard to resist.

The short-term trade offs to enable predictive policing seem slight. Shrewdly, governments do not demand we surrender all our freedoms at once. The long-term questions are grave. Do we want to be policed based on our thoughts and associations? Can a state that preempts dissent be democratic? If these questions are not asked we risk constructing a dystopia by increments.