Algorithmic Profiling and the Future of Individual Autonomy

-- By EricaPedersen - 11 Oct 2019

Defining Privacy for the Internet Age: Secrecy vs. Autonomy

Policymakers in the United States are finally beginning to acknowledge the tech industry’s exploitation of users through extensive monitoring and data collection. Unfortunately, the current policy debate is narrowly focused on enshrining individual rights to notice and consent with respect to the collection and sale of their personal data (see e.g., California Consumer Privacy Act). These ‘protections’ belie legislators’ superficial understanding of privacy as the right to secrecy, an individual’s right to hide information from certain actors. This framing will ultimately undercut society’s ability to address the significant threats that big data pose to individual freedom, autonomy, and self-determination.

Not really secrecy, because their attention is also directed at the personal information people voluntarily give to information processors, not the messages they intend to keep secret.

The fundamental importance of a right to privacy lies not in its facilitation of secrecy, but in its protection of individual autonomy and self-determination. In the seminal article considered to be the original source of American legal conceptions of a right to privacy, Warren and Brandeis emphasized that privacy “is in reality not the principle of private property, but that of an inviolate personality.”

But writing from 1896 probably doesn't fully deal with the current situation. And that's a piece about the tort of invading privacy, which is really distinct, is it not?

Preservation of individual autonomy is integral to both human freedom and functional democracy. Cambridge Analytica has proven the power of data analytics to conduct mass "psychological warfare" and manipulate human behavior to engineer a systemically destabilizing political outcome.

That's rather a strong version of what was "proved." Perhaps a less aggressive statement of what we have learned would still be sufficient?

Moreover, algorithmic profiling has increasingly emerged as a gatekeeper controlling individuals’ abilities to access to a wide array of choices and opportunities in the real world. For the most part, these processes are invisible to those whose freedoms they have restricted. Affected individuals typically have no right to notice, consent, explanation, nor any ability to effectively dispute the applicability, efficacy, or disparate impact of the methodology.

By framing the right to privacy as a right to secrecy, modern American lawmakers obscure and insulate the broader individual and societal harms inherent to current methods of monetizing personal data. This unfortunate fact is particularly obvious when one considers the effect that legislation like the CCPA would have on the thriving industry of predictive algorithms. Data collection is only one step in the development of the algorithms which now guide (or supplant) human decision-making in many areas of our lives. We cannot preserve individual autonomy without addressing the ways in which data are processed and the applications of these statistical inferences in the real world.

I don't think that explaining this in terms of over-protection of secrecy per se works very well, given that the statutes and your factual examples aren't about secrecy. In the latter portion of the paragraph, you appear to be arguing that a statute is faulty if it tries to solve one problem but doesn't solve another. That an attempted solution is partial is not evidence that it is faulty.

Potential Harms of Data Analytics and Algorithmic Profiling

Algorithms are an economically efficient way to analyze huge data sets and gain new insights based on complex statistical correlations. Algorithms are frequently used to infer or predict an individual’s personal preferences, interests, behavior, attitudes, movements, or health. The Working Party on the Protection of Individuals with Regard to the Processing of Personal Data defines profiling as the “automated processing of personal data for evaluating personal aspects, in particular to analyse or make predictions about individuals.” Although the GDPR attempts to restrict automated decision-making and decision-making based on profiling, the United States has been much slower to acknowledge such autonomy-compromising applications of technology.

"Algorithms" are computer programs. Running computer programs in order to draw inferences from data is thinking. Scientific thinking, in particular, cannot proceed in any other way in many fields. Regarding "algorithms" as an appropriate domain for regulation implies controlling thinking. Law usually controls behavior, rather than thought, for both deontological and pragmatic reasons, and many of them. It would be useful to clarify both what you are talking about, and what you propose to regulate.

The result is that privately developed algorithms are increasingly relied upon in decision-making across a broad array of industries in the United States, not just in securities trading and targeted advertising. Companies use algorithmic profiling to target and identify “ideal” job applicants, as well as to make training, compensation, promotion, and termination decisions about current employees (over whom management exercises increasingly broad and intrusive surveillance rights). Judges consider insights from predictive algorithms in sentencing decisions, despite the fact that these tools are trained on large data sets reflective of a systemically discriminatory criminal justice system. Housing, policing, the list goes on. Despite the potentially significant impact that these algorithms could have on our lives, their developers, empowered by trade secrets law, staunchly refuse to reveal the source code.

The Value of Transparency

Reframing the right to privacy in terms of individual autonomy rather than secrecy is a necessary step towards understanding how to effectively regulate the abuse of profiling in data analytics without unnecessarily impeding technological development and the innumerable benefits that could be derived from algorithmic insights. Individual autonomy and corporate accountability would be preserved far more effectively and sustainably through algorithmic transparency than through data secrecy.

What is the difference between "algorithmic transparency" and use of free software?

When privacy is conceptualized as a right to secrecy, personal data is viewed in property terms and the regulatory solution appears to lie in enhancing individuals’ abilities to restrict access to their personal information. Algorithms will continue to pervade decision-making because the promise of economic efficiency remains. However, assuming that individuals exercise this new right to exclude, these algorithms will be trained on data sets which are increasingly scant and biased. Nonetheless, profiles will be developed and used to make statistical inferences and predictions about any individual, regardless of whether that particular individual was able to prevent her own data from being collected and used to train the algorithm.

Alternatively, individual autonomy is strengthened by improving individuals’ access to information so that they can make informed decisions. Thus, regulations designed to protect individual autonomy should emphasize transparency and attempt to reduce informational asymmetries. Individuals should be notified when the choices and opportunities available to them may be impacted by algorithmic insights. Individuals should have a right to an explanation of how the algorithm functions, including the data on which it was trained, the “target variables” it is designed to identify, and the inferences drawn from the statistical correlations that the algorithm has found.

Transparency would facilitate refinement of algorithms to more accurately achieve their intended goals and to address biases overlooked by developers. Transparency would also provide a means of ensuring that algorithmic insights are not used in an arbitrary or determinative manner with respect to limiting individuals’ access to choices and opportunities simply because they fall on the wrong side of a statistical inference. If enhanced market efficiency is truly the goal of algorithmic decision-making, then the exacerbation of informational disparities will only lead us in the wrong direction.

The draft seems to me to be organized around showing that the autonomy component of privacy is important, and that contemporary legislation has mostly been instead about "secrecy." I think this is wrong, for reasons interlineated above. Making the draft stronger involves either strengthening that aspect of the analysis or replacing it with a different formulation. The subsequent argument in favor of "algorithmic transparency" is familiar, but technically a little less straightforward than it looks. I have asked some questions above the answers to which might strengthen this portion of the draft.

---

You are entitled to restrict access to your paper if you want to. But we all derive immense benefit from reading one another's work, and I hope you won't feel the need unless the subject matter is personal and its disclosure would be harmful or undesirable. To restrict access to your paper simply delete the "#" character on the next two lines:

• #Set ALLOWTOPICVIEW = TWikiAdminGroup, EricaPedersen
• #Set DENYTOPICVIEW = TWikiGuest

Note: TWiki has strict formatting rules for preference declarations. Make sure you preserve the three spaces, asterisk, and extra space at the beginning of these lines. If you wish to give access to any other users simply add them to the comma separated ALLOWTOPICVIEW list.