Interview by Ilana Gershon

https://mitpress.mit.edu/books/constitution-algorithms

Ilana Gershon: This book is about the formation processes of algorithms.  What, for you, was the sociological work involved in tracing how particular algorithms come to be, and why choose this focus over the effects of algorithms in people’s lives?

Florian Jaton: When I started to be interested in the topic of algorithms in 2013, there was already a substantial, and generally quite critical, literature on the social effects of algorithms. These important works studied the ways in which algorithms acted on our lives, while also emphasizing algorithms’ opacity. This sociological movement – whose peak might have been reached with the publication of Frank Pasquale’s masterful book The Black Box Society (2015) – was very important in making algorithms matters of public concern and in generating political affects. But already by 2013, it seemed – to me at least – that continuing with this systematic denunciation of the power of algorithms could be counterproductive. I was especially concerned that algorithms were mostly considered abstract and floating entities made of distant code and obscure mathematics. How indeed to act against abstract, floating, and obscure entities? The fact that the critical depiction of algorithms did not give them much empirical thickness made, in my opinion, a much-needed collective contestation difficult to carry out.

But I then realized the problem was mainly methodological: If algorithms looked disembodied, it was because they were most often considered from afar; from the offices of critical sociologists who observed them through their “official” accounts (reports, software, academic papers) purified of the fragile scaffolding that had previously contributed to their progressive shaping. This is where my training in science and technology studies(STS) was useful, since one of their basic postulates is to consider techno-scientific devices as the products of situated and accountable practices. A possible remedy to the disarming critical discourse on algorithms seemed then to lie in a drastic change of method, privileging the anthropology of science as framed, for example, by the in situ ethnographic works of Bruno Latour, Michael Lynch and Lucy Suchman, over distant document analysis (that yet remains important). In sum, to suggest new levers of action to better contest the powerful effects of algorithms, it seemed important to document, from the inside, some of the causes of these powerful effects.

A surprising element was that this problematization resonated with the concerns of computer scientists who worked in a polytechnical school close to my university. Some of them were quite aligned with my analysis on the limits of the then-current social debates on algorithms and saw the possibility to show the daily work of algorithmic design as an opportunity to present themselves more realistically. This fortunate convergence, which did not prevent me from having critical views, allowed me to stay for two years, as a participant observer, in a computer science laboratory (that I call the “Lab”) specialized in the construction of image processing algorithms.

Ilana Gershon: What analytical translational work were you required to do when using approaches developed to understand laboratory practices and using them to illuminate the work of computer programmers and mathematicians?

Florian Jaton: This inquiry was intended to be immersive, in the continuity of the ethnographic “lab studies” that participated in the initial development of STS. And as required by this specific analytical genre, I had to become, at minimum, competent in digital image processing, if only to speak adequately about issues that mattered to my informants and colleagues. Very quickly then, it appeared essential to follow an accelerated training in analysis and linear algebra (the main mathematics involved in signal processing, along with statistics) in order to be able to express myself in acceptable terms.

This minimal theoretical equipment coupled with the classic apparatus of the ethnographer (mainly notebooks, cameras, and audio recorders) allowed me to follow the daily work of the Lab for more than two years, conscientiously writing down observations and recording work sequences. And it is probably the long-time of the investigation – made possible by a doctoral fellowship – and a certain systematics in the note-taking that has made it possible to account for some of the practical courses of action – what I call “activities” – that participate, I believe, in the development of new algorithms, considered as cultural products.

Ilana Gershon: What methodological quandaries did you have to overcome to study computer programmers?  What did you do?

Florian Jaton: To design and publish new algorithms, my colleagues at the Lab had to write symbols on numbered lists, an activity they called – not surprisingly – computer programming (or coding). Although they did not, by far, only do this – they also spent time designing databases, studying and discussing mathematical formulas, attending seminars and conferences, taking coffee breaks, and so on ­– programming was nonetheless an important part of their daily algorithmic design work. And as an ethnographer of this specific work, it was therefore important for me to accurately document these programming situations.

The problem that quickly arose was how to do it. Since the lines of code written during these situations were quite cryptic (I did not have a rigorous computer programming background at the time), it was extremely difficult to get a handle on what was going on. Also, these situations appeared to be quite engaging for the people involved, which often prevented me from interrupting them to ask questions about what they were doing. In short, during these moments that could be particularly intense, I was clearly out of place.

After discussing this methodological impasse during a weekly Lab meeting, it was collectively decided that I should attempt to design a small image-processing project that would echo other algorithmic research that was under development within the Lab. The purpose of this exercise was to force me to learn the basics of several programming languages as well as to try to apply some theoretical elements acquired during my accelerated training in signal processing (see question 2). But most importantly, this modest project also included a helping clause that allowed me to ask for assistance from Lab members whenever I was stuck in a programming impasse. This ad hoc method first allowed me to become more familiar with several programming languages, which was a prerequisite for the close analysis of their mobilization in situ. But it also made the Lab members more comfortable during the programming situations I was trying to document. Since the project was collectively designed and could, potentially, be used for future algorithms and papers, the Lab members found it relatively relevant. Finally, and most importantly, this method allowed me to better equip and document programming situations: in addition to keeping notes describing the actions of the computer scientist programming next to me, I could also record my computer monitors as well as the discussions taking place. And at the end of the eight helping sessions I needed during this small project, I obtained handwritten descriptions, video and audio recording that I could then thoroughly analyze (pending a huge transcription work).

Ilana Gershon: You point out that for mathematics to be relevant to our daily world, a cascade of translations have to occur to connect mathematical knowledge to actants people interact with regularly.  Which translations do you think are most relevant to track to understand the consequences of the coded algorithms people might encounter during any given week?

Florian Jaton: It is important to keep in mind that what algorithms generate, and what they work upon, are digital data signals that require a very specific ecology in order to circulate and, eventually, produce differences. In this sense, many translations have to be done to, for example, render commensurable the statistics of, say, a Tweet (for example, its number of likes or retweets) and the relevance of its content. We may invest meaning in algorithmic products only by loosely assuming that this post circulates like all the other posts of all the other profiles and that the (small) world constituted by all these posts and profiles is a world that has value by and for itself; it is by building this whole chain of translations – while knowing, sometimes, their incongruity. In short, many consequences of coded algorithms require an active – and anthropologically fascinating – work of putting disparate elements (desires, small screens, icons, mobile computing infrastructures, communication protocols, and so on) into equivalence. And my feeling is that we tend to get used to these small translation efforts we make, to the point that we often don’t pay attention to them anymore.

Ilana Gershon: In what sense is this an insurgent book for you?

Florian Jaton: I guess the book is insurgent, or at least politically engaged, in two ways. First, it challenges the traditional conception of algorithms, as defended for example in theoretical computer science, and which presents algorithms as computerized procedures that transform inputs into outputs to solve problems in effective ways. By accounting for the concrete processes by which new algorithms come to be constructed, the inquiry shows that the material work of problematization – which includes, among other things, the creation of referential databases called ground truths – is an integral part of algorithmic development. Roughly put, we get the algorithms of our problematizations, that themselves rely upon material elements upon which one can act.

Also, the book is an attempt to get out of the disarming critical discourse asserting that algorithms are powerful because they are abstract and inscrutable. By proposing thick description of algorithmic design activities, along with theoretical and methodological tools to conduct, maybe, further ethnographic inquiries, the book intends to show that algorithms are very material devices, designed in specific places on which it is possible to impact.