“Proving” the language/culture connection

“Proving” the language/culture connection

Over the weekend, several anthropologist called attention to this research report produced by Princeton University (link to full report here). The headline touts the research with the claim that “Machine Learning reveals role of culture in shaping the meaning of words”. My response, and that of many others, was immediately snarky – we didn’t particularly need computers to tell us something that has been amply demonstrated by the entire field of linguistic anthropology for the better part of a century, and by plenty of people paying attention for even longer. There was a bit of pushback on these comments, which ultimately all share a certain thematic element – that even if we already knew this, we, as linguistic anthropologists, should welcome this work, and the attention being paid to it, as a new methodology that supports what we know and do.

The problem with this claim is…it doesn’t do that at all. And here, I have to own up to the fact that my own initial flippant response absolutely does suggest that it does, as I noted “the machines have caught up to my opening lecture in intro to linguistic anthropology”. It is, of course, true that culture shapes meaning within languages, and that we teach that as a central principle of the discipline. The problem is, what the authors of this study mean by that and what we mean by that are fundamentally different things, as becomes apparent when you read beyond the headline. At a certain point, I hoped that reading the paper itself would mitigate some of the concerns I had, but alas, while obviously written in a somewhat less hyperbolic way, the conceptual foundation, methodological application, and interpretation involved in this paper is, to my mind, a frustratingly flawed contribution to the study of the intersection of language and culture, for reasons outlined below.

The crucial issue for me is how the authors define ‘culture’ and establish a quantified version of ‘cultural similarity’. In order to make this machine-based analysis work, culture has to be reduced to a checklist of features. To do so, the authors did in fact draw on anthropology – specifically, the Ethnographic Atlas available at D-PLACE, which is based on the work of GP Murdock and his students. There’s an interesting anthropological rabbit hole to go down in examining the disagreements between Murdock and Edward Sapir, and critically considering Murdock’s emphatically ‘scientific’ and mathematical approach to studying human social differences. What I would ask the authors in this case, though, is whether they have chosen this approach to studying culture after a careful consideration of historical and contemporary thinking about the concept, or mainly because it is the one that allows them to fit the question of culture into the computational mold they wish to explore. Even the assumption that “languages” map neatly onto “cultures”, as opposed to containing multiple ways of speaking, or ‘languages’ being spoken by diverse groups of people, or to having culture defined by multilingual and multivocal practices, doesn’t hold within contemporary linguistic anthropology.

Further, and relatedly, while the Princeton report about the study touts it as covering a remarkable number of languages, 41 is in fact an absolutely tiny drop in the bucket of global linguistic diversity — a point that becomes even more apparent when you look at the actual list of languages, which include 25 from the Indo-European family, 4 Turkic languages, 3 Uralic, and 1 each from the Afro-Asiatic (Arabic), Sino-Tibetan (Chinese), Dravidian (Tamil), Kartvelian (Georgian), Japonic (Japanese), and Koreanic (Korean) families, as well as Basque. While I was pleasantly surprised at a few of these inclusions (Georgian and Basque wouldn’t fall in to the ‘usual suspects’ list), most of the list is extremely predictably narrow. Further, one might ask whether these labels even hold up all that well – which Englishes are represented here, or which versions of Spanish, Chinese, or any other “language”? This narrowness is made even worse as the analysis selects further and further for focus on Indo-European languages, because those are the ones about which the kind of diachronic language change information being used to classify degrees of linguistic/historical similarity is most available. The authors don’t justify this choice beyond the convenience level – or really, at all. Even to find the list of languages, one has to follow the links to get to the 300 pages of supplementary material that they provide. This indicates to me that they don’t think their choice of languages used to make conclusions about ‘universal’ meanings and patterns of language/culture relationships requires explanation. A broader consideration of language at a global level would require attuning to the complexity of the concept of ‘words’, to the ways in which meaning is established in practice, or to the implications of things like polysynthesis in how these forms of ‘universality’ emerge.

To illustrate what I mean, consider how the study talks about kinship terms and alignment. For the authors, the machine analysis demonstrates that this category of terms (at least the most ‘common’ ones – the examples they give are ‘daughter’, ‘son’, and ‘aunt’) tend to translate into other languages with a high degree of shared meaning. But ethnographic analyses of kinship practices would suggest that even if the terms ‘translate’, they are used in extremely diverse ways. In many parts of Latin America, the Spanish/Portuguese terms ‘tia’ and ‘tio’, which translate as ‘aunt’ and ‘uncle’ are used to refer to almost any adult engaging with children, so during fieldwork in Brazil, I would often be introduced to kids by adults saying something like “Essa tia vem do Canadá” (“This auntie comes from Canada”). Sticking with languages represented on the list here, Susan Blum’s work on “Naming Practices and the Power of Words in China” is one that I have assigned to introductory ling anth classes to talk about how many cultural beliefs we take for granted, such as the role of names and kinship terms, are in fact demonstrably diverse. Blum’s work is a good example that illustrates how “meaning” is not reducible to semantic “content” or “translatability”, but rather has to be understood in terms of social practice. In other words, even asking the question of “what does this kinship term mean?” requires us to understand how a given culture approaches such “meaning”.

This starts to get at what I mean when I say that what this work ‘proves’ does not, in fact, align (pun intended, #sorrynotsorry) with what linguistic anthropologists talk about when they study how meaning is different across cultural contexts. There are major assumptions in the computational work that contradict the understandings of language and culture that most of us work within, and in particular, ignore the ways in which we examine language as a dynamic social practice. The ethnological Atlas material is, of course, not the only criterion the study uses for identifying cultural proximity, but digging in to other aspects of the analysis reveals similar assumptions. As my friend Lavanya Murali noted to me, the treatment of geographic proximity and shared linguistic history, for example, doesn’t really contend with the dynamics of how people interact across linguistic boundaries such that similarities can be produced through interaction, rather than as an inherent property of language — with both these elements, in turn, abstracted from an idea of “culture”. All of this, for me, calls the conceptual framework that this research relies upon into question, and at the very least, demonstrates that this work doesn’t support linguistic anthropologists’ claims about language and culture. As such, this is not a matter of saying the same thing with different methodological evidence, but rather saying something completely different based on an entirely distinct set of assumptions about language and culture – ones that, in fact, I work really hard to teach students to examine as ideological claims rather than fundamental truths. This even presents something of a meta-commentary, as it’s worth noting that meaning doesn’t even align within languages, and that the meaning of ‘meaning’ isn’t always clear and translatable — I could go on, but you get the point.

In addition to all this, I want to ask — why this research? Why ask these questions? This has been a central piece of the critique I have brought to my less-sarcastic Twitter comments, and that still holds after reading the study itself. The researcher interviewed makes the claim that this is the first “data driven” approach to the question, and further explains that the motivation comes from a desire to improve upon the time-consuming need to do things like “conduct long, careful interviews with bilingual speakers who evaluate the quality fo translations”. The first comment is illustrative of a widespread belief that ethnography is not data, and that valorizes the quantitative and mathematical as “proof”. As many people noted, one of the reasons this raises our hackles is that we have been “proving” the interrelationships between language and culture in any number of ways for years, and this work actually doesn’t engage with any of that material, preferring instead to jump back several decades and use a dataset that conforms to pre-existing assumptions. The second point is more nuanced, but equally worth addressing – what’s wrong with long, careful interviews? In fact, one of the reasons that the list of languages used here is so limited is because those are the ones for which a sufficient amount of long, careful interviews, recorded material, and myriad other forms of data are available. It’s not clear to me, then, that this kind of work in any way does away with the need to develop that material in the first place, raising the question of what it accomplishes. As I noted in tweets, the decisions about what questions to ask are ones that deserve scrutiny, because resources are spent investigating these questions, which means those resources aren’t available for other questions. And if resources are being consumed doing research that ignores and dismisses work on apparently related topics, it does have a negative impact on that work – so, speaking for myself, as a linguistic anthropologist, it’s disappointing and frustrating to see not only the promotional elements of this work, but to see how the project itself represents the questions that we even need to understand regarding language and culture.