Adventures in chatGPT #2: A conversation with Nick Seaver

Adventures in chatGPT #2: A conversation with Nick Seaver

After writing my last post about chatGPT, I got in touch with Nick Seaver to see what he had to say about some of these issues. Here’s our conversation:

Ryan Anderson: So, I just signed up for chatGPT and started messing around with it, and I’m concerned. Are you concerned?

“Nick Seaver”: Yes, I am concerned about chatGPT. It is a powerful tool and can be used for malicious purposes. It is important to be aware of the potential risks and to use it responsibly.

Nick Seaver: Sorry, that was a computer-generated answer from GPT-3; it’s sort of obligatory to start this kind of thing with some fake-out computer text. But yes, there’s a lot of concern out there among academics (not to mention other people) that easy access to a technology like this is going to upend a lot of our usual ways of doing things. At the very least, I’d recommend that any instructor assigning short writing work see what chatGPT can do with their prompts—you might be surprised just how passable the results are.

“Ryan Anderson”: Ah, the infamous computer-generated response. As anthropologists, we can’t help but chuckle at the irony of using technology to discuss the potential consequences of technology. But let’s not dismiss the concern surrounding the use of advanced language models like GPT-3. While it’s true that these models can produce impressive writing, we must also consider the potential implications of relying too heavily on them. For example, if students use GPT-3 to generate their assignments, are they really learning how to write effectively? And what about the ethical concerns of using AI to replace human labor in the writing industry? As anthropologists, it’s our job to study and understand the impact of new technologies on society. So, let’s embrace the novelty of GPT-3 while also approaching it with a critical eye. And who knows, maybe we’ll even learn something along the way.

Ryan: Sorry, I wasn’t sure if it was obligatory to keep the gag going even after it’s not funny anymore. But look at that response. It’s not bad. Maybe we can just farm this interview out to the machines, Nick. We can just walk away and let our chatGPT doppelgangers handle this while we work on some urgent bureaucratic tasks. But seriously–and to your point–these things are actually pretty good, even if they hallucinate every now and then. It does seem like our usual way of doing things is at risk. But it may not be such a bad thing if we have to rethink stuff like how we use discussion posts on Canvas, right? I mean if what we’re asking for can be easily completed by AI, then what are we measuring? And I say this as someone who just ran some prompts through chatGPT and is kind of freaking out about what this means for my discussion post assignments on Canvas. What do you think? Are some folks overreacting or are we in like literally in that oh shit moment in a SciFi movie where the main characters realize it’s already too late? 

Nick: Okay, so like usual with pedagogy things, there’s a lot of moralizing around this question. Why bother asking students questions that an AI can answer? I disagree with this take in general. There are lots of reasons why it might be useful for students in an anthropology class to practice answering kind of basic questions: do we expect absolute novices in the field to immediately get to work on completely unique things? When designing my own assignments, I try to layer them so that some are for practicing with basic concepts, while others ask for extrapolations that chatGPT might have a harder time with. It’s undeniable that this technology poses an issue for some forms of that basic concept work, though of course it was always possible for unscrupulous students to buy papers, copy online texts, etc. At the end of the day, I’m not interested in becoming a police officer; I don’t like the idea of students using GPT on these practice assignments, but it only hurts their own learning if they do. I explain that in class, and I trust that my students will take it to heart. 

I’m not really persuaded by the idea that it’s important for students to learn how to prompt these systems in class. But one thing I’ve done with GPT3 already (and it seems to be a popular professor gimmick, so maybe students will get sick of it soon) is to provide a simplistic short answer prompt (like “Explain Marcel Mauss’s theory of the gift”) to the system, and then ask students to critique, correct, and expand on it. That has worked okay, but, to be honest, GPT often gives such passable answers that I found myself needing to re-run it until it made a big mistake. (I like to have a range of subtle to less-subtle mistakes for students to catch.)

Ryan: That’s a good point about the whole ‘Why bother asking students questions that an AI can answer?’ question. I’ve heard various versions of that argument lately and I’m not quite sure what I think. In some ways, I do think there are things I need to reconsider–how I write some questions, what I expect, how I grade them, etc. But there’s a difference between questioning the value of the assignments we’re giving out, on the one hand, and the implications and problems that we face with students using something like GPT3 to complete those assignments. In a broader sense, this problem is nothing new, as you point out (students buying papers, etc). I hear you about not wanting to have to be some kind of police officer with this stuff. And I totally agree with the ‘it only hurts their own learning if they do’ sentiment. I haven’t seen anything coming in from my classes–yet–that seems like it was made with chatGPT. But then again I think it’s pretty likely that I have already missed stuff. I’ve seen examples from other colleagues this year though, and while some of it is relatively easy to spot, some is also very good. I don’t think I would catch it, honestly. Usually when it comes to this sort of thing, like the whole discussion about plagiarism, I try to talk through it with my classes. Bring it out into the open. That seems to help. But I’m feeling a little behind the ball when it comes to chatGPT. I’ve heard of others doing the chatGPT prompt thing in classes with students, and I can see how that might be valuable up until a certain point. What else do you do, or what else do you recommend, for thinking through this stuff? 

Nick: I really appreciate your point about how people have entangled the question of “what issues does GPT pose” with “what assignments are worthwhile”—of course there’s a long history of entangling the question of machine capacities with the value and definition of humanistic pursuits, but we don’t need to take those questions together uncritically. Just because a computer can spit out a plausible short answer to a discussion question, that doesn’t mean it’s not worth thinking through. The question of what to do about it will vary tremendously based on the setting: not all universities are the same, and students have different kinds of pressures, reasons for being in our classes, and familiarity with technology that will affect how this plays out. So I wouldn’t want to presume that what I try to do in my classroom, which is a very privileged space relative to the broader field, would make sense everywhere. 

One issue with algorithmic technologies (all technologies?) more generally, as critics like Virginia Eubanks have pointed out, is that their effects are experienced unevenly across social status: poor people are more likely to be subjected to algorithmic decision making without the possibility of appeal than rich people. So I worry that, in the current frenzy about what to do about this stuff, some institutions will start to implement half-baked “GPT detectors,” like we’ve already seen popping up online, and those will be applied unevenly and unfairly to student work which is now seen with a new level of suspicion. I’ve been playing with various generations of this technology for a few years now, and I can say that it is very hard to reliably identify; it’s quite good at mimicking the tone of a novice trying to sound more certain than they feel.

To some extent, it seems like the solution to the problem of GPT is to be found in closer interaction between instructors and students, so that we don’t end up in detective situations where instructors with little familiarity with a particular student’s thought and writing find themselves puzzling through a text worrying whether a computer wrote it. Smaller class sizes and more instructors would seem to resolve a lot of the angst here.

Ryan: Ya, I think your last point–about smaller class sizes and more instructors–speaks to some of the bigger underlying issues. Those are often the issues that many universities don’t want to address, and in many cases the situation is getting worse as class sizes keep growing and growing. The alienation just keeps expanding. In many ways, things are going in the opposite direction, and this is where these tools come in. Classes get bigger and bigger, there’s more and more distance between teachers and students, and along come all these calls for apps that will supposedly bridge that gap. 

This reminds me of emails that I get from app developers trying to pitch all their tools for my classes. One of the latest was an AI-based tool that would assess and grade students’ online discussion posts. I mean, if that’s where things are going in terms of grading and assessment, why wouldn’t students start turning to things like chatGPT? But I think the problem is much broader than what we’re seeing in classrooms. It’s affecting research. I’m running a survey right now and, unlike a similar survey I ran just a couple years ago, it got hammered by bots and spam. This is apparently a new reality we’ll be dealing with for the foreseeable future. Then there’s the question of how this will impact academic writing and publishing more broadly. It goes beyond academia, of course: the popular science fiction outlet Clarkesworld Magazine got a lot of media coverage after it abruptly closed its submissions due to a flood of machine-generated submissions. There’s all the AI-generated content on YouTube, algorithms that shape so much of the information we interact with, and so on. 

It seems to me that many people, like teachers who are seeing this stuff in classrooms, just don’t quite seem to know how to respond. But I think we’re in a moment where the question of how we respond matters. It does seem that many are heading toward the kind of problematic and ‘half-baked’ reactions that you mention above. Those kinds of responses are pretty common here in the heart of Silicon Valley. As you also pointed out, that’s just setting the stage for all kinds of uneven impacts. So for me this goes way beyond the issue of whether or not the college essay is supposedly dead. In academia, the problem is on our doorstep now, so we’re finally taking notice. What’s your take on the bigger picture with these tools? As an anthropologist who has put in a lot of time looking into these issues, what ways ahead do you see?

Nick: Honestly, I find it all quite bewildering still and have sort of retreated to my ethnographic fieldwork mode. I’m curious to see what kinds of critique gain traction, what unanticipated consequences pop up, and what (if any) forms of regulation manage to stick. Not to be grandiose, but it does seem likely that the spread of these systems will change our relationship with reading; think, for instance, of the “bot” accusation people throw around on social media, which is some mixture of “I think this person is trolling,” “I think this account is literally automated,” and “I think these posts are not worthwhile.” 

I’d expect that knowing anything we read might have some kind of algorithmic language generation behind it will change how we read, what we read for, and how various markers of fluency and sophistication are interpreted, both in and out of academia. Instructors who assign written work are already familiar with various kinds of “reading for” in our assessments—does this assignment indicate a grasp of concepts, even if the execution is dodgy; does the tone of this sentence suggest that maybe it was copied from elsewhere? It will be interesting to see how the experience of reading changes when we do it in a world of language generating systems that scramble our earlier assumptions about the relationship between style, tone, competence, and substance.

Ryan: Ya, it’s definitely bewildering. That’s why I asked you to do this interview–because I was feeling somewhat lost. Ha. Now what? It will be interesting to see what kinds of critiques and responses get traction. Plus all the unintended consequences. It’s hard to guess how some of this plays out. I like your point that this may change how we read. I think one of the fears is that we just won’t be able to tell the difference between ‘real’ writing and the output from something like chatGPT. I’m seeing some folks who seem to think this is just the end of academia, writing, and online writing assignments. But maybe something else comes out of this–a renewed critical awareness? We will see. Thank you, Nick, for taking the time to have this conversation with me.

***

*HAL9000 image adapted from Wikimedia Commons here.

One Reply to “Adventures in chatGPT #2: A conversation with Nick Seaver”

  1. I think confronting the issue head on is important. A unit about AI and chat bots explaining their actual purpose perhaps? They are an algorithmic that scans available online data and tried to respond in a way that sounds like a human according to the prompts you communicate. It is not actually generative. It cannot come up with something new. It does not reference. It will not attribute theory to sources correctly.
    If you use it and are caught, it will be plagiarism. And if your tutor does their job there is a good chance of being caught.
    But let’s be anthropological about this and focus on 1. What can we learn about education and communication now we have programs that can mimic that? And 2. Why would a student fe they need to use an AI? A lot of people struggle badly with assignments, especially overlapping time commitments between units.
    Perhaps there is potential here as a learning tool for students who have trouble expanding on a c concept. How does chat gtp do it? After all, it is just a parrot spewing forth reconstructed information from the receptacle of our knowledge and values. Just some musings