How I Write Interview Instruments – #RoR2018

How I Write Interview Instruments – #RoR2018

It would be interesting to sit down and look at the interview instruments from every year that I’ve been doing research in Senegal to see how they evolve. From 2012, my junior year in college, we would find leading questions or questions that otherwise confine respondents to certain answers. Some questions just didn’t make sense. There were probably no probes. In 2013, my advisor worked closely with me to make sure that I was phrasing things more clearly and in ways that were more likely to invite conversation. By 2015 and 2016, I was working in a variety of probes and finding ways to make interviews feel more informal and relaxed. Instruments from 2012 were a rollercoaster of general and specific questions, while newer instruments always follow a flow from general demographic and life history questions to topics of discussion that increase in specificity like a funnel.

Tony Hawk on the downward loop
One of the more interesting results from a Google search for “funnel technique.”

In July and August 2017, I had the great privilege of participating in the NSF-sponsored Health Equity Alliance of Tallahassee (HEAT) ethnographic methods field school. For six weeks, we worked closely with local leaders and activists to design and carry out a research project that met certain needs in the community, while working together in a course on research design, field methods (with a special emphasis on community-based participatory research), analysis, and data management. Writing interview questions has been largely self-taught with some guidance from mentors, but this past summer I learned a more concrete method for instrument writing, one that holds me accountable for avoiding confirmation bias, leading questions, and respondent fatigue.

Justify Yourself
In my last blog post, I wrote that proposal writing should link expected data to the methods and analysis, the methods and analysis back to the research questions, and those questions back to the research objectives. Here, I link the interview questions in the same way by forcing myself – in writing – to justify asking the question. In the HEAT field school, we called this the “Intents” list.

Ron Swanson asks "What's your point?"
Pictured: NSF reviewer.

Open a spreadsheet. Label your columns from left to right: Interview Question, Purpose, Research Objective, and Notes. Each row is dedicated to reflecting on each interview question.

You can come up with questions in one of two ways: I look to my research objectives to think of specific inquiries that might help me achieve those objectives. I might also think of a question – inspired by related publications, my own fieldnotes, or focus group interviews – and try to reverse engineer its linkage to a research objective.

For example, my first research objective is “To determine the extent to which there is a gendered division of knowledge about prenatal care.”

I would like to ask, “How do people take care of pregnant women?” This will go in the Question column.

What is the purpose of asking this question? It asks respondents to identify particular practices associated with a cultural domain (prenatal care). It hinges on a respondent’s understanding of “care” and what it means to provide that care to another person; it narrows the provision of care to pregnant women; it leaves the provider of care (“people”) open to interpretation so that I can ask later, “Do men do this? Do women do this?” It provides an opportunity to flesh out the things that people might do which they might not classify as care, but are nonetheless unique in some way.

I know that “Objective 1” belongs in the third column, though I might also link it to a more specific Research Question from my proposal.

Under Notes, I might remind myself that this question comes from a previous study, or that it leads me to another related question, or I can note the kinds of probes I should use. Demographic and introductory questions like, “Tell me about yourself, where you grew up, and how long you’ve lived here,” do not necessarily need to be linked to an Objective or have Notes, but it’s still a good practice to justify the purpose of asking.

My Secret Weapon: The Pregnancy Journal
I also have a sort of unique way of developing interview questions for my project: I bought a pregnancy journal. The journal invites expectant mothers to write about their experiences while also guiding them through what is “normal” or what they can expect during pregnancy. The journal is (of course) aimed at women, rather than men. I begin with the first question from the journal – “How did you find out you were pregnant?” – and I envision myself talking to a Senegalese man. “How did she find out she was pregnant?” but also, “How, and when, did you find out she was pregnant?” I add “when” because I know that men are often the last to find out. The next question, “How did you feel when you found out?” translates for men – I can ask the same thing. Next, “How did you tell your partner?” in a Senegalese man’s context might be rendered, “Who told you?” (because it isn’t always the expectant mother) and “How did they tell you?” And so on. The journal is also aimed at Americans, not Senegalese, and it therefore forces me to reflect on determining the questions that are more meaningful for Senegalese research participants. The book has a section on “Gear” and asks expectant mothers what gifts they want to receive at their baby shower. Related questions would prove to be inappropriate in Dakar where baby showers are non-existent (and potentially dangerous affairs that would draw the attention of evil spirits to a pregnancy). On the other hand, seven days after the birth of a child, they are baptized and given a name, and the family does receive gifts. So, while the journal provides a handy way to plan a baby shower – “Who is invited? What will we eat? What is expected?” – I can easily translate the guide into the context of a Senegalese baptism.

Respondent Fatigue and Following Up
The pregnancy journal has generated dozens and dozens of important questions – on top of those I already had – which I am currently working to justify and link to research objectives. (Actually, I’m writing this blog post as a means of procrastinating from writing my interview instruments.) It’s a good thing that I’ll be doing follow-up interviews with a core group of expectant parents every two weeks, otherwise a single interview with all of these questions would take many hours. You’ve got to give yourself and your participants a break – at a certain point, you’ll get annoyed, they’ll get annoyed, answers become shorter, probing stops working, and it’s a mess. I find 90-minute interviews are the sweet spot, but that might depend on the topic of conversation.

Boo (Monsters Inc.) blinks to stay awake.
When research participants start looking like this, it’s time to bring it to a close.

What this means is that I’m actually writing multiple instruments – not just for different samples of different stakeholders, but also for multiple interview session with the same participants. It works out, because I can organize these interviews around themes that arise during pregnancy. For instance, I might write an entire interview about prenatal screening and clinical care, or diet and morning sickness, or how and why one hides pregnancy, or how the baptism is planned. And if these interviews don’t pan out (as they often don’t in Dakar), writing the instruments should guide my attention to these themes as a participant-observer.

Once I have a list of viable and sufficiently justified questions, I work on reordering them so that they flow from general to specific and so that the question topics follow from one to the next thematically. We want respondents to be primed and already thinking about the next question before we ask it. In fact, I think of this way: At the very least, I should always be able to write “This question primes respondents for the next question” in the Purpose column. Finally, before I ever sit down for an interview, I go through the instrument with close friends in Dakar. These dry-runs are critical, as my peers can help me fine-tune the wording; sometimes questions don’t translate conceptually, sometimes there is room to ask delicate questions more carefully, and sometimes my French just comes off like a Parisian textbook.

Eddie Izzard: "Mais, la souris est en dessous la table..."
…le chat est sur la chaise, et le singe est sur la branche!

Thanks for reading. I’m trying to have these out every Monday morning (so far so good) and next week I think I might write about ethical review. Someone did ask me to write about what I’m taking to Dakar, but I think I’ll save that post for after I get there. If you missed my first post, go back and read it to catch up on what my research is about and why I’m writing this series. Also, just to follow-up on that post: I HAVE BEEN APPROVED by both the ethical review board at the Senegalese Ministry of Health and by Fulbright-Hays. My ticket is purchased, and I’ll be in Dakar the first week of January.

2 Replies to “How I Write Interview Instruments – #RoR2018”

  1. Dick, what you are doing sounds great, something from another galaxy far from the “Go forth and find something interesting” of the minimal methods training I received at Cornell in the late 1960s. Allow me to ask, however, where are the questions you ask yourself, e.g., about observed changes in behavior as people answer your questions. I assume that some questions will elicit standard “pat” answers, others will stimulate interviewees to think beyond their habitual preconceptions, some may make them uncomfortable, and how they respond to that discomfort can be informative. I recall Vic Turner’s teaching that ethnographers deal with three kinds of data, their own observations, the native exegesis, i.e., what people say when asked “What does that mean? Or what is that for?”, and other information, bits of theory and data that we bring to the field as our intellectual baggage. Vic was adamant that none of these three kinds of data could provide a definitive explanation, not least because they frequently contradict each other. The ethnographer’s job, he said, was to create a coherent picture in which all three played a part. This is the context in which I read your account of your protocols as sophisticated attempts to improve the quality of the native exegesis you hear, an important goal in itself. But I wonder where observation of behavior suggesting unusual interest or discomfort, in particular, come in.

    1. Hi John. Thanks for your thoughts. I appreciate that. I know exactly what you’re talking about and I did consider writing a little on that bit, but I thought it best to just stick to writing about the instrument itself. Those kinds of reflections about interest or discomfort, for example, do get filtered out in a way when one writes an instrument this way. For instance, when I was writing out questions from the Pregnancy Journal, I found myself wondering, “Are lower and working class Senegalese couples interested in tracking the fetal development of their yet-born-child?” This is a question I could ask explicitly, but I’m pretty sure I know that the answer. (I’ll probably ask early on in focus group interviews just to confirm, though.) I will probably end up leaving this question of the instruments, but that doesn’t mean leaving it out of the research – I just have to move it over to the participant-observation realm and keep my eyes and ears open for it. Another kind of question that I didn’t get into in the article are those in which we ask for a lot of information at once. I might start with the demographic/life history question, “Tell me about yourself, where you grew up, and how long you’ve been in Dakar?” On the instrument itself, I would list three check boxes for probes beneath the question: 1. Where, 2. Duration in Dakar, 3. Social ties (the people in your life). These are things I’m keeping an ear out for, but if they don’t mention them, I can ask. I suppose that researchers might also include a space to write about the respondent’s emotional state or body language, right? I haven’t done that, though I do write about those things when they become salient.