Consuetudium lectorum Mirum est notare. Eodem modo typi qui nunc nobis videntur parum clari fiant sollemnes in futurum? Assum Typi non habent claritatem insitam est usus legentis in iis. Claritatem Investigationes demonstraverunt lectores legere me lius quod ii legunt saepius Claritas est etiam. Nam liber tempor cum soluta. Est etiam processus dynamicus qui.

only to have it end up looking like this?

Consuetudium lectorum Mirum est notare.
Eodem modo typi qui nunc nobis videntur parum clari fiant sollemnes in futurum? Assum Typi non habent claritatem insitam est usus
legentis in iis.
Claritatem Investigationes demonstraverunt lectores legere me lius quod ii legunt saepius Claritas est etiam. Nam liber tempor cum soluta. Est etiam
processus dynamicus qui.

Most word processors have a command that lets you see invisible markers like spaces (usually represented as a faint dot “•”) and what are still quaintly called “carriage returns,” or “line feeds” (generally shown by the symbols ”¶” or “↵”).¹ If you turn that feature on, you will see that there are way too many such return symbols in the above text. It might seem like the solution would be to find and replace all those returns with spaces, but then you would have no paragraphs at all in your document. What you want to do is replace all the mid-paragraph returns, but leave those between paragraphs.

Using Regular Expressions (RegEx), as discussed in the last post in this series, what we would want to do is search for every return (or line feed) that is not followed or preceded by a return (or line feed). In addition–since some paragraphs are separated not by a blank line but by a tab or sequence of spaces at the start of the new paragraph–we want to look for those as well. I find the following search works pretty well for me: (?<=[^\r\n\t ][^\r\n])\R(?=[^\r\n][^\r\n\t ])  It is easy to find many patterns like this in online forums, as I did, saving you the trouble of having to re-invent the wheel.

If you prefer not to have to muddle around with code, there are a number of tools out there which can automate this kind of text cleanup for you. On macOS my favorite is the package of free WordService menu extensions from DEVONtechnologies. These are extensions that work with the built-in “Services” menu that pops up on macOS whenever you control-click on some selected text. The package offers a number of useful commands to do things like change the capitalization of the selected text (e.g. turn “THE APPLE” into “The Apple,” or “The apple,” etc.), reformat line breaks (or remove them altogether), and one that can give you useful statistics such as the word or character count of the selected text, etc.

Considering that WordService is free and does pretty much the same thing, you might not want to spend $45 for TextSoap, but if you already have a subscription to the Setapp bundle of macOS apps then TextSoap is included with your subscription. Another option is Textwell which works on both macOS and iOS and can do much more than just clean text. It has some built in tools, much like those offered in WordService, but (if you aren’t afraid of tweaking the JavaScript in the example code) you also can make your own actions. I really like that these can be synced between the desktop and iOS. Clean Text for iOS is even easier to use, but less customizable. Since I don’t use Windows, Linux, or Android, etc. I’ll leave it for others to recommend their favorite text cleanup tools for those platforms in the comments.

List of posts in this series

• Free Your Mind, the Text Will Follow (Working With Text 1)
• RegEx 101 (Working With Text 2)
• Text-laundering (Working With Text 3)
• Lazy PowerPoint (Working With Text 4)
• Roll Your Own QDA (Working With Text 5)

Kerim

P. Kerim Friedman is a professor in the Department of Ethnic Relations and Cultures at National Dong Hwa University in Taiwan. His research explores language revitalization efforts among indigenous Taiwanese, looking at the relationship between language ideology, indigeneity, and political economy. An ethnographic filmmaker, he co-produced the Jean Rouch award-winning documentary, ‘Please Don’t Beat Me, Sir!’ about a street theater troupe from one of India’s Denotified and Nomadic Tribes (DNTs).

kerim.oxus.net/

Source

What we will do is look for every instance in your text of two words in a row that are both capitalized and then reverse the order of those two words. While this scenario may seem somewhat far-fetched, the technique we will use to solve the problem, using Regular Expressions (RegEx), is one of the most useful things I ever learned in my life. I use RegEx several times a month to solve all kinds of problems. You may find it a little frustrating to use the first time, but once you get the hang of it, and discover some tools to help you get the code just right, it will become a regular part of your toolkit. Examples of things I use RegEx for include cleaning up extraneous line breaks in text copied from a PDF, extracting information (like email addresses) from a large text file, or combining cells imported from a database into a properly formatted document. The uses are endless, and once you know how to do it you might find yourself thinking “If I spend 10 min figuring out the correct RegEx for this I can save myself hours of work.”²

Let’s start by grabbing a sample text to use for the tutorial. I’ve copied the following from the intro to the latest issue of Cultural Anthropology, which is useful because it has a bunch of names, but also poses some unique challenges:

The year’s final issue of Cultural Anthropology features a new contribution to our Openings and Retrospectives section, an Openings collection on “Chemo-Ethnography.” Nicholas Shapiro and Eben Kirksey make the case for a critical engagement with modern chemistry in its political, economic, and affective valences, inviting us to take “chemo” seriously as that which can both cure and poison. In her essay, Michelle Murphy evaluates technoscientific practices that have come to materialize chemical exposure, indicating how infrastructures of chemical violence are cloaked even as the violent effects of exposure invite surveillance and pathologization of those living in hostile conditions. Elizabeth Povinelli’s piece inhabits a chemical burn from the inside, meditating on intoxication and exposure; fire and fog; invisibility, velocity, and the training of neural noticings.

Since we now live in the Canine Republic, “Nicholas Shapiro” needs to be written as “Shapiro Nicholas,” but “Cultural Anthropology” should not be re-written. Also, we will run into a problem with the fact that Elizabeth Povinelli’s name is written with the possessive “’s”. Let’s take these problems one by one.

First, we have to find the proper names in this paragraph. To do that I will start with the simplest definition of what a proper name should look like: a word that starts with a capital letter and is followed by a number of lower case letters and then a space. This is what that looks like in RegEx: [A-Z][a-z]+\s Let’s break that down:

• [A-Z] looks for any capital letters
• [a-z] looks for any lower case letters
• + extends the search for lowercase letters by saying we can have one or more matching results for the previous search parameter (i.e. “a” or “aa” or “aaaaaa”)
• \s looks for a space

This is what that gets us (click on any of the images in this post to go to the RegEx101 site where you can play around with the code to see how it changes the results):

As you can see, this search result catches too many fish in its net. We want to catch the big fish, but leave the small fish (like “The”) out. The easiest way to do this is to replicate the search so that we only find cases where two proper names appear in a row. Like this: [A-Z][a-z]+\s[A-Z][a-z]+\s

But now we have one catch that isn’t a fish: “Cultural Anthropology” and one fish that got away: “Elizabeth Povinelli.” We can easily tell the search to ignore names starting with “Cultural” by adding (?!Cultural) to the start of the search. But why didn’t Elizabeth Povinelli come through? It was excluded because of the possessive apostrophe, so we can add the apostrophe to the list of characters we search for in the second name. Now our search looks like this: (?!Cultural)[A-Z][a-z]+\s[A-Z][a-z’]+\s

So far, so good. We have caught all the fish we wanted, and only those fish. One problem with RegEx is that it is rare that you can find a search that will work for every single use case. That is why some websites might reject your email address as invalid even though it isn’t. (Actually, a good RegEx should really work with almost all email addresses, but some programmers are just lazy.) For this reason, it is good to write new search parameters for each search you do and to test them against the document you are working on before using them. RegEx is powerful, but that very power can wreak havoc on your text if you aren’t careful!

So far we have only solved the first part of the problem. Once we have found the proper names, we need to reverse their order. To do that we have to mark off the first word and the second word with parentheses in our search, as such: (?!Cultural)([A-Z][a-z]+)\s([A-Z][a-z’]+)\s Our search hasn’t changed, but we can now use those bracketed results in our replace command, which will look like this: \2 \1  That tells the program to write out the second search result, followed by a space and then the first search result, followed by another space. This is almost perfect, except for one small problem you may have already guessed:

The year’s final issue of Cultural Anthropology features a new contribution to our Openings and Retrospectives section, an Openings collection on “Chemo-Ethnography.” Shapiro Nicholas and Kirksey Eben make the case for a critical engagement with modern chemistry in its political, economic, and affective valences, inviting us to take “chemo” seriously as that which can both cure and poison. In her essay, Murphy Michelle evaluates technoscientific practices that have come to materialize chemical exposure, indicating how infrastructures of chemical violence are cloaked even as the violent effects of exposure invite surveillance and pathologization of those living in hostile conditions. Povinelli’s Elizabeth piece inhabits a chemical burn from the inside, meditating on intoxication and exposure; fire and fog; invisibility, velocity, and the training of neural noticings.

Unfortunately, we ended up with “Povinelli’s Elizabeth” instead of “Povinelli Elizabeth’s” which is what our canine overlords wanted. Rather than changing our original search and replace to avoid this problem, it is easier to follow the first search and replace operation with another one that will clean up these mistakes. In this case the search command would look like ([A-Z][a-z]+)’s\s([A-Z][a-z]+)\s  and the replace command would be \1 \2's . At this point you should be able to understand how this works. Take a moment to see if you can understand the code before moving on.

Obviously, it would never make sense to write code to fix a single instance in a document, it would be easier to do it by hand, but if you had a 20-page paper to work with, not to mention a book manuscript, you might appreciate knowing how to code the solution in RegEx. True, RegEx does require learning some specialized code which is hard to remember if you aren’t a computer programmer who uses this stuff every day, but fortunately, there are lots of tools out there which can make this easier for you. The RegEx101 website linked to all of the searches in this tutorial is my favorite, but there are many other resources out there as well, not all of which require you to actually learn RegEx for yourself. In my next post, I plan to talk about some more user-friendly options, but I think it was important to first understand the underlying principles before moving on. If you don’t understand how tools like RegEx “see” text it is hard to understand what some of these other tools are doing. Hopefully, even if I haven’t convinced you to learn RegEx for yourself, you have gotten an idea of how it works and what kinds of things it can do.

List of posts in this series

• Free Your Mind, the Text Will Follow (Working With Text 1)
• RegEx 101 (Working With Text 2)
• Text-laundering (Working With Text 3)
• Lazy PowerPoint (Working With Text 4)
• Roll Your Own QDA (Working With Text 5)

Kerim

P. Kerim Friedman is a professor in the Department of Ethnic Relations and Cultures at National Dong Hwa University in Taiwan. His research explores language revitalization efforts among indigenous Taiwanese, looking at the relationship between language ideology, indigeneity, and political economy. An ethnographic filmmaker, he co-produced the Jean Rouch award-winning documentary, ‘Please Don’t Beat Me, Sir!’ about a street theater troupe from one of India’s Denotified and Nomadic Tribes (DNTs).

kerim.oxus.net/

Source