Category: Word Repetition in Fiction

It’s / its, there / their / they’re, your / you’re

I’m not going to insult your intelligence by suggesting that you don’t know when to use each of the above words. Of course you do. But your spell checker doesn’t know the difference between them. Regardless of which one appears on the page as you type your 120 words a minute, your word processor will not raise a red flag.

When you’re in the zone, when your creative side is buzzing away at speeds faster than your fingers can type, that’s when the most embarrassing typos put in an appearance. You’re not even looking at what you type, which is why you often end up with three lines in ALL CAPS before you realise you hit the caps lock key. Your fingers know the words, but they occasionally type the wrong one.

Maybe it’s a muscle memory thing, or maybe it’s because your fingers don’t have a brain. The reason doesn’t matter.

It’s the worst amateur mistake a writer can make. Scorn will be heaped upon you. Your fans will desert you. They’ll demand you return the Pulitzer, and the film rights will be back up for sale — with no takers. Yet we’ve all done it, and cringed and wept afterwards.

These words used to form part of the Possibly Misused Words list, but the blowback from writers was so severe that I removed them. (Who, me? Make a childish mistake like that? Never!) Yet you still need to watch out for them. Typos like these, it hardly bears thinking about.

Which is why they were added to the monitored words list by default. This particular check — available in both the free and licensed versions of SmartEdit — allows you to keep track of any word or phrase you wish, and to see each instance alongside the sentence it sits in. The sample below is from a blog post I wrote a few months back.

Misused Words List

There were no errors or typos in the post, but running it through SmartEdit allowed me to quickly check for just those high-school type mistakes (which you never make). Reading the sentence fragments only took a minute, and I could post my post reassured that whatever other grammatical or spelling mistakes I might have made, at least I hadn’t made one of those.

I’ll be running this post through the same check in a couple of minutes, so don’t bother trying to catch me out. Seriously though, you might never tell your friends, fans or writing partners that you check for this sort of thing in your work, but you’d be a fool not to.

If you’re running an earlier version of SmartEdit, these words may not appear in the Monitored Words List, but you can add them yourself in a few seconds, along with any other word of phrase you want to keep an eye on.

08 Apr 2014

Phrase and Word Repetition in your Novel

The earliest version of SmartEdit, way back in 2011, contained a small set of tools that counted words and phrases in a manuscript. Word repetition counters were not new — they had appeared in one form or another in various tools and utilities, the most common incarnation being the ever popular word cloud that showed up on the sidebars of blogs a few years ago.

The phrase counter was new — or if not completely new, it had not been incorporated into other software packages that I was aware of. Hardly surprising, as unlike the word counter, coming up with a list of unique phrases and the number of times they are used is not an easy task — not if you want it to work on a 150,000 word novel as well as a 500 word blog post.

Writing code to extract useful information from a snippet of prose is easy, which is why web based utilities pop up every few weeks doing just that. Extending that functionality to handle novel length works is not easy. Which is why these same web based offerings fall over at the first hint of an 80,000 word novel.

The word and phrase counters formed the core of SmartEdit, as it first appeared within PageFour and later as standalone, free software. They still exist in SmartEdit Lite, the free version of SmartEdit available to download from this website.

Instead of selling the benefits of the phrase counter to you, I’m gong to show you. As part of the early (and ongoing) testing of SmartEdit, a large number of published novels were run through the various checks, as were a number of fan-fiction novels.

My reasoning behind running the fan-fiction works through SmartEdit was that they were likely to be of less quality than published novels by established authors, and as such, would throw up a lot more issues for SmartEdit to catch. Basically, they would be a better representation of a first-draft of a novel than a work that had already suffered through an editor’s pen and many months of editing by the writer.

This proved to not always be the case. In some instances, published works were delivering shocking results, the kind of results that might mean the book should never have been published. I have little doubt that if the authors of some of these works had been no-name writers submitting a first novel, that that novel would have been rejected.

I won’t name the writer whose work was used when writing this blog post — I wouldn’t like to embarrass them — but will say that they are a New York Times best selling author, with double digit blockbusters under their belt.

The novel runs to about 150,000 words, not untypical for the genre, and the results are a fair representation of this writer’s most recent work. Here are some samples from the phrase counter list in SmartEdit:

Phrase #
for that matter 90
on the other hand 60
at the moment 43
shook his head 62
shook her head 39
in the first place 37
whether or not 37
after a moment 32

There’s a lot of head shaking going on in this novel, and probably amongst readers of the novel as well.

Phrase repetition is a tool often used deliberately by writers — Martin Luther King’s I Have a Dream speech being one of the most famous examples. But in the case of the sample data shown above, there is nothing deliberate going on. It’s simple laziness and lack of editing. The handful of phrases I pulled out represent only the tip of the iceberg for this writer — once the number of occurrences get lower, the number of repeated phrases rises dramatically.

The nine phrases shown above would have been problematic if they had occurred even ten times each. The actual figures demonstrate the lack of editing that can happen when a writer becomes so popular that they feel editing is no longer necessary.

If you think your writing is so polished and your editing abilities so strong that you wouldn’t benefit from seeing a list just like this, then I challenge you to send me a copy of that draft. Many writers are afraid of lists like this, feeling it detracts from the creative process, but how creative does that list above look to you? Is a novel full of these sorts of repeated phrases something you would be proud to put your name to?

You don’t need to buy the professional version of SmartEdit to avail of this particular feature. It’s available in the free, Lite version as well, though the user interface is not as powerful.

SmartEdit Lite — Free

SmartEdit — Licensed Version

22 Mar 2014

Word and Phrase Repetition — the Proximity Question

I can write an algorithm to do pretty much anything with a simple list of words–which, at its most basic level, is what a short story, novel, screenplay, blog post, article or essay is.

Count the number of times a word appears–no problem. Count the number of times each phrase appears–a little trickier, but still no problem. And I can see the use of these checks to writers. Are you relying a little too much on a particular word or phrase? Do you have a habit of beginning one sentence too many with a particular word or phrase? Has an unusual or rarely used word turned up four times in your 80,000 word novel?

This is what computers do. It’s what they’re for. Spotting patterns, counting things, making it possible to draw conclusions by presenting facts that are difficult for the human eye to spot.

But these checks are only as useful as they are useful. Adding complexity just because you can–not a good idea.

I’ve been asked a few times by SmartEdit users to improve the word and phrase frequency counters to show the proximity of repetitions. Other software does this, so why doesn’t SmartEdit?

It’s been suggested that using the phrase “On the other hand” 250 times in a novel might not, in itself, be a problem, not if those 250 occurrences are spread out evenly over the entire work. Not if one use of the phrase doesn’t pop up within a page of another.

I disagree. Using that particular phrase half a dozen times in a full length novel, not to mention 250 times, is a problem regardless of how close one occurrence is to another.

The proximity checker?

Unless there’s real value for the writer, there’s no gain in adding such complexity. Data for the sake of data, complexity without purpose. Rather than add value, it detracts–from the software and from the results. It makes the writer’s job of editing more difficult, not less; makes them question deliberate use of phrase and word repetition.

The power of this check is not in catching excessive word or phrase usage in a paragraph or on a single printed page, it’s in catching the minor repetitions of words and phrases that jar the reader when encountered more than once.

I remember reading a novel where the writer used the word opined once in every chapter. Fair enough, they were long chapters, and the writer may have felt comfortable with the distance between each opined. But for me, the reader, every time I encountered that dreadful dialog tag was a slap in the face. What was he thinking? Had I not been inflicted with enough opines to last a dozen life times? Can his characters not simply say something?

Why write about this today?

I’m reading Jeanette Winterson’s children’s novel Tanglewreck, and have just reached a passage describing a character who I’m guessing will prove to be one of the bad guys. This wonderful description of Abel Darkwater would be flagged in bright red flashing lights by any sort of word repetition proximity checker.

Abel Darkwater was a round man.

He had a round face, and a round body, and round rings on his round fingers. The gold loops of his pocket-watch chain were round, and when he drew out his watch, which he did as the taxi pulled up to the door, his watch was round and fat and gold.

But the repetition is deliberate–obviously. As is the phrase repetition in Dr. King’s I Have a Dream speech.

Repetition, when those words or phrases are in near proximity to each other, is not the purpose of the various repetition checks in SmartEdit. Their purpose is to catch those dreadful opines, scattered like acne over the length of a novel, forever popping at the wrong times and spewing all sorts of unpleasantness upon the reader.

So, for now, the word and phrase repetition counters in SmartEdit will remain as they are. Because even though the algorithm could be written without much difficulty, I’m not convinced it should be.

07 Aug 2013