Phrase and Word Repetition in your Novel

The earliest version of SmartEdit, way back in 2011, contained a small set of tools that counted words and phrases in a manuscript. Word repetition counters were not new — they had appeared in one form or another in various tools and utilities, the most common incarnation being the ever popular word cloud that showed up on the sidebars of blogs a few years ago.

The phrase counter was new — or if not completely new, it had not been incorporated into other software packages that I was aware of. Hardly surprising, as unlike the word counter, coming up with a list of unique phrases and the number of times they are used is not an easy task — not if you want it to work on a 150,000 word novel as well as a 500 word blog post.

Writing code to extract useful information from a snippet of prose is easy, which is why web based utilities pop up every few weeks doing just that. Extending that functionality to handle novel length works is not easy. Which is why these same web based offerings fall over at the first hint of an 80,000 word novel.

The word and phrase counters formed the core of SmartEdit, as it first appeared within PageFour and later as standalone, free software. They still exist in SmartEdit Lite, the free version of SmartEdit available to download from this website.

Instead of selling the benefits of the phrase counter to you, I’m gong to show you. As part of the early (and ongoing) testing of SmartEdit, a large number of published novels were run through the various checks, as were a number of fan-fiction novels.

My reasoning behind running the fan-fiction works through SmartEdit was that they were likely to be of less quality than published novels by established authors, and as such, would throw up a lot more issues for SmartEdit to catch. Basically, they would be a better representation of a first-draft of a novel than a work that had already suffered through an editor’s pen and many months of editing by the writer.

This proved to not always be the case. In some instances, published works were delivering shocking results, the kind of results that might mean the book should never have been published. I have little doubt that if the authors of some of these works had been no-name writers submitting a first novel, that that novel would have been rejected.

I won’t name the writer whose work was used when writing this blog post — I wouldn’t like to embarrass them — but will say that they are a New York Times best selling author, with double digit blockbusters under their belt.

The novel runs to about 150,000 words, not untypical for the genre, and the results are a fair representation of this writer’s most recent work. Here are some samples from the phrase counter list in SmartEdit:

Phrase #
for that matter 90
on the other hand 60
at the moment 43
shook his head 62
shook her head 39
in the first place 37
whether or not 37
after a moment 32

There’s a lot of head shaking going on in this novel, and probably amongst readers of the novel as well.

Phrase repetition is a tool often used deliberately by writers — Martin Luther King’s I Have a Dream speech being one of the most famous examples. But in the case of the sample data shown above, there is nothing deliberate going on. It’s simple laziness and lack of editing. The handful of phrases I pulled out represent only the tip of the iceberg for this writer — once the number of occurrences get lower, the number of repeated phrases rises dramatically.

The nine phrases shown above would have been problematic if they had occurred even ten times each. The actual figures demonstrate the lack of editing that can happen when a writer becomes so popular that they feel editing is no longer necessary.

If you think your writing is so polished and your editing abilities so strong that you wouldn’t benefit from seeing a list just like this, then I challenge you to send me a copy of that draft. Many writers are afraid of lists like this, feeling it detracts from the creative process, but how creative does that list above look to you? Is a novel full of these sorts of repeated phrases something you would be proud to put your name to?

You don’t need to buy the professional version of SmartEdit to avail of this particular feature. It’s available in the free, Lite version as well, though the user interface is not as powerful.

SmartEdit Lite — Free

SmartEdit — Licensed Version

22 Mar 2014