Nerd-Author Fun v2: Text Analysis

I promised this week that my blog post would be about some of my C# coding, which also happens to dovetail in beautifully with my writing. I’ve taken my earlier work and begun the super-charging process. That being said: this is just the beginning. In the future I plan to make it available, far more powerful and with a few of the bugs ironed out.

The general premise behind the program is that it can load your story from a text file, and then allow you to analyse it. At the moment it is sans-UI – which means it doesn’t have pretty user windows, checkboxes and other controls. I’m calling it Text Analysis Command Line (TAC). As it’s a command line program you have to type commands in to operate it.

So what can it do?

Like any good program it contains help – typing ‘?’ will give you a list and basic description of the available commands; typing ‘<command> ?’ will give you detailed options on that particular command.

wordcount-1

When you see a pipe symbol ‘|’ it means or. Square brackets (‘[‘, ‘]’) mean optional.

Most commands can either display output on the screen or save the results to a file. If using a single greater-than symbol (‘>’) the file will be saved (unless it already exists). Using the double option ‘>>’ will save the file, overwriting it if necessary.

Below is a description of all of the currently available commands. The results are based on processing Vengeance Will Come, my scifi/fantasy adventure (available now):

wordcount. You can display the frequency of every word used. Earlier in the year I bought Scrivener (left). For the most part it’s a great program but I was disappointed there was no way to export (or even easily query) word count data. The image of TAC (right) shows a snippet of both the textual version (default) and the ‘basic mode’ (using -b option). The basic mode is valuable if opening the file in Excel to do pretty graphs.

wordcount can also provide wordcount-word lengththe number of words which begin with given letters (-f) or the length of words (-l).

Just in case you’re curious the longest word at 20 characters is ‘uncharacteristically’. The three 16’s are: ‘conspiratorially’, ‘incontrovertible’ and ‘responsibilities’.

For the purpose of completeness, I’ll briefly mention the data command. At the moment it’s limited, a means to interrogate the data. In order to do all of this (and future) processing I painstakingly categorise every character of text into a type. Using the -expseg option outputs this information.

datacmd-2
-expseg option

 

At this stage the only two other data commands are -sen (output sentence). For example outputting the sentence at segment 128 is:

At first light they invade my mind, besieging it to the point of exhaustion.

And -block (output block) at 128:

“I wish that I didn’t know the future; that I couldn’t see the prophecies unfold before me. At first light they invade my mind, besieging it to the point of exhaustion. Even in my fitful sleep they haunt me as wild animals stalk the scent of blood, turning what little rest I get into an extension of my waking nightmare. I cannot escape.

The find command is powerful and will be leveraged heavily in future updates.

find-1

Unsurprisingly, find locates the occurrences of a specified word. Importantly the before and after options allow displaying the word in a variable level of context (e.g. want to see 10 words preceding the word, or only 5?).

find can also locate every instance of a specified type of punctuation. Want to know how often I use exclamation marks? Typing ‘find -p’ brings up a list of punctuation options from which a selection can be made.

find-2

The answer is of course 45 (as displayed on the screenshot). However, now I know exactly where they are (and in what context).

find-3.PNG
The first use of ! occurs at the 660th character in my novel.

I’m a big believer in not over-using the exclamation mark, so a tool like this would let me easily see how often I’ve used it in a given book (and calculate the amount of text between each usage). More importantly, it can also let me track down when I’ve used a ” instead of a “ which seems to happen no matter how careful I am.

This brings me to the end of the tour of TAC v0.0.1, I hope you liked it.

Advertisements

Writing: The Passing of Time

sundial-3692590_640One of the tricks to master in writing is how to show the passage of time in a story. For example if Chapter 2 occurs two weeks after Chapter 1, how do you show that? It’s one area I’m still honing in my writing.

I first highlighted these lines in book 11 of the Wheel of Time series:

“…the sun more than halfway to the horizon, by the time he saw what he was looking for.” (Page 151)

“In a morning ritual, his fingers made another knot mechanically, then slid down the cord, counting. Twenty-two knots. Twenty-two mornings since Faile was kidnapped.” (Page 156)

The first obviously makes use of the position of the sun, and the second describes in a clever action-oriented manner the time that has passed. I decided to exclude these two quotes from my upcoming highlights post, because I wanted to examine the topic further.

Of course the crudest way to show time is simply to tell the reader “10 days later…” I’ll admit earlier drafts of Vengeance Will Come had this a lot. There were two reasons for this: I didn’t know any better  and I was also using the prompts to aid my own keeping track of time. It is a crude approach which pulls the reader out of the story. There are still a few instances of itin the book, but it’s something I use rarely now.

A better way, as the old adage goes, is to show the reader instead of telling them. The goal, I think, should be to show the passage of time through the setting and/or character.

Here are some of the ideas I brainstormed. If you have any other ideas, please add them in a comment below.timescale

 

 

Vengeance Will Come (Very Soon)

My goal was to finish the revision of my first fantasy-adventure novel, Vengeance Will Come by the end of September. I am very close to finishing (95%).

There are 4 more scenes to revise and a few ‘TODO’ items I’ve listed for follow-up. So the end is just around the corner. I could have finished it and achieved my goal, if I’d been willing to be sadistic with myself.

As much as I love writing, and do it as a hobby, it is still tough work. Many break-in writers share how they work two jobs – a day job to pay the bills and then their writing, which they treat like a second job. It does feel that way. You’re always under pressure to produce and to perform. If you’re not writing, you’re thinking about writing. Which can be tiring in my experience.

The older I get the more I realise that life is too short. I decided I wasn’t going to make myself unhappy to achieve a self-imposed deadline. I would have some leisure time on the weekend – to read, to cook and to continue getting the garden under control.

That being the case, I believe I should be finished in approximately a week.

Fighting Paralysis

I’ve hit the difficult spot in my story.

I believe I know what is wrong with it, but I am not quite sure how to fix it yet. And the change seems so big that it results in a fear-like response. It is too big a problem to fix, my brain says, shying away from the task. The worry expands and grows: have I changed that character’s motivations earlier in my revision? What is the timing of the different scenes, and can I fit them together?

It’s an irrational fear. I know I can work through the problems, however it feels like I’m at the base of Mt Everest and have one gigantic, massive mountain to climb.

I can’t let paralysis win. I can’t let it chase me away or stop me dead. I need to choose the fight response (and not flight or freeze).

How goes the Revision?

A quick writing update on the revision of my first novel, Vengeance Will Come. It was my goal to finish the revision by the end of this month and I’m currently sitting at 70% complete. I’ve been making a few structural tweaks and expanding it out a little, as well as improving the language.

As an example, I just came across this:

The exclamations of surprise and dismay reached their climax.

Now that I re-read that line I am embarrassed by it. It’s talking about shock in a way which would put people to sleep. (And considering I am very sleepy, it’s not helping).

I always felt there was a problem at the 5/6th’s point of the story, where I leap-frog forward in time. More than one beta reader was surprised that the ending came quite so quickly. This suggests to me a stunted story. It was my intention to soften this leap by writing new content – whether that be some decent-length scenes or even a couple of chapters.  However, I’m also aware that I don’t want to write a chapter if it’s only ‘padding’. That would be a bad move. The goal of revision – whether it’s expanding, contracting or completely renovating should be to improve the story. This means I’ll need a good chunk of time to think through and write the content so that it is valuable and can be blended in to the story. At this stage I have no idea how I’m going to do it.

Fortunately, after discovering I haven’t had a meaningful holiday since March 2016, I’m taking a week off. Expect a writing surge. (Or feel free to shame me if there’s not).

And I’m thinking of releasing my novel for free…

 

A Race to the End

I’ve got work-related activities that I need to focus on coming up. In order to focus my attention, I want to finish my revision of Vengeance Will Come by the end of September. That’s possibly an unachievable goal.

Given that I suspect I need a new chapter at the 9/10ths part of the story, I need to leave a good chunk of time for writing that. So it’s a race to the end and I hope you’ll be seeing some massive jumps in the progress bar to the right.

The progress starts now. Go!

For Your Consideration: Publishers Need to Adapt in the Digital Age

I’ve recently been re-listening to the Writing Excuses podcasts and found them as enjoyable as ever (including LOL moments every few episodes). In this blog post I want to drill-in slightly to Season 1, Episode 20 and then pivot from that onto my own thoughts, especially about how I think publishers need to continue to evolve.

Setting the Scene

(Bare in mind that this podcast is now a decade old. Predictions made about the future of the industry may be well-along to either fruition or stunted after the passing of time, fruit unrealised).

In this episode Brandon asks a question of audience member, author Mike Stackpole who had this to say (slightly paraphrased, mostly verbatim):

“It’s only been the last 70-80 years we’ve been churning out literature, prior to that, we were entertainers. Now with the coming of e-books we can sell direct to endusers and cut out the publishers. … The problem for traditional publishing is they only own the electronic rights on manuscripts since about 1996 and since now were in 2008 they’ve got 12 years’ worth of novels and no writer needs to sell electronic versions to them anymore because we can sell direct. They’re in trouble.

The big problem for traditional publishers is their business has never been about bringing us fiction. It has been about them bringing to us blocks of wood. Their whole business is based around printing, shipping and warehousing blocks of wood. Their not that vital any more. We can find editors, copy editors, etc ourselves.”

To emphasise his point he says he can make greater profit by selling a short story from his website for $2 than he makes selling 2x $8 paperbacks. Additionally he gets the money instantly, not 9-15 months later as he would with a publisher.

As it relates to the Author

It’s pretty hard to argue with the real-world example given. There’s more profit in it for the author to sell direct-to-consumer, rather than via a publisher.

And there’s certainly truth to the point that the writer’s efforts are the main commodity being sold by a publisher. That’s not to say that the publisher doesn’t add value. The publisher brings professionals who have expertise and more experience than most authors ever will in their chosen field: editors, marketers, illustrators etc.

As an example, there’s a reason why most authors aren’t consulted about what their front cover look like. Publisher’s know what will sell, and also have a better understanding of what’s in the marketplace at the present time.

Publishers also bring credibility. Until you’re an established author you’re trading on the brand of your publisher.

However, the publishing world is changing. It is now easier for the author to editors, line editors and research assistants online. Websites like 99 designs exist to let you choose between competing artists for book cover designs. Sites like Patreon make it easier for creative people to be funded by those who love their creativity.

Personally, I don’t want to have to make all of those decisions. I prefer to think of myself as focussed on the writing, not lazy. Just like I could change the oil on my car, but I’m OK with paying someone else to do it.

Perhaps I’ve been reading too many fantasy books lately but I wonder in an internet full of people offering their services if we will see a return to guilds? Where groups of similarly skilled people will form to enhance their own credibility and exposure to potential customers.

For the Publisher

I am not unbiased in this discussion. Not too long ago I submitted a manuscript sample to a US publisher via the postal system as prescribed. And it disappeared.

Thinking about this podcast and the changing nature of technology it made me wonder: why would a publisher ever want to receive a manuscript by mail? The only reason I can possibly think of, is that it lowers the volume of submissions. What if embracing technology could actually make the submission process easier and cheaper?

What I’d suggest is that publishers should provide an electronic submission process. They could even charge an administration/submission fee (which would cover the costs of printing at their end, if printing is still necessary?). There are numerous benefits for the publisher:

  • An additional revenue stream (submission fee)
  • Automated-checking to see if the document complies with the submission guidelines. If it doesn’t you can reject it outright. (If you’re nice you can let the author know; if you’re so inclined you can reject it ‘quietly’ and keep their submission fee without much guilt. They, after all, didn’t follow the rules, so be it on their own head).
  • Use machine-learning to automatically check and redirect to the recycle bin. (Too many spelling errors? Rejected. Tense confusion? Rejected. Swear-word tolerance breached? Rejected. Foul content? Rejected).
  • Author-filtering. Get a submission from someone who is truly terrible at writing? Auto-reject anything they submit for the next year. (Or if reject is too strong, though I doubt it, at least put them toward the bottom of the slush pile).
  • Implement a “submission alpha reader” process where unpaid minions can up-or-down vote manuscript samples.
  • Environmentally friendly: less printing.
  • Author-history. Hit gold with an excellent author? Want to see everything else they’ve submitted before?
  • Faster submission process. When you’re already waiting for months, it’d be nice to streamline any part of the process as an author.

Am I a genius or diabolically evil?