I thought about going to the workshop but decided against it. Partly it is an issue of taking time off from writing, but I also have mixed feelings about how the first DARPA project went. Mainly I was disappointed at how little useful output we produced for the amount of taxpayer money we spent. I had hoped to do more. This was not necessarily anyone's fault, just politics; but still, I'd like to find ways to spend my time that have a stronger positive impact if I can.
Still, listening to stories in order to "scan the horizon" and discover emerging threats and opportunities is as good an idea now as it was then, for governments and for everyone else too. I wish the new group the best in their laudable goals. To do my little bit to help, I thought I'd send some hopefully helpful advice to the program manager of the project. But then I thought, why not make it a blog post? That way the advice, for what it's worth, can help other people with similar goals too.
The workshop had three stated goals (this is from its description):
- To survey narrative theories. These empirically informed theories should tell us something about the nature of stories: what is a story? What are its moving parts? Is there a list of necessary and sufficient conditions it takes for a stimulus to be considered a story instead of something else? Does the structure and function of stories vary considerably across cultural contexts or is there a universal theory of story?
- To better understand the role of narrative in security contexts. What role do stories play in influencing political violence and to what extent? What function do narratives serve in the process of political radicalization and how do they influence a person or group’s choice of means (such as violence) to achieve political ends? How do stories influence bystanders’ response to conflict? Is it possible to measure how attitudes salient to security issues are shaped by stories?
- To survey the state of the art in narrative analysis and decomposition tools. How can we take stories and make them quantitatively analyzable in a rigorous, transparent and repeatable fashion? What analytic approaches or tools best establish a framework for the scientific study of the psychological and neurobiological impact of stories on people? Are particular approaches or tools better than others for understanding how stories propagate in a system so as to influence behavior?
Surveying narrative theories
My own survey of narrative theories resulted in my understanding of three dimensions of story: form, function and phenomenon. Pulling in an explanation from Working with Stories:
Story form is the internal structure of a story: things like setting, characters, plot and point. A good story uses effective narrative form to deliver a message well.
Story function is its utility to our thinking and learning: things like meaning, understanding and connection. A good story helps us learn what we need to learn, find out what we need to know, or remember what we need to remember.
Story phenomenon is the story of the story: things that describe context, like where and when and why a story was told, who heard it, how it can and will be retold, and so on. A good story lives on because it sustains the health of the community.So the answers to the questions "what is a story" and "what are its moving parts" can take any of three forms, all of which are correct in context. The moving parts of story form are the tools of the fiction craft: arcs, beats, plot twists. The moving parts of story function are the bones of narrative knowlege management systems: practices, discoveries, lateral thinking paths. With story phenomenon it is not so much that stories have moving parts but that they are the moving parts of society: rumors, urban legends, fantasies.
Does the structure and function of stories vary considerably across cultural contexts or is there a universal theory of story? Yes. Both are true. Some elements of story are undoubtedly universal, but there are also differences in the way people from different backgrounds tell and listen to and trade and pass on stories. Oral storytelling traditions, with their memorable repetitions, still impact the way some cultures tell stories, but less so others. That is probably just one difference of many. Others might be the interplay of objective and subjective truths; the importance of place in storytelling; the expectations on an audience, to keep quiet or help build the story; expectations about self-expression versus contemplation of collective issues; what sorts of stories are taboo and expected. I'm not aware of any exhaustive research on this topic, but my own experience in having read tens of thousands of personal stories told by people from different cultures (as well as thousands of folk tales) tells me there are differences worth exploring.
When looking for authoritative definitions in narrative topics, it is important to realize that dozens of fields of inquiry touch on narrative in some way. Many of these fields have different definitions, assumptions and goals, and many of them interact little with other fields. If you want to understand the landscape of human knowledge about narrative, it is worth learning a little about every single one of those fields. Here is a partial list of phrases worth googling:
- primarily story form
- narratology - Meike Bal's book Narratology is good
- professional fiction writing
- professional screenwriting - of course Robert McKee's Story is the authority here
- professional live storytelling
- primarily story function
- narrative in knowledge management
- scenario planning
- case-based reasoning
- knowledge representation / information retrieval
- primarily story phenomenon
- folklore study
- oral history
- narrative in cultural anthropology
- narrative community therapy - for example the Dulwich Centre
- narrative in community activism - for example the Theatre of the Oppressed
- mostly story form and function, less so phenomenon
- literary theory
- narrative analysis
- mostly story function and phenomenon, less so story form
- narrative inquiry (participatory and otherwise)
- organizational storytelling / business narrative
- narrative medicine
- narrative therapy / counseling
- narrative psychology
- narrative journalism
- narrative in law - some universities have started programs in this area; see the book Minding the Law
- narrative policy analysis - see Policy Paradox and other books
- narrative in foreign policy - see Thinking in Time and other books
Understanding the role of narrative in security contexts
This section is essentially about tying the research fields related to story phenomenon to psychology, sociology and political science. If I wanted to address this I'd convene people from all of these backgrounds and ask them to work together on a comprehensive literature review and exploration of fruitful ways forward. I don't know that there is anybody working in this particular junction, but I could be ill-informed on that point.
Making stories quantitatively analyzable
This set of questions is both the one I have the most direct experience with and the one I think my readers will most want to hear about. So I'll write much more about it, but I'll broaden the situation to make it more generally applicable.
Let's pretend that you have collected or discovered some stories, say hundreds or thousands, and you want to do more than simply read them. You want to find useful patterns in them that will help you make decisions and detect emerging trends. Participatory narrative inquiry is impossible because the original storytellers are inaccessible for one of any number of reasons. What options are available? Normally the projects I work on rely heavily on storyteller interpretation, for reasons I have written about elsewhere. But I have done some projects where I supplemented storyteller interpretations with analyses of my own, as well as some projects where only stories were available. The results are not as strong, but they can still be real and useful.
Before I start I should say that there is a large and complex literature on narrative methods of inquiry in the social sciences. There is much you can read there if you want to explore those topics. However I must admit that I tend to get in a huff every time I read a textbook on methods of narrative inquiry. The universal assumption that expert interpretation of stories told by others is tantamount to proof bothers me very much. Examples of stories interpreted by experts that entirely ignore the obvious fact that any other expert, as well as any other person, might give a different or even opposite intepretation, abound. I find the "I know what this means" mindset to be very similar to the "I am the world" mindset of some artificial intelligence practitioners, about which I have written previously.
It seems - to me - like some researchers who use and promote narrative analysis see expert interpretation as something that adds value, while I see it as something that removes value and should be used only as a tool of last resort. In my experience, applying expert interpretation to a story is like using a chain saw to harvest tomatoes. You will get something out of it, but it may not have much use in practical terms.
So, when I think about what options you have if you have stories but no interpretations, the options form into groups of increasing difficulty and decreasing utility.
Option group one: Asking proxies
If you have stories and you can't ask their tellers what they mean, you might be able to ask people close to the tellers what they mean. This could be people related to them in some way - their grandchildren maybe - or people similar to them in some way. For example, say you have a thousand anonymous stories told by customers of your product. Would it be better to consider the interpretations of (a) an outside expert, (b) your CEO, or (c) other customers? Of course the ideal is all three, because the more interpretations the better your view. But if you could collect only one set of interpretations I would suggest asking your other customers.
Sometimes you can't find anyone close to the original storytellers. Perhaps they are isolated or long dead. Perhaps nobody who is related to them or like them will talk to you - they are activists against your policies, for example. In this case you need to move on to the next group of options.
Option group two: Examining storytelling events
The second way you can work with stories without intepretations is to watch storytelling - the narrative event - to look for interpretations storytellers have embedded in their stories. People include meta-narrative content (essentially, a story about their story) in their stories all the time, and it's not that hard to find if you know what to look for. There are even some stock phrases people pull out when they want to communicate their story's importance or truth or authority or unimportance or triviality or entertainment quality. For example:
- I learned a lot!
- When I told Joe he was surprised about this...
- I never saw anything like this before!
- I shouldn't be telling you this but...
- Have you ever heard anything like this before?
- I will never forget that day.
- Listen, I happen to know that ....
- This is just my experience but ...
- Maybe you heard something different, but...
- This just beat all!
- I laughed and laughed!
- Can you believe this?
People also communicate information about why they are telling their story by the intensity of the words they use to describe their feelings. In one part of the story they might say "I didn't like that much" and in another part they might say "I was devastated." People turn the emotional intensity of their stories up and down as they speak in order to communicate which parts of the story they most want others to hear. I've always thought it would be great to have what I call a "ranked thesaurus," where words are ranked by their emotional intensity, value, complexity and other meta-information they provide. If you had such a ranking you would be able to chart the emotional rollercoaster of a story by the word choices made. The peaks and troughs of the ride would tell you things about the storyteller's intent.
If you have audio or video records of storytelling events, you can go much further into watching storytelling. You can watch prosody, or the rhythm and intonation of our speech, which is another way we add meta-information to our stories. If you have video you can watch body language. If there are others on the recordings you can watch audience reactions and the way the stories change as a result of the interchange.
Another way to watch storytelling is to watch the pattern of story back-and-forthing. You can do this in any situation where stories are told in response to other stories: in a chat session, in a room, over the phone. The way stories invoke other stories can provide information you can use to understand the intent of storytellers. There are many other nuances in the negotiations that take place as people trade stories - the field of conversational analysis is related where it touches on narrative.
Option group three: Analyzing texts
Studying exactly what was said in a story is still outside the realm of expert interpretation because anyone can agree on the words that were used. But it is weak in its ability to detect true intent, which puts it lower in the value scale.
Concordances are lists of non-trivial words in story texts - houses, dogs, weapons, iPods, babies - the things you see in those "wordle" pictures on the web. When a person mentions something in a story, it means that thing matters in some way, and that can mean something across many stories. For example, say you have three hundred stories that mention babies, two hundred that mention weapons, and fifty that mention both. Concordance methods can also include words placed close together - did they say "baby" within five words of "weapon" and so on. I don't think there is that much value in concordance all by itself, just because there are so many reasons people might mention the same word. It's a weak indicator of meaning. But if concordance can be combined with other information that is more emotionally meaningful, such as prosody or proxy evaluations, it can increase in value through juxtaposition. If stories that included the word "friendship" were more likely to have been seen by same-community proxies to have been told "to defend a position," that is a pattern you can examine.
Evaluation statements are phrases where people express a value they set on something. There are not that many ways to say whether something is good or bad, and you can search for and collate these statements. The simplest thing is to look for obvious value words like "good" and "best" and so on. That gives you the roughest approximation. Next you can look for other ways of expressing value, like "I liked that" or "that's fine" or "I was happy." I don't think you can automate this entirely, because finding the value statements doesn't easily show you what was being valued. But a system that highlights possible value statements and helps people enter data about the thing referred to are still a step up. I haven't done this, but I can imagine annotating a set of stories with metadata like "government - bad" and "family - good" and seeing what comes of it.
Statements of fact are another useful thing anyone can see and mark in stories. Statements of fact are not the same as actual facts, but they can be telling nonetheless, especially when they disagree or are clearly wrong. Rumors are especially full of stated facts - everyone knows the government was behind that - so they may be especially worth mapping in projects where public opinion is being considered. These are not hard to find; you just look for places where people claim that something is true or untrue, like "nobody reads newspapers anymore" or "people annotate their stories with meta-narrative."
To be honest, though, I haven't spent much time on textual analysis of stories. I've done it here and there to try out ideas, but have never been very happy with the results. The output is full of false positives, dead ends and weak trends. If it has to stand alone I wouldn't trust it to tell me anything important.
Way back in IBM in 1999 I did a little research project where I put a variety of batches of text through one of the data mining tools IBM was selling at the time. Some of the texts were non-narrative listings of facts, some were conversational, some were newspaper articles, and some were short stories, movie scripts and folk tales. I tested tools for clustering, summmarization and feature extraction. All of these tools did very well on fact lists and news items, but very poorly on the stories. The problem was that the stories contained so many nuances and subtle variations of word use that the software was befuddled. It's hard to parse a story because it depends on a structural form that operates at a higher level than word use. The crux of a story may appear to a textual analysis system as a part where nothing important is happening.
Here's a bit from Bleak House, which I've just been reading:
"You may bring the letters," says my Lady, "if you choose."No computer would understand that her ladyship has just taken a dramatic step in her last (seemingly unimportant) statement by embedding two meanings in the word "please" - one stilted and the other beseeching. She has crossed a threshold into a space in which she and Mr. Guppy are aware of the secret the letters contain, and she asks him to help her without forcing her to admit anything out loud. He takes the hint and moves on. A concordance that simply throws that "please" up onto a shelf with all the other pleases in the novel would be useless to understand this scene. Many stories are like this, and not just novels but the anecdotes we tell every day.
"Your ladyship is not very encouraging, upon my word and honour," says Mr. Guppy, a little injured.
"You may bring the letters," she repeats in the same tone, "if you --please."
"It shall be done. I wish your ladyship good day."
I did bring away two conclusions from trying to use automated textual analysis to look at stories. First, nearly all textual analysis systems that exist to date concentrate their efforts on nouns. In stories the nouns are not as important as the verbs, followed by the adverbs. If I was put in a box and forced to build an automated textual analysis system for stories, I'd go after those. (When I wasn't trying to break out of the box, that is.)
My second observation about textual analysis of stories is that I think to some extent people do use textual cues in reading stories, but we can't articulate what it is we are doing. If a computer were to watch a group of people interpreting stories, they might find out things we hadn't noticed we are doing. You could even do things with those devices they have now where you can track what words people are looking at and how long their eyes linger. An automated system trained in this way might find unanticipated ways to do what we do naturally.
But again, the closer your proxies can get to the storytellers the better you will fare in such an enterprise. An algorithm trained on the actions of people who share much context in common with the storyteller will derive different methods than an algorithm based on watching people from a different group. I've watched myself read stories, and I do read them differently if I know I share a lot in common with the storyteller than if I know I don't. Have you ever caught yourself reading three lines into an article, then going back up to the top to check the name of the writer for clues to their similarity to yourself? Probably few people can avoid taking such context-detecting actions, and they surely impact the way we read things.
Option group four: Asking story experts
This group of options crosses the line into expert interpretation, but it keeps the expertise in the realm of narrative and away from the subject matter of the stories themselves. When I have few answers from storytellers or when the answers present a garbled picture for some reason, I develop an emergent set of story subjects or gists. These are shorthands for story plots, like "I did my part but somebody else didn't" or "We help each other out" or "I faced a difficult challenge and succeeded."
The way I do this is based loosely on grounded theory and generally takes the form of three passes through the stories, thus.
- First I write up one or more gists for each story, not caring much about reuse. If a particular story cries out for a gist I have already written, I will copy it and paste it in, but I don't force conformity.
- The second time through I read the stories again and compare the gists for each to my overall list. At this time I also whittle the gist-list down so that each gist is uniquely meaningful (none are redundant), each is populated sufficiently (none have few stories), and the total number is manageable (typically around twenty or so).
- Then I go back and read the stories a third time. This time I am not allowed to create any new gists or do any lumping or splitting unless I feel a story presents a serious challenge to the organization derived. When I finally feel like each story has been well described by its gists, I am finished.
There is also structural analysis, which is the consideration of story form by itself. It looks at things like characters, settings, plots, conflicts and story arcs, and essentially treats stories like they are movies or novels. I have used some methods of structural analysis to supplement storyteller answers at times. Some questions I have asked myself are:
- Who is the protagonist of this story? Is it an individual, group or role?
- Who acts in conflict with that person or group or role? Who helps them?
- What scope of time and space is covered by the story? How many people does it involve - a few or many?
- Which official roles are important to the story? Police? Aid workers? (etc)
- Do certain events of interest occur in the plot of this story? Is there cooperation? Deception? Self-deception? Discovery?
- Which genre of fiction does this story best resemble?
- How much non-narrative content is included in the story?
Option group five: Asking subject-matter experts
The very worst option for making sense of stories, in my experience, is to ask people who study the people who told the stories to interpret them. Why? Because they tend to have very strong opinions - based on their own experiences - about the storytellers, for or against. The most biased interpretations of stories I have seen have been made by people who consider themselves experts in the subject matter of told stories. If I had no choice but to work with subject matter experts in dealing with a batch of stories, I'd look for experts who disagree, or experts who come from different educational or geographic or cultural backgrounds, or experts who are expert for different reasons. I would try to increase the diversity of views in some way to counteract the shared assumptions held by experts in any field. There is no field of inquiry, academic or otherwise, that does not have its arguments and schools of thought; so why not use that to your advantage?
Another thing you can do when you expect the interpreters of stories to have strong unexamined (or even examined) assumptions about them and their storytellers is to confuse the assumptions by removing information about context. You can "scrub" stories so that they hide gender identities, localities, references to religion, and other contextual tie-ins. I once had some confidential stories I wanted to use to demonstrate the success of the narrative approach. How could I make my point without revealing the actual stories? I thought about the situation described in the stories and translated each story into a different subject matter domain, keeping the structure of the story intact. The real situation was highly sensitive and the fictional one was not, but the mixture of conflicts and goals was similar enough that the underlying patterns still made sense. This took some work, but when I finished I had a data set that perfectly demonstrated the utility of the sensemaking system without revealing the original stories. If I had to use the interpretations of subject-matter experts on a set of stories I would consider doing a similar translation to a domain in which the assumptions of the experts would not be called into action. If you are an expert in, say, energy technology, you will respond in a different way to a set of stories translated into the domain of, say, biomechanics.
In addition to removing context, it also works to insert distracting context. On a project whose purpose was to help executives improve their leadership skills, Dave Snowden and I intermixed stories about contemporary leaders with ones about historical leaders - Helen Keller, Napoleon, Abraham Lincoln, and so on. In another project we injected stories of historical conflicts into a data set based on recent events. The historical stories were not disguised, but they were intermixed without notice so that when viewing contemporary stories, project participants would discover unexpected linkages. Here the goal was to stimulate people to break out of old ways of thinking, but the same technique could be used to help subject matter experts avoid interpreting stories based on knee-jerk reactions. For example, if you wanted to ask experts to interpret a collection of stories about political violence, you might inject some stories about political activism, state suppression of free speech, utopian or dystopian communities, and so on. The point would be to disrupt the tendency of experts to fall back on easy answers without reflection. These sorts of methods of keeping experts on their toes, along with as much diversity of thought as can be found among experts in a field, would help to buffer the impact of accepted-thought bias.
One last method of guarding against interpretation bias is to ask experts and value their opinions, but also ask non-experts - people on the street - and compare the answers. This does not mean that experts are wrong or that people on the street are wrong. It means that their differences can be instructive.
Summing up
So those are all ways in which anyone can make sense of a group of stories without the ability to ask their tellers what they mean. Some of them are similar or identical to what you can find described in textbooks on narrative inquiry and some are based only on my own experience helping people collect stories and make sense of them. Which of these techiques are best? Each has its benefits and detriments. My recommendation is, if possible, to mix multiple approaches so that you can bounce different sources of information off each other. In that way the different methods can help each other over their various difficulties like the deaf watching while the blind listen. Together they are bound to find something they can use.
No comments:
Post a Comment