Author: Rainer von K�nigsl�w
Date: July 18. 1995
Version 0.2
Date: Dec 17. 1995
Version 0.3
There is a common notion that information comes to us from the outside through the sense organs and into the brain, where it is assembled and interpreted and then leads to a response. This type of notion in its simplest form is known as stimulus-response. As part of this approach it is commonly assumed that the external parts, the stimulus and the behavioural response are most real, and best understood. I will propose an alternative approach, where the internal representation is "reality". In this view perception is a motivated activity, i.e., we use our senses to interpret the world to fit the internal reality. Similarly we behave in accordance with the internal reality. Let us furthermore posit that the "atomic" units of internal reality are stories and plays. Let us assume that we carry a collection of
Another way of expressing this concept is that we are forever acting "as if ...". We would claim that the stories or representations that underly the "as if" constitutes the core reality from which we act (c.f. Philosophy of As If).
When we address the question: who am I, we often do it with analogies - I am like this person in this play or story.
Both stories and plays are historically important means of communication, often formalized and ritualized. Initially in books, and now in movies, stories and plays are everywhere.
Like morality plays they carry values on preferred ways of behaving. We have heroes whose behaviour we should imitate.
Stories and plays are a compact representation of organized and systematic behaviour. It is a possible answer to the question: what do we learn when we learn to play a role, to be a mother, a doctor, a bus driver, etc.
Using this approach, the concept of behaviour is fairly straightforward: we are acting a part in a story or play. The more difficult questions are how we select the story to use from our repertoire, and how perception fits into this model. Let us start with the problem of perception.
According to this model, perception is an organized activity with a purpose. We use and focus some of our senses to collect information. These will normally be an anticipation of what kind of information we expect to perceive. Other information that is not relevant at that moment may be ignored. Use of senses is selective, i.e., the input from some senses may be ignored.
For instance, let us assume that we are looking for a friend in a crowd. We will use our eyes to systematically scan the environment. We may even step on something to raise us up to see better. We may ignore sounds and touch, i.e., ignore conversations going on around us, and the occasional touch of other people against us.
Following up further on this model, we can use a computer analogy, where input is filtered through a format to make it meaningful in the context. Thus the same information can be recognized as a number or alternatively as a string of characters with different format filters. In this analogy the incoming information is matched against a set of expectations which determine alternative interpretations of the stimulus information.
This analogy is essentially equivalent to the Kantian notion of a filter. A restricted notion of SR can be accomodated within the model, as we can have unexpected perception events cause errors or interrupts which interrupt the normal sequence of purposeful perception.
Examples of such errors or interrupts might include: (in our crowd scene)
Even with such interrupts, we would still claim that the stimulus does not carry meaning, but that the interrupt causes context switching, where new filters or formats are applied to interpret the information
There are several extensions we need to make to the model. In most traditional views of perception, there is a concept of assembling complex meaning from basic "atomic" pieces of direct sensory data. Let us try an illustration, of recognizing a person running toward us:
With purposeful perception we might want to have top-down perception:
Furthermore, we might very high level scripts, like miniature movie parts against which we test our perception:
One might extend the model, therefore, to see us as carrying around scripts or plot lines, like from Harlequin romances or from movies. Rather than "perceiving reality", accoring to this model, we would be applying all the different novel or movie scripts and selecting which of the scripts best fits the incoming perceptual data.
We might borrow Herb Simon's notion of "satisficing" to postulate that we do not test againsts all the alternative scripts that we carry around. Rather we might test the alternative scripts in order until we find one that sufficiently fits the perception to satisfactorily explain what is going on. We "satisfice" rather than "optimize" fit. Some people may put in more effort than others to carry more alternative scripts, to note more discrepancies in matching, and to go through more alternative scripts before they are satisfied that they understand.
Further integrating notions of primacy and recency found in memory research, we might explain the order in which we test and apply the scripts. Primacy would give emphasis to scripts we obtained early in our lives, which recency would emphasis recent new additions to the alternative script library we carry.
Just as anecdotal support for the recency notion, I have certainly noticed that I am influenced by novels I read, and that I might see my wife as similar to the heroine of the last novel I read, and suddenly "perceive" new intentions and motivations, as laid out by the author of the novel.
Above we have focussed on "people" stories, i.e., accounts that help to guide our own behaviour. However, the concept can be extended to other endeavors. We might see science as a systematic approach to story telling. Theories and formal models give us a systematic and "public" way of representing the story. Experimentation gives us a systematic way of evaluating stories and of comparing competing stories that purport to be about the same subject matter.
We can apply the notion of stories in business at a variety of levels. For a company as a whole, we have the fiction of a corporate entity with a mission. Within our departments, and for our jobs we have the concepts of job descriptions and roles which give a verbal description on how we should act in various contexts.
How does this model apply to the design of information products such as electronic books and software applications? We can take the same approach and say that data has to be recognized to become information, and that information has to be recognized in context to become knowledge. We shall posit that we carry equivalent patterns to the movie scripts which allow us to recognize data and attribute meaning to it.
However, the data often is ambiguous and will support more than one interpretation. We can be dreadfully wrong in interpreting data.
One famous example arose with the nuclear plant at Three Mile Island. Signal gauges gave off warnings, alerting the operators. The operators saw the warnings and inferred that there was a problem with a pump and acted to counteract the problem. Unfortunately, the problem was not with the pump but with another signal. The operator actions, since it was addressed to "cure" the wrong problem in turn caused a serious malfunction.
There are many such examples, all of them pointing to the fact that the correct action is
The above example dealt with process control. Similar examples can be constructed for legal reasoning, forensic accounting, project management, piloting aircraft, and in many other areas.
The above examples looked at this process of understanding from a defensive orientation, i.e., how can we prevent errors of interpretation. One can certainly move along this path by helping the operator see alternative interpretations quickly, even when he/she is under pressure to respond immediately.
An alternative approach is to look at this process of understanding as an opportunity to build systems that are easier to understand, by fitting them to mentals models and scripts that users carry in their heads.
At the core of this model we have positioned the concept of a story. What we mean by that is:
There should be a relatively small set of skeletal stories (or story fragments) with well-understood structures (goal driven or event driven -- single actor or multi-actor -- ...)
Part of the model is the claim that a story is the "atomic" element of meaning. This contrasts with linguistic theories which claim that words or sentences are the smallest element. (Paragraphs - K. Pike). The concept developed here is that words, whether in reference to entities or actions are implicitly part of a story which assigns them their place. (There is some support for this notion in the anthropological literature, e.g., empty gas cans as entities or events).
The story must be a representation of "programming", i.e., it must be usable as instructions on how to behave (both active perception and motor activity)
Limits of the representation
We do not know how stories are "stored", or how they are represented when they are in use. It also seems likely that there are large individual differences. A reasonable goal at present is to look for functional analogies that have the appropriate properties. Books, movies, audio-tapes, video games, computer simulations, and electronic multi-media systems offer vehicles to construct functional analogies.
Even though the story is the "atomic" building block for meaning, we normally work with small story fragments. I would claim that even in chivalrous gestures, such as a gentleman opening the door for a lady, the gesture is a fragment that casts the two people momentarily into the roles of lady and gentleman, and it is through that symbolic link to a set of "gentleman & lady" stories that the gesture becomes meaningful.
I believe we normally participate in multiple stories more or less at the same time, as we act in more than one role. (Explore also the link to Goffman - phenomenology).
When two people interact they have to match roles -- there is some subtle negotiation about the nature of the story they are acting out. If there is no agreement then there can be major disappointments & frustration. I find in generally in personal interactions, there is a lot of pressure toward a single story -- i.e., one becomes defensive if one is playing the role of a bad person in the other person's story. -- It is very difficult to maintain one's own story if the other person has a very strong and convincing view of who one is. Some people seem to project stronger and be more persuasive than others.
Similar concepts apply in business and in science. One can be very influential in meetings just by having a very clear vision, and understandable story to tell. Once can exert leadership through the simplicity and persuasiveness of the story, even without any authority. The same is true in politics. There are typical stories such as: we are all victims and therefore we are justified in aggression.
Predictive/interpretive power of story skeletons
If we see ourself as participating in a story we have to match our past behaviour, and that of other participants to the story line, up to the "current" scene.
By following the story to its conclusion we can predict how the current situation will play itself out (assuming we and other actor/participants will continue to adhere to the story line)
The story assigns us and other actors a role in the story, and thus helps us understand our relationship to others.
We can look at cognitive ergonomics from a requirements perspective - what do we need to deal with a complex situation, a complex system.
We need a mental model that tells us how to act
Like programming a robot, there is a macroscopic control sequence and a microscopic "implementation" of macroscopic instructions
There is a multi-layered representation: stories within stories
According to this model, to put together an ideal 2 actor story we have to match story lines at all levels.
Children practice role playing. Children observe role models & imitate. Kids play doctor, nurse, mother -- and act these out as play dramas - work on the core activities and interactions.
We take formal courses and instruction to learn how to act systematically. We use apprenticeship.
How do we detect when a story is inappropriate & needs change?
There is always imperfect correspondence between our story and reality. We also often have the choice between changing our story and changing reality. It may well be that one cause of marital unhappiness is theis lack of match. In some cases it may be easier to change marriage partners than to change the story - especially if the story is central to our view of ourselves.
In science we have explicity models of experimentation, establishing correspondences between the model and the world, using systematic measurements and established (statistical & other) evaluation procedures. Even in science the points of correspondence are sparse, i.e., there may be a lot of content in the story or model that does not have a direct correspondence with observable or predictable things. Our internal stories for our personal life are almost at the other extreme, in that there are no established methods for evaluating correctness.
In the multi-layer model, correspondence is required at all the levels, but for complex stories is is unlikely to be exact. Furthermore, with our model of active perception, we would predict and active process of trying to match the elements.
We can see the active perception and the active matching when the correspondence breaks down & is verbalized:
Both the expectation and the "perception" in fact are story components and rarely describe actual behaviour in objective, detached detail. Rather they tend to include intent, feelings, etc.
A common type of error in matching to reality is illustrated by the multi-layered example of the romantic evening:
His story of an embrace might include some cues that he expects from her that she is ready for an embrace. So he might be ready and willing for an embrace, and he might think that he has given the gestural signals that he is ready for an embrace, and he is now waiting for the gestural cues that tell him that she is ready for an embrace, but they do not come.
Her story of an embrace might be that he should be tempestuous and sweep her off her feet. She might feel that she should pretend to be reluctant, as part of her story line -- rather than being forward and "initiating"
When the embrace does not happen, both might be disappointed and feel that the other did not share the story of wanting to share a romantic evening
He might feel that she was not ready and willing to be intimate at that level
She might feel that he does not find her sufficiently attractive to be willing to be romantic
In other words, both might misinterpret the mismatch to an element in the higher story line, the interest and willingness to have a romantic evening together. They might not realize or understand that they have a mismatch in a lower story: how to proceed to engage in an embrace
There are two main approaches to the design of instructional materials. The two approaches complement each other, but they are quite different in assumptions underlying the learning.