Measurement and evaluation of a model

= Measurement and evaluation of a model

We now have a set= ting and a systems perspective with its emphasis on interfaces. The next two topics are measuremen= t and model implementation. By measurement we mean to sort out what it takes to make the model believable = and insightful. We have to evalua= te how well it corresponds to the real phenomena so that we can believe that the insights gained from the model pertain to reality, and help us understand reality.

Before we can mea= sure and evaluate, we need to understand what the real issues are, so that we are not mislead by surface phenomena such as how realistic the model seems to be to= the naked eye. The measurements s= hould relate to the problems of understanding that we are facing, and to the ques= tion of how well alternative hypotheses address these problems.

= Information density

In the previous c= hapter we discussed that the world has no memory, but that we would follow it fram= e by frame. An obvious question is= to estimate how much information is involve for a single actor for a single fr= ame at the world – brain interface for that actor. With this estimate we are not inte= rested in the information content of the world but only in the much more contained problem of how much information has to flow into and out of the brain of the actor to deal with that single frame.

Information driving action=

Let us start by f= ollowing the mechanical side – moving the bones. For a single frame we need informa= tion for each bone and for each potential direction of motion for that bone (deg= rees of freedom). There must be a = range of values and of precision. In muscle – nerve-ending terms this is reflected by the fan-out into distinct nerve-endings as well as by the maximum and minimum firing rate (refresh rate).

Note: Unlike our model, nerve firing is not synchronized into frames, where all the information for a single frame is delivered essentially simultaneously. However, trea= ting each nerve-ending as bit, the question of precision still applied since the muscle cannot have more distinct responses than it has combinations of firi= ng patterns by distinct nerves.

So for 250 bones = each with three degrees of freedom and a precision of only a thousand distinct values we would have 250 * 3 * 10 =3D 7500 bits for each frame. This is only 225,000 bits per seco= nd if we assume 30 frames per second. Chances are that there is more redundancy and precision in the nerve= to muscles information transmission, so that an information rate 10 to 100 tim= es our estimate might be more reasonable.

Information from perception

Let us approximat= e the information coming in from visual perception. Again, we are not concerned with t= he information available in the world but only with the information flowing in= to the brain.

It is a ‘pull’ situation, where the information is not updated unless t= he eye actively looks. This requ= ires the eye to update the information whether or not there has been a change. Let us also assume the same frame = rate as visual refresh rate (which is high by a factor of 2 or so). We therefore have the pixel resolu= tion of the eye as well as the colour depth (gray scale). Let us assume a resolution of only= 1 megapixel (lower than cheap cameras) with a colour depth of 8 bits (low). Even with a third of the frame rat= e, we are at an information rate of 80 megabits per second.

Learning:&n= bsp; Information from 17 minutes of a single perception-linked action

Let us look at the information associated with 1000 seconds of action, where the action is associated with visual information. (We are disregarding any information from other senses and proprioception that may also be associated with this action.) Just to store the experience, with= out any comparison to similar experiences, would take over 10 gigabytes.