High-level architecture:

Programs and datastructures

Actor (mid-level) architecture:

Programs and datastructures

3D"Oval: <= span style=3D'font-size:11.0pt;mso-bidi-font-size:12.0pt'>

----------- the programs / executable representations -----------

The ‘maj= or skeletal action’ component is responsible for moving the skeleton to perform the language-based action, based on script segments. The output is = the position of the bones in the world at each of the time-instances of the successive frames. This component is the major focus of the current research=
The ‘hands&= #8217; component looks after grasping, writing, etc.
The ‘visual recognition, reading’ component gets and interprets visual data = from the world
The ‘hearin= g, speech recognition’ component gets and interprets auditory data = from the world
The ‘attent= ion management – conscious mind’ component works with high-lev= el scripts and directs and controls skeletal action etc. For this research we are assu= ming an actor on a stage following a memorized script

----------- the data representations -----------

The actor carries knowledge in the form of a language-based script, which in turn is interpreted by the ‘attention management – conscious mind’ component
The ‘attent= ion management – conscious mind’ component sends lower-level script fragments to control the action of the ‘major skeletal action’ component
The ‘maj= or skeletal action’ component moves bones of the skeleton over = time and relative to the stage – which is the data ‘in the world’ – at any given instance, and relative to the stage. This data can be = picked up by a camera or by the eyes of an actor (including the eyes of the a= ctor who is performing the action)
Auditory and othe= r data is simplified for purposes of this simulation

‘Major skeletal action’ (low-level) architecture:

program(s) and datastructures

3D"Oval:

----------- the programs / executable representations -----------

The ‘script interpretation, expansion’ component is responsible for interpre= ting the script segments received as instructions. The output is the most specif= ic and detailed script (symbolic) components detailing the movement and posit= ion of limbs for the specific action.
‘script synchronization’ coordinates movements, arm swing and leg moveme= nt for walking.
The ‘visual recognition, reading’ component gets and interprets visual data = from the world
The script-based symbolic representation has to be translated into a hardwired, neuron-based representation that signals muscle action on bones. At the same time the differen= ce between present limb position and goal position has to be interpreted = to regulate the action in ‘real-time’, i.e., neural firing ti= me
The action signal= s are further translated and expanded, and feedback is incorporated – through further stages

----------- the data representations -----------

Instructions to a= ct are conveyed through script-fragments (language-like symbolic encoding). Once received they are beyond ‘conscious control’ – (unless interrupted, or possib= ly with new instructions)
The ‘script-fragments’ are further interpreted and expanded. E.g., walk is interpreted into swing left leg then swing right leg, based on practiced and habitual knowledge not under conscious control. The results are still symboli= c but very low level
The ‘symbol= ic instructions’ have to be interpreted into neural firings, i.e., hardwired signals. At th= is point timing moves from subjective to firing frequencies, and finally = to parallel firings across bundles
Limbs and joints = have positions and orientations that can be represented in Cartesian space (relative to the stage), in polar coordinates, and in frames relative = to the orientation of other limbs.

Time and timing-representation

Timing is typically discussed relatively to the world. If the moment of some activity in = the world is time zero, then the image of this action as recorded by the camera= is available slightly later, say at time one, and describes the past. The neural firings that caused the action must have preceded the action and thus must have happened at time mi= nus one, i.e., in the future relative to observing the action. The script must have been interpre= ted even before that, i.e., at time minus two or earlier. Any conscious thought and decision= making about that action would have been even earlier, say at time minus three. If the decision making involved vi= sual information about the scene on the stage, that information would have to ha= ve been available to attention and planning at least by time minus four, and m= ust have happened on the stage at time minus five.

Skeletal action is continuous, i.e., the neural activity continues even while new instructions are being processed, and while the actor is visually scanning = the world, listening to other actors, and consciously planning and deciding wha= t to do next. It is therefore safe= to infer a high level of parallelism at the level of distributed process contr= ol, i.e., the distributed process control components within the actor model. It is also plausible to infer parallelism within the component for major skeletal action, where low-level script interpretation parallels neural firing sequences. We also know about the diversity o= f time representation. High-level sc= ripts (instructions) follow steps (first this then that) without precise timing.<= span style=3D'mso-spacerun:yes'> Lower-level script instructions, especially those involved in coordinating such as leg swing and arm swing, likely follow some rhythm or beat (a more subjective measure, like music).<= span style=3D'mso-spacerun:yes'> Timing for neural impulses have be= en studied and seem to show their own frequencies and rhythms and are generally much faster than music beats. For the world (external reality) we have clocks. 30 frames per second is fairly typ= ical for animation (24/sec. and 60/sec. are common alternatives).

Just from this simplified sequence and discussion, we can see that timing, time sequencing, time representation, and synchronization are important but difficult aspects.