The basic time management mechanisms by which a skill integrates perception and action

Action can be seen as the control of joint angles over time:

• the joints with their joint angles
• the geometry of joint angles
• the timing of changes in joint angles

Perception of gravity is 'always on', and does not appear to require any action. It provides a vector from approximately the eyes to the centre of the earth.

Visual perception can be seen as the control of the line of vision and the return images:

• the geometry of the line of vision relative to the eye and the head
• the timing of changes in the direction of the line of vision, and of focus
• the timing of changes in the perceived images
• the extraction of information from the images, such as the identification of objects

Basic timing is similar between visual perception and physical action. Images less than 50msec apart tend to get integrated. Neural information gets renewed at intervals of 10msec or more.

• Basic video runs at 24 frames/sec, which is about 40msec apart. We can use the concept of frames for both action and perception to simplify the simulation models.

Skills as stored instructions for an information-processing system: in general I don't care whether skills are hard-wired into the brain and neural system, whether they are software, or whether they are a combination of the two. However, since my background and experience is primarily with software, I shall use that as primary analogy. I shall focus on the system design, looking at architecture and feasibility. The hardware / software question then becomes an implementation issue.