The basic geometries by which a skill integrates perception and action

Action can be seen as a sequence of coordinated changes in joint-angles. Each joint has its own geometry of possible joint movements. I simplify the geometry of joint angles in the following way:

• I use polar coordinates with its origin at the centre of the joint. I use two or three orthogonal dimensions. I treat limbs as straight and rigid.
• I treat the hips, the chest, and the head as rigid, with the waist and the neck as joints.
• I treach hands and feet as single rigid limbs.
• I treat the eyes as a single joint.
• I use common terminology to describe the angles, forward, sideways, and twist.

Perception, in its most basic form, can be analyzed in terms of geometry. Again, I greatly simplify the geometry:

• For the basic modelling, I ignore the content of vision, except for the eye-joint angles when the line of vision goes through the centre of the recognized object.
• I treat line of vision as emanating from the centre of the eye-joint somewhat like a limb, with its direction controlled by the eye-joint angles.
• I treat the perception of gravity as a straight line going through the centre of the eye-joint. The information content of the perception is the angles relative to the head position.

Vision, at the retinal level, is raster-scan (pixellated). For purposes of analysis and simplification, I shall treat internal visual information as a vector representation. The vector representation supports the application of geometry.

Geometry is different for vision and action. For vision it makes sense to think of a 3D Cartesian space, with a viewpoint that depends on the line of sight. Action at the joints is in polar coordinates, with each joint in its own coordinate space which depends on the limb more directly connected to the waist or centre of gravity. Gravity depends neither on the line of sight nor on the current pose.

• Gravity can 'normalize' the 3D space of vision by defining the vertical axis.

Skills as stored instructions for an information-processing system: in general I don't care whether skills are hard-wired into the brain and neural system, whether they are software, or whether they are a combination of the two. However, since my background and experience is primarily with software, I shall use that as primary analogy. I shall focus on the system design, looking at architecture and feasibility. The hardware / software question then becomes an implementation issue.

Perception of gravity is 'always on', and does not appear to require any action. It provides a vector from approximately the eyes to the centre of the earth. (For modeling convenience, we shall also assume such a vector from the waist (centre of gravity) to the centre of the earth.