Videotaping behavioral sequencesTwo night vision cameras are used for filming the specimens. One camera is placed above a designated filming area ensuring a top view of the specimens, while the second is place on the side of the designated filming area optimizing all viewing angles.
Digitizing Behavioral SequencesFollowing the filming process, videotapes are reviewed and edited to digital videotapes. Digitizing behavioral sequences then allows us the capability of finite behavioral analysis.
We have developed computer controlled video technology for reverse animation and kinematic analysis of animal behavior (Ayers, 1989, Ayers, 1992). This multi-media system allows correlated acquisition of kinematic and electrophysiological data by simultaneously recording behavior in the video signal and electrophysiology on the audio channels of a high resolution digital VCR. We developed extensions to a public domain image analysis program (NIH Image) which include the capability for color based acquisition and image segmentation as well as time-based quantification of kinematic parameters and correlated analog acquisition (ColorImage, Ayers and Fletcher, 1990; Ayers, 1992). This system allows us to measure animal orientation, joint angles from video on a frame by frame basis to establish the detailed movement strategies kinematics of compensatory, orientational and taxic reflexes as well as the underlieing neuromuscular control signals. As a result it has been possible to establish the coordination patterns and control signals underlieing omnidirectional walking (Ayers and Davis, 1977) as well as undulatory swimming (Ayers, 1989).
To directly transit from behavior to robotic controls, we perform finite state analysis task groups that mediate locomotion and searching individually to determine which synergistic sets are operant during different behavioral acts (Schlichting and Ayers, 1996; Ayers, Mehta and Dragich, 1997). Our analysis of the sequencing of these task groups borrows from a technique utilized by astronomers to detect motion of galactic objects. As the analysis proceeds through each frame of the digital movie, the program flashes between temporally adjacent frames of the movie with a brief pause after each cycle. Appendages that are moving the most flash in these projections. A panel of buttons that represent different states of the task groups (eg. elevation vs depression of the chelipeds, etc. are available to the investigator to specify which groups are active
By clicking on the appropriate buttons for each frame, it is possible to efficiently quantify the activity of all task groups at high temporal resolution from video tapes of specimens behaving in a variety of situations. These state diagrams are used to establish control sequences for the robots based on the behavior of the model organisms.
We define the fundamental atom of lobster behavior as an action component. An action component is a particular state which may be triggered by a releaser during which the state of all the task groups is held stabilly.
We define a sequence as an orderied series of action components which are linked as a unit. Theses sequences correspond to modal action patterns.