Spatiotemporal representations for visual navigation

The study of visual navigation problems requires the integration of visual processes with motor control. Most essential in approaching this integration is the study of appropriate spatio-temporal representations which the system computes from the imagery and which serve as interfaces to all motor activities. Since representations resulting from exact quantitative reconstruction have turned out to be very hard to obtain, we argue here for the necessity of representations which can be computed easily, reliably and in real time and which recover only the information about the 3D world which is really needed in order to solve the navigational problems at hand. In this paper we introduce a number of such representations capturing aspects of 3D motion and scene structure which are used for the solution of navigational problems implemented in visual servo systems. In particular, the following three problems are addressed: (a) to change the robot's direction of motion towards a fixed direction, (b) to pursue a moving target while keeping a certain distance from the target, and (c) to follow a wall-like perimeter. The importance of the introduced representations lies in the following: – They can be extracted using minimal visual information, in particular the sign of flow measurements or the the first order spatiotemporal derivatives of the image intensity function. In that sense they are direct representations needing no intermediate level of computation such as correspondence. – They are global in the sense that they represent how three-dimensional information is globally encoded in them. Thus, they are robust representations since local errors do not affect them. – Usually, from sequences of images, three-dimensional quantities such as motion and shape are computed and used as input to control processes. The representations discussed here are given directly as input to the control procedures, thus resulting in a real time solution.