Tracking and Identification via Object Reflectance Using a Hyperspectral Video Camera

Recent advances in electronics and sensor design have enabled the development of a hyperspectral video camera that can capture hyperspectral datacubes at near video rates. The sensor offers the potential for novel and robust methods for surveillance by combining methods from computer vision and hyperspectral image analysis. Here, we focus on the problem of tracking objects through challenging conditions, such as rapid illumination and pose changes, occlusions, and in the presence of confusers. A new framework that incorporates radiative transfer theory to estimate object reflectance and particle filters to simultaneously track and identify an object based on its reflectance spectra is proposed. By exploiting high-resolution spectral features in the visible and near-infrared regimes, the framework is able to track objects that appear featureless to the human eye. For example, we demonstrate that near-IR spectra of human skin can also be used to distinguish different people in a video sequence. These capabilities are illustrated using experiments conducted on real hyperspectral video data.