Searching recorded speech based on the temporal extent of topic labels

Recorded speech poses unusual challenges for the de- sign of interactive end-user search systems. Automatic speech recognition is sufficiently accurate to support the automated components of interactive search sys- tems in some applications, but finding useful recordings among those nominated by the system can be difficult because listening to audio is time consuming and be- cause recognition errors and speech disfluencies make it difficult to mitigate that effect by skimming automatic transcripts. Support for rapid browsing based on su- pervised learning for automatic classification has shown promise, however, and a segment-then-label framework has emerged as the dominant paradigm for applying that technique to news broadcasts. This paper argues for a more general framework, which we call an activation matrix, that provides a flexible representation for the mapping between labels and time. Three approaches to the generation of activation matrices are briefly de- scribed, with the main focus of the paper then being the use of activation matrices to support search and selec- tion in interactive systems.