Fermüller Awarded Two NSF Grants to Develop Adaptive Cognitive Systems

Cornelia Fermüller, an associate research scientist in UMIACS, has been awarded two National Science Foundation (NSF) grants totaling approximately $1.5 million to build cognitive systems that will be able to function in complex, noisy and cluttered environments.

The grants will support Fermüller’s ongoing work in developing cognitive robotic systems that integrate perception with action, reasoning and language.

One project, “MONA LISA—Monitoring and Assisting with Actions” seeks to develop a smart manufacturing system, called MONA LISA, which assists humans in assembly tasks. The technology will help robots recognize a specific assembly action and determine whether it is correct. The robots would then communicate to humans any possible errors or suggestions on different ways to proceed.

The project will use advanced visual sensing and perception, action understanding grounded in robotics and human studies, semantic and procedural-like memory and reasoning, Fermüller says. There will also be a control module linking high-level reasoning and low-level perception for real-time, reactive and proactive engagement with the human assembler.

Another project, “Cortical Architectures for Robust Adaptive Perception and Action,” sets forth a novel idea in the design of intelligent systems with perception and, generally, in the development of synthetic intelligence.

Just about any task an intelligent system solves involves the interplay of four basic processes—context, attention, segmentation and categorization, Fermüller says. Her research team will study these processes by combining neural modeling with neural and behavioral experiments, theoretical and computational modeling, and implementation in robotics.

The motivation for this bio-inspired approach is to design systems that perceive and act in cluttered and noisy scenes that they have never experienced. This stands in contrast with the state-of-the-art in computational engineering systems that need to be retrained each time they confront an unanticipated environment.

Fermüller serves as the principal investigator (PI) on both grants.

On the “MONA LISA- Monitoring and Assisting with Actions” grant, she is collaborating with UMD researchers John Baras, a professor of electrical and computer engineering with an appointment in the Institute for Systems Research (ISR), and Yiannis Aloimonos, a professor of computer science with an appointment in UMIACS.

On the “Cortical Architectures for Robust Adaptive Perception and Action” project, Fermüller is working with UMD researchers and ISR members Shihab Shamma, a professor of electrical and computer engineering, and Timothy Horiuchi, an associate professor of electrical and computer engineering, and Johns Hopkins University researchers Ralph Etienne-Cummings and Andreas Andreou.