UMD Receives NSF Major Research Instrumentation Award in Support of Virtual and Augmented Reality

Visualization tools that allow a surgeon to “see through” a patient during surgery. Wearable sensors that encourage healthier behavior. Virtual-based headsets that train soldiers how to remove a wounded comrade from the battlefield.

Innovations like these and more under development at the University of Maryland just got a big boost.

The National Science Foundation (NSF) has awarded UMD a $600,000 Major Research Instrumentation grant in support of virtual and augmented reality research and education. Virtual reality (VR) either mimics real-world settings or creates fantasy worlds; augmented reality (AR) embeds digital information into real-world settings.

Both fields are expected to expand exponentially in the near future in applications tied to scientific visualization, entertainment, military uses, architecture, navigation, education, prototyping, collaborative communication, and more.

The NSF grant will be used to purchase new equipment and provide infrastructure support for a 1,000-square-foot “augmentarium” now under construction. The Virtual and Augmented Reality Laboratory will feature interactive projection displays, robotic mounts and high-speed computing clusters that, university officials say, will position UMD as a leader for the effective visualization of large and complex data.

When launched later this year, the interactive lab will bring together researchers from across campus and beyond to explore ideas and technologies that combine real-time data within virtual settings and backdrops.

“These technologies will engage our faculty and students to explore new pathways of discovery that can have far-reaching scientific and societal benefits,” says Jayanth Banavar, dean of the College of Computer, Mathematical, and Natural Sciences (CMNS).

The virtual and augmented reality lab is the latest addition to the university’s Institute for Advanced Computer Studies (UMIACS), with its director, Amitabh Varshney, named as principal investigator of the NSF award.

Varshney and other researchers in the Graphics and Visual Informatics Laboratory—an NVIDIA CUDA Center of Excellence—have led recent efforts in virtual and augmented reality research at Maryland and are expected to play a lead role in the new lab when it is fully operational.

Projects supported by the NSF grant include: understanding large-scale astronomy data (Lee Mundy in astronomy); weather and climate prediction (Kayo Ide in atmospheric and oceanic science); characterization of stem cells (Peter Bajcsy at the National Institute of Standards and Technology and Antonio Cardone in UMIACS); large-scale simulation of rotorcraft brownouts via accelerated algorithms (Ramani Duraiswami in computer science and UMIACS); data visualization for cybersecurity (Tudor Dumitras in electrical and computer engineering and UMIACS); user interaction with augmented reality (Catherine Plaisant in UMIACS and Rama Chellappa in electrical and computer engineering and UMIACS); visualization of big data (Joseph JaJa in electrical and computer engineering and UMIACS, and Varshney); and augmented reality-based, image-guided surgical interventions (Rao Gullapalli in radiology at the University of Maryland, Baltimore, and Varshney).

The grant also provides for education and training, with plans for the Maryland Center for Women in Computing to develop workshops that encourage middle school and high school girls to participate in VR and AR projects.

Other research will study how people interact with VR and AR technologies, and how they can best be used in an educational setting.

The virtual and augmented reality lab will also take advantage of several cross-institutional partnerships supported by MPowering the State, which joins scientists at the University of Maryland with physicians, clinicians and other health experts at the University of Maryland, Baltimore.

“We look forward to further collaboration with our colleagues in Baltimore to identify opportunities and leverage our combined strengths in computing power and clinical expertise,” says Varshney.

Researchers from both institutions envision specialized headgear that surgeons can wear in an operating room, providing real-time patient and surgical data that is “overlaid” on top of a patient during surgery.

Additional projects will look at wearable sensors that can track the movement of people suffering from neurological disorders like Parkinson’s disease, keeping tabs on the disease’s progression in hopes of providing better therapeutic outcomes. Also under discussion are wearable technologies that monitor human activity and offer visual feedback via warnings or positive responses, both of which can promote healthier behavior.

The federal government is very interested in VR and AR applications, including the development of specialized headwear that can provide soldiers with critical information like weather, combat efficiency, and the location of both friendly and adversarial forces. Other applications can train soldiers in essential tasks, such as evacuating wounded comrades while under fire on a virtual battlefield.

Initial seed funding for the UMD virtual and augmented reality lab came from CMNS, with additional support from the university’s Division of Research and the provost’s office.

To view illustrations of several virtual and augmented reality projects under development at UMD, go here.