The University of Maryland School of Medicine (UMSOM) is the joint recipient of $4.75 million in funding from the National Science Foundation (NSF) to establish a new Center for Medical Innovations in Extended Reality. The center, known as MIXR, will help accelerate the development of virtual reality, augmented reality, and other immersive media technologies — referred to as “extended reality” — for use in clinical trials and eventually widespread use in medical care.

From left, Sarah Murthi, Adam C. Puche, Nicholas Morris, Luana Colloca, Amitabh Varshney, E. Albert Reece.

“Immersive technologies have the potential to fundamentally change, improve, and reduce the cost of medical training and maintaining clinical skills across all aspects of health care,” said Sarah Murthi, MD, associate professor of surgery at UMSOM and co-investigator on this project. She is an attending physician at the R Adams Cowley Shock Trauma Center and director of the Critical Care Ultrasound Program. “They also have the huge potential to improve patient care, in everything from how pain is managed to how procedures are performed.”

Murthi plans to continue working to understand how extended reality can fundamentally change medicine. She is focusing on using these tools in education and training and to help improve the patient experience in those who are critically ill. She is also working to implement innovative extended reality medical displays that could improve how bedside procedures are done. She helped develop and test an augmented reality prototype that overlays ultrasound, CT, and video data directly on the patient.

Collaborators on these projects include Adam C. Puche, PhD, professor and vice chair, Department  of Anatomy and Neurobiology at UMSOM, who is working on virtual reality technologies for surgical education and training, and Nicholas Morris, MD,  assistant professor of neurology at UMSOM, who is investigating the use of virtual reality programs for pain management in critically ill patients. Luana Colloca, MD, PhD, MS, MPower Professor and TL1 Program Director from the University of Maryland School of Nursing, also is a collaborator and the Principal Investigator of a series of MPowering the State- and National Center for Complementary and Integrative Health-funded studies. These studies explore the neurophysiological mechanisms of virtual reality for pain modulation directly in pain-free and chronic pain due to temporomandibular disorders.

The initial work started in the Maryland Blended Reality Center (MBRC), which Murthi launched in 2017 with Amitabh Varshney, PhD, professor and dean of the College of Computer, Mathematical, and Natural Sciences at UMCP. Funding for this initial effort came from MPowering the State, the strategic partnership between UMCP and UMB. Varshney serves as the lead-site principal investigator on the MIXR project.

“The synergy in MIXR will be contagious,” Varshney said. “Our industry partners will push forward new ideas and novel technologies. The scientists and physicians will help refine and test those ideas. And we both will work with the Food and Drug Administration [FDA] to bring these technologies from the lab to the proper health care setting where they can have an exponential impact.”

Varshney and his colleagues at UMCP will soon unveil a “HoloCamera” studio, where more than 300 immersive cameras are fused together, creating a unique 3-D visualization technology. The aim is to create a holographic type of image to assist in the training of health care practitioners conducting intricate medical procedures.

In 2018, Murthi and Varshney co-authored an op-ed in the Harvard Business Review that detailed how augmented reality could improve patient care and lower costs in hospital settings. They also teamed up with the Maryland Institute College of Art to test a virtual reality platform that can help patients deal with physical and emotional trauma through immersion in another world with a focus on quadriplegic patients who are hospitalized with acute spinal cord injury.

Murthi also plans to continue her collaboration with Toby Guerin, JD, associate director for the Center for Dispute Resolution at the University of Maryland Francis King Carey School of Law, on developing virtual reality interventions to improve empathy and understanding between police and community members as an innovative form of violence reduction.

The award is part of NSF’s Industry-University Cooperative Research Centers (IUCRC) program, designed to jumpstart breakthrough research by enabling close and sustained engagement between industry innovators, world-class academic teams, and government agencies.

Behrooz Shirazi, acting deputy division director of the NSF’s Division of Computer and Network Systems and a program director for IUCRC, called MIXR one of the first national centers at the intersection of medical and computing sciences. “We expect this vibrant collaboration to produce significant societal and health care impacts,” he said.

In addition to Google, Microsoft, and Meta, other technology companies involved in MIXR are Sony, Magic Leap, Health2047, GigXR, Brainlab, and apoQlar.

Another key partner in the MIXR initiative will be regulatory experts at the FDA, who will provide input and review of the new technologies to ensure that they are safe and effective for patient use. 

“We are so grateful to NSF for their support of this important collaboration. Together with our colleagues in academia, industry, and government, we can advance state-of-the-art medical technologies and provide faster benefits to patients,” said UMSOM Dean E. Albert Reece, MD, PhD, MBA, executive vice president for medical affairs, UMB, and the John Z. and Akiko K. Bowers Distinguished Professor.