By Paul Nicolaus
January 12, 2017 | Strap on your 3D headset, look around, and you’ll see virtual reality (VR) entering the medical realm in a variety of ways.
It’s helping individuals overcome phobias like heights or spiders, assisting those dealing with the effects of post-traumatic stress disorder, and bringing the benefits of rehab to stroke victims. Patients with some chronic diseases are reducing stress and pain thanks to VR, and children can temporarily escape hospital walls and feel like they are back in the comforts of home. Medical students are able to better understand the struggles of aging, and virtual emergency departments, clinics, and delivery rooms offer new training environments. Neurosurgeons are even using VR to portray the anatomy of a brain or spine for diagnostic and surgical planning purposes.
Simulation Models
When UCLA assistant professor of neurosurgery Luke Macyszyn uses VR to create simulation models of the spine and brain, it all begins with imaging data gathered using a high-resolution MRI or CT scan. That data is then segmented into parts and pieced together to form a 3D representation.
Used in conjunction with a VR headset, it becomes possible for Macyszyn, who specializes in the treatment of complex spinal disorders, to observe the geometry, view the relationships between critical structures, see which parts of the spine are abnormal, or determine which nerves are being compressed.
While some of this can be accomplished using two-dimensional images on a computer monitor, neurosurgeons need to extrapolate that information and imagine the 3D relationship. And that’s where the VR becomes particularly useful.
“It’s hard to get a good sort of gestalt view of all the nerves and the whole spine, especially a whole global alignment of the spine, without using some sort of augmented technology to show us all those nerves better,” he explained. “Deformity inherently is a three-dimensional problem or disease, and on a computer monitor it just all looks flat.”
Using a computer screen alone, it can be difficult to grasp in what direction and in which point of the spine it is bent or twisted whereas in virtual reality that understanding comes almost instantaneously.
For Aria Fallah, assistant professor of neurosurgery and pediatrics at UCLA, VR enables presurgical planning and rehearsal. “It allows me to visualize the critical structures of the brain surrounding the pathology, in essence giving me views that are not even possible during surgery,” the pediatric neurosurgeon explained. “This allows me to perform surgery more effectively, efficiently, and safely.”
Educational Tool
Fallah also uses the interactive visual platform to explain the underlying medical diagnosis to children and their parents. This type of patient education is where VR has made its biggest impact to date, from Macyszyn’s perspective.
If a picture is worth a thousand words, what then, is the value of a 3D image? “It’s the exact same concept,” he explained. “You’re able to communicate something to someone so effectively even if they might not have any medical training because the image kind of speaks for itself.”
“When you show that to them in virtual reality or in this kind of 3D representation they understand the anatomical relationship very quickly and easily,” he said, “whereas if you do it on your regular computer monitor using 2D slices it’s a little bit more abstract and it’s harder for them to grasp what’s going on.”
It’s much of the same when it comes to educating residents, which is also part of Macyszyn’s role. Teaching anatomical relationships is easier and better in VR than it is using a textbook or any other combination of 2D materials, he said.
For a long time, plastic models have been used to separate the various parts of the brain such as the cortex or the cerebellum into pieces. The ability to take it all apart and put it back together like a puzzle to understand the relationships in 3D form is useful, but it also has its limitations.
“That’s good for the average, normal appearing brain,” Macyszyn said. “That is not good for understanding the relationship on someone who has a specific pathology.” The VR helps create a patient’s specific 3D model that can be shown to the patient or a resident or a medical student to help them understand that relationship.
Room for Improvement
Mount Sinai Health System has become involved with VR because the potential is apparent, according to Patricia Kovatch, associate dean for scientific computing. Patients have been shown 3D printouts of their tumors, and neurosurgeons are beginning to use VR as a simulation tool.
Looking ahead, the vision is that the use of these emerging technologies will ultimately help reduce risk and cost through patient-specific neurosurgical rehearsal. In addition, VR could be used to evaluate neurosurgical skill before accepting new residents or hiring neurosurgeons.
The National Research Council of Canada’s NeuroTouch and Surgical Theatre’s Surgery Rehearsal Platform have been tested at Mount Sinai, but in order to use them for surgery rehearsal or neurosurgical skill assessment a number of improvements are needed.
“The brain tissue is like wet tissue paper,” Kovatch explained, and tumors have varying textures and densities. Within one tumor, some parts can feel like rebar and others can be quite soft, so more realistic tissue models will need to be a part of the solution.
More sensitive haptic feedback will also be required to create a more realistic experience. Picture a Nintendo Wii, for example. “You know how you hit the ball and it feels like you’re hitting the ball?” she asked. “That’s a haptic feedback device. It’s just a device that gives you some kind of feedback that makes you feel like what’s happening is happening.”
The devices tested at Mount Siani to date haven’t been sensitive enough to make recepting the brain actually feel like recepting the brain. Increased visualization size, complexity, and speed would help enhance the overall experience, but the ability to move around in the brain and view features in real time requires the right computational power and algorithms, which is no small feat.
“The technology, I would say, isn’t ready for prime time. We’re sort of in super beta mode,” Kovatch said. “It’s just early days for the technology, so I guess I’d say it’s going to be a while until it’s mature enough that it can be used at scale and a lot of patients can be helped by it.”
While Macyszyn is already seeing the benefits of simulation models firsthand, he agreed that the entire field and its complimentary technologies would have to advance significantly to be able to truly rehearse operations or medical maneuvers in virtual reality.
After all, this extends beyond the use of a visual representation to include medical tools, surgical instruments, a representation of the user’s hands and arms, and haptics.
Video Game Connection
There are plenty of players, so to speak, offering the hardware (the actual headset) needed to view images in VR. Google has a platform, and Oculus Rift is another option. In his lab, Macyszyn and his team have experimented with a variety of products, including the Samsung Gear VR.
“It requires a Samsung Galaxy phone, and it requires a $100 or $200 headset that the phone clicks into,” he said. “You put that on your face and you’re in virtual reality. That’s the most common one we use because of the ease of use, and the low price point allows us to buy multiple units and share them with patients.”
HTC’s Vive is another used by Macyszyn. “That one’s nice because there are these external monitors that kind of monitor the motion of your head, so you are placed in this immersive 3D environment,” he noted.
The goal of experimenting with all these different platforms is to see which platform is easiest to program and incorporate into a clinical workflow. Because of all the companies getting involved in this space, the cost will likely continue to be driven down over time, improving access.
Much of VR’s progress in healthcare will likely hinge on the software component, according to David Gaba, associate dean for immersive and simulation-based learning and a professor of anesthesiology, perioperative and pain medicine at the Stanford School of Medicine.
In the gaming industry, creating a high-quality product like World of Warcraft or Halo costs about $40 million, he’s been told. But there are also millions who plunk down money every month to play World of Warcraft or go out and buy the new version of Halo, so there’s also an ability to recoup that investment.
The healthcare simulation arena is another ballgame. At this point, there are still plenty of unknowns, and Gaba reminds me that VR is just a tool. Some learning comes from the experience of a simulated scenario and setting, but a lot of the learning comes from interacting with real people who can provide quality feedback following that simulated experience.
He anticipates that the hardware and some of the software creation abilities will trickle down from the gaming and entertainment arena, but the degree to which this can be accomplished at a reasonable cost will likely be one of the biggest determinants of just how far and fast the healthcare industry enters into the VR sphere moving forward.
“You look at video games and they are so realistic in a lot of ways, so some of this is trying to adopt the technologies and the approaches from other fields and bringing them to the medical field,” Kovatch said.
Even so, there’s still plenty we don’t yet know or fully understand about the human body or how people work, so we still have a long way to go. “But it’s also what makes it a very interesting problem, right? And we could argue that it’s the most important problem to work on,” she added. “That’s what makes it very cool.”
Ramping Up
So where are we now, and in what directions could we be headed? Well, we’re not anywhere near the “Star Trek” holodeck where you put your brain in a vat or “The Matrix,” where you have experiences that are completely indistinguishable from real life, Gaba said.
“This is still a growing field and a relatively nascent field,” Macyszyn said, and it will take time before VR is used on a routine basis and trusted for real medical decision making. “And I think that’s okay because this is just where we’re starting.”
In his mind, accuracy will be the biggest ongoing challenge. “In medicine, we have a very small margin of error to begin with,” he added, “and in neurosurgery, we have almost no margin of error.”
Still, projecting out into the future is a bit of a thrill. Although the intricacies of brain operations are incredibly complex, Fallah is excited to see the advancement of VR to the point where it almost completely mimics reality. “When that day comes, surgery will feel like a deja-vu,” he noted.
At the moment, it is probably appropriate to temper all excitement about what could happen with the reality that it is not going to be a simple development, Gaba said. Like any new technology, VR is subject to plenty of hype. What winds up being doable and workable remains to be seen.
“The VR thing is really just starting to ramp up, and I think we’re going to see a big explosion over the next five or ten years,” he added. “We’re just sort of on the launching pad of all that and it’s going to be great to see what all happens.”
Paul Nicolaus is a freelance writer specializing in health and medicine. Learn more at www.nicolauswriting.com.