The Microsoft headset fits over the surgeon’s head and displays transparent images that hover in the surgeon’s field of vision. The apps align images of the patient’s anatomy with the real-life view. When it’s all set up, a surgeon can walk around the patient, viewing three-dimensional holographic images of internal structures—such as arteries, veins, and internal organs—from different vantage points.
The headset could mean surgeons never have to look away to get information while they perform a procedure. They can use voice commands or hand gestures to enlarge images or move information around. Even the patient’s vital signs can be projected onto the field of vision. “It’s going to allow surgeons to be able to do things faster, more efficiently, and more accurately in an environment that is real,” says West. MediView is currently testing its HoloLens application in liver, kidney, and other abdominal tumor ablation procedures.
In a typical procedure, an interventional radiologist uses a biopsy needle to deliver microwave energy that heats and destroys abnormal issue. Seeing the patient’s anatomy in 3D will provide “a markedly increased ability to target cancerous masses deep within the body,” West says. (MediView has been using HoloLens 1 and plans to upgrade to HoloLens 2 when it becomes available.)
In May, New York startup Medivis received FDA approval for its HoloLens-based imaging product, SurgicalAR, which can be used for holographic surgical planning. It is the brainchild of two medical residents at New York University—a neurosurgeon and radiologist, two technophiles who began brainstorming about how mixed reality devices could help reveal surgical anatomy.
LEARN MORE
</p> <p><img alt class="responsive-image__image" src="https://i2.wp.com/www.ultimatepocket.com/wp-content/uploads/2019/11/hey-surgeon-is-that-a-hololens-on-your-head.jpg?w=640&ssl=1" srcset="https://i2.wp.com/www.ultimatepocket.com/wp-content/uploads/2019/11/hey-surgeon-is-that-a-hololens-on-your-head.jpg?w=640&ssl=1 1600w, https://i2.wp.com/www.ultimatepocket.com/wp-content/uploads/2019/11/hey-surgeon-is-that-a-hololens-on-your-head.jpg?w=640&ssl=1 1280w, https://i2.wp.com/www.ultimatepocket.com/wp-content/uploads/2019/11/hey-surgeon-is-that-a-hololens-on-your-head.jpg?w=640&ssl=1 1024w, https://i2.wp.com/www.ultimatepocket.com/wp-content/uploads/2019/11/hey-surgeon-is-that-a-hololens-on-your-head.jpg?w=640&ssl=1 768w, https://i2.wp.com/www.ultimatepocket.com/wp-content/uploads/2019/11/hey-surgeon-is-that-a-hololens-on-your-head.jpg?w=640&ssl=1 640w" sizes="100vw" data-recalc-dims="1"></p> <p>
Somehow amid the long clinical hours, Osamah Choudhry and Chris Morley formed a company. They now have partnership agreements with Microsoft, Magic Leap (for an educational use of its device), and Verizon (to test products in the Verizon 5G Lab) in New York). A 3D image helps surgeons visualize the best approach in a way that traditional two-dimensional images cannot, says Choudhry, Medivis’s CEO, who is in his seventh (and final) year of his residency. “That is the most accurate representation of the patient during the operation,” he says.
As surgical navigation systems improve, a primary challenge is lining up the data from the pre-op scans with the actual patient, who, of course, is breathing. A surgical probe can alter the anatomy. As the patient’s body and organs shift, the digitally imposed image needs to shift in unison.
One fix is to place trackers on the patient, which the AR system can detect. Whenever the markers shift, an algorithm makes real-time adjustments. “The accurate fusion of the medical images with physical patient is the most critical step in the acceptance of AR for medical procedures,” says Raul Uppot, an interventional radiologist at Massachusetts General Hospital in Boston, who has used virtual reality in teaching at Harvard Medical School.
Physicians who have donned a HoloLens and reached out to touch the 3D images are struck by how the headset could transform surgery. They can physically rotate a virtual image to see it from a different angle. They can give spoken commands. “You can say ‘show me the next scan,’ and it goes to the next scan,” says Uppot.
Holographic surgical guidance systems need FDA approval. MediView and Medivis hope to gain that and bring their products to market by 2021—opening up a new, more visionary world of surgery.
More Great WIRED Stories
Comments