Overview of 15 Companies Using VR and AR Technologies in Healthcare

Source: Siyi Medical Device Observation

Reprint Requirements: Can be reprinted 24 hours after the first release, source must be noted

Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) technologies have demonstrated their value in medical imaging, surgery, clinical treatment, telemedicine, and medical/health education, further promoting precision medicine. They enable doctors to understand patient conditions more vividly and intuitively, quickly identify problems, and determine better treatment solutions.

In recent years, companies focusing on “VR/AR/MR + healthcare” have sprung up like mushrooms after rain, with frequent financing activities. According to incomplete statistics from Qingtian Network, in 2020, 13 companies globally received financing for the development of VR/AR + healthcare service projects (3 for AR, 10 for VR). In July 2021, there were three more financing mergers in the global VR/AR + healthcare field. According to Markets & Markets analysis, by 2022, the global market share of medical scenario simulation software and equipment will rise from $1.2 billion in 2017 to $2.5 billion.

(Image Source: EY)

One of the first applications of VR technology in medicine is to simulate upcoming surgeries in advance. Surgeons can browse 3D views of patients’ MRI or other scans to understand the situations they will encounter during surgery, allowing them to plan their work. VR is becoming the gold standard in the healthcare industry, especially in surgery. Surgeons are currently using various tools, from VR headsets to haptic gloves, to simulate real surgical procedures, improving preparation and efficiency in the operating room.

VR technology has significant advantages in assisting the treatment of mental disorders. The virtual environment of VR can alleviate uncertainty during psychological therapy. With VR glasses, patients can easily enter the virtual scenarios set by doctors and work to overcome their psychological barriers in an immersive experience. Abroad, VR is used to treat PTSD in veterans, phantom pain in disabled individuals, ADHD in children, autism, cognitive dysfunction, and is effective in improving acrophobia, claustrophobia, and fear of flying. In China, VR systems for social anxiety and autism are in clinical trials, and a certified VR detox system featuring real-life shooting short films has accumulated usage records from tens of thousands of drug addicts at the Zhejiang Provincial Drug Rehabilitation Center and is being promoted nationwide. As a remedial measure, VR also provides help for individuals whose vision cannot be restored, allowing those with macular degeneration, glaucoma, or retinitis pigmentosa to wear headsets and watch their favorite TV shows.

AR applications in the medical industry are also continuously growing. It helps both doctors and patients; patients can learn about their diseases through AR, while doctors can perform high-precision, complex surgeries with AR assistance. Microsoft’s HoloLens glasses can view human anatomy; Philips’ AR applications are used to monitor patients’ vital signs; IIIusion’s 3D augmented reality equipment provides significant assistance to doctors.

Currently, although AR is still in its early stages of development, it is highly compatible with medicine and can provide strong assistance in diagnostics, plastic surgery, and education. Computer-assisted AR technology in surgery can currently be used as displays and models, including live surgery broadcasts and recordings, anatomical assessments, and remote guidance. Additionally, AR medical education systems can provide students with the most realistic observation objects. Through the three-dimensional construction of teaching models, students can observe real three-dimensional models closely and learn by simulating various anatomical specimens. In the field of women’s beauty, AR products have immense potential. AR devices can capture users’ body curves and calculate different plans through software, projected onto users, allowing them to choose whether to undergo surgery based on the final effect demonstration.

MR is actually a further development of virtual reality technology, which introduces real scene information into virtual environments, establishing an interactive feedback loop of information among the virtual world, the real world, and the user, enhancing the realism of the user experience. Currently, mixed reality technology has been applied in several industries, including healthcare. MR provides faster and better care at a lower cost by deploying mixed reality before, during, and after surgery. Using mixed reality to overlay MRI images and 3D views from CT scans assists experts and surgeons before and during surgeries, improving accuracy and reducing the need for additional interventions. The application of MR allows doctors and clinicians to quickly share imaging results, improving doctor-patient communication, promoting more informed patient decisions, thereby enhancing care quality and patient experience. It equips frontline healthcare workers with assistive care solutions to improve services and care for aging and underserved populations.

Proprio is a medical technology company located in Seattle, Washington. Proprio integrates artificial intelligence, computer vision, augmented/virtual reality, and robotics into a proprietary platform that creates new ways of observation during surgery. Proprio combines machine learning and AR to create ultra-precise 3D medical images. Imaging tools help surgeons see obstacles and collaborate with colleagues to develop surgical plans. The built-in AI platform can also analyze 3D renderings and store and share surgical data in real-time.

Proprio has conducted pilot projects in neurosurgery and orthopedics at hospitals such as Seattle Children’s Hospital and the University of Washington School of Medicine.

ImmersiveTouch creates platforms to enhance doctors’ capabilities, develop minimally invasive techniques, and create new categories of care that redefine optimal patient outcomes, creating virtual reality solutions for surgical planning, training, and education. The company’s ImmersiveView Surgical Plan platform generates 3D replicas from patient scans, allowing surgeons to study and collaborate with their teams to develop surgical strategies. Using the Oculus Rift headset, surgeons can utilize various tools such as cutting, drawing, and measuring tools to simulate real surgeries.

Johns Hopkins University, the University of Chicago, and the University of Texas Hospitals use the ImmersiveTouch platform to assist their surgeons in studying and planning surgeries.

TrueVision develops 3D digital visualization software and augmented reality tools to support surgeons in ophthalmology, neurosurgery, and microsurgery. The digital microscope platform is TrueVision’s main product for digital surgery, converting existing microscopes into digital surgical systems capable of generating AR and 3D images.

TrueVision collaborates with OCULUS Pentacam AXL to use 3D imaging and real-time analysis computer-guided software in cataract and refractive eye surgeries.

EchoPixel is dedicated to developing technology that changes the way patient care is delivered and achieves precision health. EchoPixel’s software platform is the first turnkey solution that allows doctors to use standard medical images to create a digital twin of patients and experience it as a 4D interactive hologram. The EchoPixel interactive virtual reality surgical platform helps doctors identify anatomical structures of interest and allows them to enhance detailed medical images in real-time using the platform’s depth information.

EchoPixel True 3D uses various image datasets to generate anatomically correct, patient-specific 3D images. The company gained international news for its 3D images of the colon, which help surgeons plan surgeries and perform real-time analysis.

Osso VR is a virtual reality surgical simulation platform that provides realistic hand-based interactions. The company’s VR focuses on practicing with virtual tools used by surgeons for orthopedic and spinal surgeries. Osso VR offers medical device companies and healthcare professionals a better way to share, practice, and learn new skills and procedures using virtual reality.

Osso VR has developed an interactive platform where multiple surgeons can interact and complete training courses or practice procedures together.

FundamentalVR is a deep IP company dedicated to the intersection of immersive technology (XR), haptics, and machine learning. FundamentalVR provides haptic simulators that allow surgeons to practice and improve their skills. The haptic technology uses realistic vibration patterns to give doctors the sensation of holding actual tools. The FundamentalVR ecosystem combines with VR to provide doctors with a lifelike platform to study and practice surgeries.

FundamentalVR collaborates with well-known leaders in the health and technology industries, including Amazon, Oculus, Bayer, and University College London Hospitals.

SentiAR is a holographic augmented reality platform for intraoperative clinical use. During surgery, surgeons can view real-time 3D images of specific anatomical structures (floating above the patient on the operating table). The visualization is hands-free, allowing surgeons to gain in-depth insights into the patient’s health throughout the procedure.

The SentiAR platform has successfully treated and analyzed arrhythmias in laboratory settings. Therefore, Microsoft chose this company to collaborate on their mixed reality platform.

Augmented Intelligence creates AR and VR educational tools for doctors and medical students. The company’s Body Map helps students and professionals observe a complex, anatomically correct human model to understand different systems and practice their surgical skills.

Doctors at the Chang Gung Proton Therapy Center use augmented intelligence’s AR tools to map the optimal patient positioning for proton therapy. The center employs artificial intelligence and algorithms to treat patients’ cancer more quickly and effectively.

Proximie is a technology platform that allows clinicians to virtually “enter” any operating room or catheter lab from anywhere in the world. Proximie’s AR toolkit helps doctors identify patients’ diseases and describe surgeries using 3D models. They can scan the patient’s body, point out health issues (such as tumors or fractures), and use images to show patients the step-by-step process of their potential surgeries.

Proximie was recognized as one of the top 40 digital innovators at the World Summit Awards 2018 for its breakthroughs in healthcare and AR.

Surgical Theatre provides visual assistance to surgeons, patients, and healthcare institutions by offering experiential 360°-3D at every step of the surgical process. Surgical Theatre created a VR-based surgical rehearsal platform (SRP) for preoperative planning and rehearsals in neurosurgery. The company’s platform scans 2D brain images, and on-site VR experts create personalized 3D models. The brain models are then used to help guide patients in understanding the surgical details and as planning tools for the surgical team.

New York University, Mayo Clinic, and UCLA all use Surgical Theatre’s VR brain modeling to plan surgeries and thoroughly explain procedures to patients.

Novarad’s OpenSight is an FDA-approved augmented reality surgical navigation system that enhances surgical accuracy. OpenSight renders highly detailed holograms from 2D, 3D, and 4D digital images from any modality, using patented virtual tool technology with an integrated targeting system, allowing holograms to be accurately overlaid directly onto the patient’s body. Virtual annotations covering incisions define pathologies and anatomical structures and create virtual needle or instrument insertions.

Novarad is located in the technology corridor of the Salt Lake City metropolitan area, supporting over 1,000 installations globally and serving clients in most regions of the world.

Microsoft’s HoloLens 2 is designed for precise, efficient hands-free work, ergonomically designed, unrestricted independent holographic devices with enterprise-level applications that improve user accuracy and output. With HoloLens 2, healthcare professionals can connect with remote experts, pull up patient data, and go beyond X-rays to review 3D MRI images at the point of care. HoloLens 2 achieved a 177% ROI and a net present value of $7.6 million within three years, with a payback period of 13 months.

Intuitive Surgical’s da Vinci robot transcends direct interaction between doctors and surgical instruments, allowing the robot to observe the surgical process through the doctor’s eyes. Computers and robots are positioned between the doctor and the patient, translating the doctor’s hand movements on the computer into movements of surgical instruments, and transmitting a 3D view of the surgery through customized displays or monitors. Once this system is established between the surgeon and the patient, it can simulate the operating room and the surgery itself.

Founded in 2015, Jincheng Medical Technology Co., Ltd. provides integrated online and offline medical services covering the entire process before, during, and after surgery, continuously developing visualization medical applications based on artificial intelligence, augmented reality (AR), mixed reality (MR), and virtual reality (VR). Jincheng Medical possesses cutting-edge augmented reality technology and multi-channel HD audio and video transmission, enabling hospitals and command centers to obtain patient conditions in real-time during rescue and transfer processes, achieving real-time information synchronization in pre-hospital emergency care, allowing emergency personnel to receive support from hospital experts at any time. Their AI 3D automatic reconstruction holographic projection allows for real-time observation and interaction from multiple angles, achieving precise remote planning before surgery. Centered around a remote consultation center, it ensures connectivity between operating rooms and experts, and between experts, providing efficient real-time remote surgical collaboration services. Additionally, it can offer remote ward rounds and intelligent follow-up services based on various postoperative conditions.

Established A:MLAB Laboratory and closely collaborated with the Chinese Academy of Sciences and Tsinghua University to continuously research and open up a new era of visualization in healthcare. The company’s products have been applied in several international and national projects, including the medical guarantee project for the 2019 Beijing World Horticultural Exposition, medical guarantee for the 70th National Day parade in 2019, and the national emergency medical drill on Nanzhidao, receiving unanimous praise from project stakeholders. The company’s technological innovations won second place in the World 5G Application Design Competition.

As an artificial intelligence software company, SenseTime is committed to the mission of “adhering to originality and leading human progress with AI,” and the vision of “connecting the physical and digital worlds with artificial intelligence, promoting sustainable social productivity, and bringing better virtual and real-life experiences to people.” SenseTime’s SenseCare intelligent diagnosis and treatment platform is a high-performance auxiliary diagnosis and treatment solution independently developed by SenseTime that integrates rich imaging post-processing technology and leading AI algorithms. The platform is based on medical big data, combining high-concurrency three-dimensional rendering and clinical application engines, aiming to provide AI intelligent applications covering diagnosis, treatment, and rehabilitation for different clinical directions. Intelligent three-dimensional image processing enables doctors to log into the platform easily and use a rich set of three-dimensional post-processing functions such as MPR, CPR, and VR.

Currently, SenseTime has achieved clinical applications in the following areas: intelligent clinical systems for chest CT, intelligent analysis systems for chest X-rays, intelligent clinical systems for coronary heart disease, intelligent clinical systems for liver CT, intelligent surgical planning systems for bone tumors, intelligent surgical planning systems for lower limb force lines, precise analysis systems for gastrointestinal pathology, and precise analysis systems for cervical cancer TCT.

Sources: EY, Juxiang Technology, Entrepreneur, Hongke ARVR, SenseTime, Microsoft HoloLens 2, builtin.com