A screenshot of a trainee using Osso VR. Photo / Washington Post
A screenshot of a trainee using Osso VR. Photo / Washington Post
Justin Barad never would have guessed that he'd help operate on a gorilla during his residency at the UCLA Medical Center, where he was training to be an orthopedic surgeon. Yet in August 2014, The Los Angeles Zoo and Botanical Gardens needed specialists to help Jabari, a 400-pound gorilla who was limping and unable to put weight on his leg. The surgery was successful - but it was a challenging endeavor for a team that didn't normally operate on animals.
The zoo's request was an unusual one, even by industry standards, but unpredictability is commonplace in the medical world. His experience with Jabari is one of many reasons Barad sought a solution for surgeons and doctors so often faced with unknowns. These moments drove him to dedicate his career to making surgery training more efficient by using video game technology.
Along with a team of software developers and medical experts, he founded Osso VR in 2016. It's an attempt to curb unpredictability, giving surgeons on-demand information via a virtual reality headset, and tools to properly assess how to best handle surgery. Its success is hard to ignore: It was one of Time's best inventions of 2019, and according to a study done by the UCLA, training with Osso VR is 230 percent more efficient than traditional methods. Now, despite launching just three years ago for the Oculus Rift, the virtual reality-based surgery training program is now used to train over a thousand surgeons a month worldwide.
Other training methods range in cost and effectiveness. Some surgeons use simulation centers, which are big, bulky machines used to simulate surgeries, but access to them is limited. Many residency programs and medical schools don't have one, since they are extremely expensive and require adequate space, and even at schools that do, certified doctors can't pop back into a simulation as easily as medical students can. Moreover, because the machines tend to be old, simulations are far from modern. Students practice on cadavers and animal labs moreso than virtual simulations.
But even with these resources, surgeons still make frequent mistakes: A 2015 study conducted at the Massachusetts General Hospital showed that its staff made drug-related errors in 50 percent of its surgeries.
Osso VR is an attempt to bridge the gap, elevating the quality by bringing antiquated simulations to a modern era.
"If there was a way, kind of like in the Matrix where you just plug in for five or 10 minutes and refresh yourself or get up to speed and then assess and make sure you're ready to go, that would be a total game changer and have a massive impact on patients and the health care system as a whole, all thanks to video game technology," Barad told The Post in a phone interview.
Osso VR may not exactly be equivalent to plugging into the Matrix, but it allows surgeons to hone their skills by operating on virtual patients with tools they'd normally find in an operating room. This lets trainees experience it first-hand without the fear of error. Right now, Osso VR is mainly used for orthopedic procedures, but Barad's team is working to expand into other fields like vascular, heart, urology and surgeries conducted with robotics.
Barad says that surgical training is still "primitive," relying heavily on learning through observation. He says it was during his residency at UCLA where he "experienced firsthand the biggest problem facing medicine today."
Osso VR even has a multiplayer component too, letting surgeons work together simultaneously in virtual reality. Photo / Washington Post
"I'd be in on a surgery and they'd be like, hey, get up and go run to the computer real quick and Google what we're supposed to be doing," he said.
Medical students would look up YouTube videos (the platform has become a popular resource for surgeons in training, with tens of thousands of videos on the subject) for guidance to prep for an operation. Sometimes Barad and his colleagues would even watch one in the operating theater if the surgery was especially complicated, or sprawl out an instruction manual to better understand medical devices.
Osso VR even has a multiplayer component too, letting surgeons work together simultaneously in virtual reality. Barad compares surgery to a "team sport."
"It's like soccer or like a symphony," he said. "With VR, you can get everyone in the same virtual operating room so you can train as a team and you can practice knowledge of technical skills."
Because of surgery's unpredictable nature, and since the bulk of its training relies on hands-on experience, Barad says that the only time he'd undergone a "technical assessment" was when he was asked to play the board game Operation during his residency interview. Part of the problem, Barad said, is that there are no "scalable, low cost ways" to easily measure surgical skill. Osso VR offers a solution: It has built-in analytics that objectively measure your surgical performance.
Osso VR is one of many emerging medical technologies to come from virtual reality. Sea Hero Quest, for example, is a VR game that's been used to research early signs of dementia, and SnowWorld was one of the first virtual reality simulations to help burn victims with pain management; studies say it's more effective than morphine. But virtual reality is far from the only player in the future of health care tech.
Saving lives by playing mobile games
Schell Games, a studio based in Pittsburgh that built the VR action puzzler I Expect You to Die, has regularly produced games for education and entertainment. The bulk of their library is for a younger audience, and when they've tapped into the medical field, it's been mostly for patient-facing technology, such as therapy for children on the autistic spectrum. But with their new project Night Shift, Schell Games pivoted towards making a mobile game designed specifically for emergency room physicians.
Schell Games is working closely with a University of Pittsburgh doctor, Deepika Mohan, to develop Night Shift, a training sim disguised as a story-driven adventure game that puts you in the shoes of a trauma doctor. It's currently in closed beta and accessible to a limited window of physicians before its launch.
The response from real-life doctors has already been positive. In 2017, Mohan published her research and shared that doctors who played Night Shift outperformed those who didn't.
"One of the big decisions that doctors have to make when a trauma comes in is, do they deal with it there or do they send this person to trauma center?" Schell Games CEO Jesse Schell said. "And [Mohan's] observation was that doctors don't get very good feedback about whether they made the right decision or not."
The idea was born out of Mohan's struggle to make training engaging, when past efforts had failed. She had a hypothesis: Would an interactive story with emotional beats to accompany training make it more memorable and effective? Night Shift is proof of that concept. Its physician protagonist moves to a new town and works in the emergency room. You spend shifts diagnosing and treating patients, then deciding whether their care is critical enough to transport them to a trauma center. But there's more to this world than just the hospital.
"It's not just any town," Schell said. "It turns out it is the town where [the protagonist's] grandfather had lived. And part of the reason he's moving there is because his grandfather had recently passed away and he's moving there to help settle his grandfather's affairs."
Schell calls Night Shift an "interactive soap opera" with elements of mystery and adventure. Trying to understand who your grandfather was plays a big part, but you also develop relationships with your patients.
"You get to know some of the patients," he said. "If you end up making misdiagnoses and put them in a bad way, you see them suffer for it or sometimes you see them pass away, and then you're confronted with their sobbing relatives."
While Night Shift has more of a cartoon-y aesthetic, Level Ex is a Chicago-based studio making mobile games for doctors, which feature "hyper-realistic" simulations that physicians can play on the go.
Level Ex CEO Sam Glassenberg has been in the video game industry for two decades. Despite coming from a long line of doctors, he decided against going to med school, making him the "black sheep" in his family. Instead, he became a video game developer.
Glassenberg worked at LucasArts making games for the PlayStation 2 and the original Xbox, had a stint at Microsoft where he led the DirectX graphics team, and worked with publisher FTX Games to produce mobile games based off popular Hollywood movies like the Hunger Games and Mission Impossible. But things changed when his father, an anesthesiologist, requested that he make a game to help train his colleagues in fiberoptic intubation (a technique that opens up a patient's airways). With his father's help Glassenberg "threw together" the game, called iLarynx, in a matter of weeks and published it to the App Store. To his surprise, it was so popular that a hundred thousand medical professionals were still playing it two years later.
"Clearly there was demand for this," he said. "[I wondered,] what if instead of just throwing something together, you put a team of top video game developers, designers and artists on the problem of capturing the challenges of the practice of medicine in video games? That's what started Level Ex."
Now, half a million doctors worldwide are playing Glassenberg's free mobile games. Level Ex has a team consisting of video game industry veterans, medical experts and software engineers. The studio has created four games, each with their own focus on a different specialty: pulmonology, gastroenterology, anesthesiology, and cardiology.
"Studies have shown that if a surgeon plays an action game before they go into surgery, they will perform better at laparoscopic and endoscopic procedures, even if that game has nothing to do with their actual procedure. So we know that there is a lot that can be done there," Glassenberg said.
Level Ex's realistic simulations - which run impressively well on a smartphone - let you see inside the human anatomy as you play through different cases, which involve scenarios like removing a foreign object from a patient's airways or performing a colonoscopy on a virtual Crohn's Disease sufferer. The better you are, the more you progress. And as you progress, you unlock new levels, tools and techniques. It was also important to Glassenberg to bring high graphical fidelity to his games, since most antiquated surgical simulators "look like a video game out of 1995."
Video game tech can also assist with highly irregular situations or those that would be impossible to replicate. For instance: What happens when an astronaut gets injured in space? Flight surgeons help in training and advising, but they're stuck at the NASA Mission Control Center rather than on the spacecraft. For astronauts, the onus is on them to be informed and take the right precautions and steps to stay safe.
For example, if an astronaut clutches his chest and falls unconscious in zero gravity, then he needs to be brought over to an ultrasound as a radiologist in Houston navigates the situation.
"An ultrasound is the only visualization you can do [in space]," Glassenberg said. "But the heart physically changes shape in micro-gravity depending on how long you've been up there. So how do you know what you're looking at is normal for somebody who's been in space for nine months or if it's abnormal?"
Space travel is challenging enough, and medical emergencies make it even tougher to navigate. That's why NASA asked Level Ex to build a video game simulation that shows "the human body's physiological and anatomical changes" in space, and what procedures and devices are necessary to address them. This project is still underway and not yet available to NASA, but with a significant grant from TRISH (the Translational Research Institute for Space Health), Glassenberg is positive his team is equipped to make it happen.
Glassenberg says he still has "a million questions," but he's working hand-in-hand with TRISH, NASA's medical teams and astronauts to figure it out as best he can.
As Level Ex focuses its ambitions on outer space, Osso VR is keeping its goals firmly within virtual reality. But the team still has big plans for 2020 and beyond.
Osso VR isn't yet available on Oculus Quest, which is a powerful and wireless headset untethered from consoles and PCs, but it will be coming to the platform this year. When it does, surgeons will be able to bring a portable (and affordable, in comparison to the aforementioned traditional simulation centers) VR headset to any operating room.
Barad is intrigued by Oculus Quest's hand-tracking innovations, which rolled out late last year, since surgery requires fine motor skills and accuracy to do right.
"There's a lot of interest in like gloves and hand tracking and things like that, and those are very exciting areas that are still very early," Barad said. "Using controllers works great for now, but we're trying to figure out how we can more intuitively get people's hands and fingers in the mix."
Like many of his game development peers, Barad grew up on video games, and was coding them since middle school. He even has a game credit to his name (1998′s Heretic II) with Activision. It was his "dream" to become a video game developer, and he also wanted to make a positive impact. That's where Osso VR came in.
"I'm really grateful to the video game industry, video games and the art form in general," Barad said. "This wouldn't have been possible if it weren't for video games and everybody involved in the community that basically got VR up off the ground and into really a mature product."
