AR-Tho

IMMERSIVE MULTI-SENSORY TRAINING FOR EDUCATING SURGEONS.

The role

Product Manager, User Design (User Interface and User Research)

The Problem

In surgery, high proficiency in medical knowledge and surgical technique is necessary for successful patient outcome. However, instructor surgeons and residents are facing increasingly limited time for feedback and training due to increasing patient cases and complex procedures. This has created a gap in medical knowledge and technique.

Outside of cases with live patients, residents use various forms of simulation training which are resource intensive and don’t offer immersive, multi-sensory learning. Coupled with events such as pandemics and lockdowns, residents are increasingly limited in realistic practice opportunities.

The Process

The highly specialized product space required 4 months of initial research to understand the medical environment, AR technology, and user requirements properly. In this initial stage, we uncovered a need for reusable and accessible multi-sensory medical training.

Residents are increasingly constrained by time, cost, and limited practice resources. My team and I conducted 12 interviews with teaching orthopaedic surgeons, AR experts, and a medical technology director, and engaged in observations to answer our questions:

  • What skills do residents repeatedly have knowledge gaps in?

  • What technology do medical schools currently use?

Our questions took us on a field study at the UW Orthopaedic and Sports Medicine school. We were able to interact with some of the current simulation training available to residents, and although highly sophisticated each solution had at least one major drawback:

  • Limited access, whether through restricted location or single-use material

  • Relatively high cost

  • Lack of realism - poor or no physical feedback

Common themes began surfacing from the interviews, observations , and field study that had heavy design implications.

We then developed breakdown of development assumptions and personas of both teacher and student to better understand our user’s perspective. The discovery phase was starting to paint a picture of what our product should look like.

Using AGILE methodology, we began to simultaneously prototype the product and test it with medical professionals. The findings refined product scoping and product specifications, and led us to the MVP.

The MVP

AR-Tho needed to provide medical knowledge and instructions and have a physical component for students to practice on.

The decision to use augmented reality was due to its ability to overlay information in the physical world. Taking inspiration from movie visual effects, we utilized silicone encased 3D printed bones for its durability, similarity to human flesh, and low-cost manufacturing.

Challenges

  • High learning curve in product space and proposed technology. I needed to become knowledgeable in surgical education and augmented reality within a month in order to propose the project and gather a team to develop it. In researching, I had to consider possible consequences of AR in medical training, current state of technology, financial investment, and true customer value.

  • Significant potential legal, financial, and health risks. Resolved with multiple quick pivots in surgical application and context. The product moved from an operating room application to classroom training, and we simplified surgical technique used in the MVP.

  • Scope creep! Working with such cutting edge technology in a traditionally tech-conservative industry led to numerous lightbulb moments from our subject matter experts. It got everyone on the team excited to develop everything they thought was a good idea, but every meeting had to start with answering how the feature would move us closer to the MVP and solving the core problem.

  • Understanding how technology constraints can limit the MVP. 70% of the way through app development, we could not resolve a key issue: accurate tracking of AR elements over physical objects. Due to the technology at the time, slight misalignments, trace shaking, or strange visual perspective would occur. In addition to pivoting final surgical technique in the MVP, we designed a functional evaluation for app limitations and benchmark satisfaction.

The Solution

AR-Tho is an augmented reality training platform that overlays virtual anatomy onto physical models composed of silicone and 3D-printed bones. This platform allows doctors to practice procedures repeatedly in an immersive and low-risk environment.

Takeaways

  • Prototyping and proof-of-concept can be done without thousands of dollars in investment. With the amount of open-source software and at-home machinery available, I have to be creatively resourceful in application.

  • Getting buy-in from stakeholders can be difficult, but is essential for a product. I had to learn the technical knowledge to gain confidence, but also effectively communicate the potential to inspire and earn investment.

  • Be the first penguin - be willing to take risks for the reward! My entire team (myself included) didn’t have any experience developing for augmented reality or working in the medical field. We had no idea where to start or even what the final product would look like. Instead of letting that discourage us, it pushed us to pursue the project even more in order to learn, test ourselves, and truly innovate.

Next
Next

Calypso Smart Mask