This abstract was part of the 2026 UCSF AI and Education Symposium
Neurosurgical training depends on graduated autonomy and high-quality coaching in the operating room, but despite vast technological advancements in the operating room, training programs and trainee evaluations are similar in structure and design to decades ago. Real-time feedback is constrained by the pace and risk of live surgery, and residents often cannot pause to document teaching points or revisit specific moments that illustrate an opportunity for improvement. At the same time, key elements of technical development, economy of motion, surgical skill, and operative capability remain difficult to capture objectively and track longitudinally across residency. Dissecting film to improve performance and preparation has become standard in other competitive disciplines such as professional athletics, but during surgical procedures this valuable data source is not captured and therefore cannot be analyzed or utilized to improve training and education.
We propose LoupeCoach, an AI-enabled educational platform that uses loupe-mounted operative video to support resident self-improvement and faculty coaching. Residents will wear a high-resolution loupe camera (e.g., Designs for Vision NanoCam 4K) during craniotomy cases, creating a video record of operative performance across PGY levels. LoupeCoach will apply computer vision to detect and track instruments and operative actions over time, enabling automated extraction of interpretable performance metrics such as: (1) time spent in key operative steps, (2) idle time, (3) number of instrument exchanges, (4) efficient use of one vs both hands, (5) error events (frequency/severity), and (6) indicators of independence within operative stages. These objective metrics will be paired with a structured human review of higher-level skills (smoothness, anticipation of needs, appropriate equipment selection), supporting a comprehensive, learner-centered feedback framework.
Prototype and feasibility: The hackathon prototype will demonstrate an end-to-end workflow: secure ingestion of loupe video, automated metric extraction on short segments, and a resident-facing dashboard that highlights time-stamped coachable moments for rapid review during monthly faculty-resident feedback sessions. Prior to this project, we performed proof-of-concept work that included the analysis of operative microscope video from six posterior fossa tumor resections using a surgical video analytic platform. This analysis measured tool movement, time spent using tools, and tool-crossing events, as well as computed an "efficiency score" for each surgery. We found the AI algorithm identified surgical instruments with a mean average precision (mAP50) of 0.988 for tools and 0.994 for retractors and efficiency scores between 0.6 and 1.0, with higher values indicating increased efficiency. The new prototype will capture video from a different source (surgical loupes) and focus on a small set of high-yield, reliably measurable metrics (instrument exchanges, bimanual usage proxies, step timing, and flagged events) to ensure technical soundness.
Contact
Braxton Morrison