Developing High-Quality, Adaptive Educational Content Using Large Language Models
This abstract was part of the 2026 UCSF AI and Education Symposium
Background & Problem:
Vascular surgery trainees rely heavily on standardized question banks—particularly SESAP, SCORE and VSITE materials—to master core knowledge and clinical reasoning. However, explanations within existing resources vary widely in clarity, depth, and clinical relevance.
Trainees often struggle to understand why an answer is correct, and current question banks lack adaptability to individual learning needs.
My previous research demonstrated that large language models (LLMs) can outperform traditional human-generated explanations in both accuracy and pedagogical quality. In the previous study evaluating AI performance on vascular clinical questions, we found that AI produced more comprehensive explanations and more clinically coherent reasoning compared with human-written rationales. These findings support the potential of AI tools to meaningfully enhance trainee education.
Project Goal:
To develop an AI-driven vascular surgery question bank designed specifically to improve reasoning, conceptual understanding, and exam preparedness for VSITE/ABSITE-style standardized assessments.
Methods & Innovation:
This project will use LLMs (e.g., GPT-5.2 or similar clinical fine-tuned models) to generate high quality practice questions mapped to the core VSITE vascular curriculum. The system will:
- Generate New Questions
LLMs will create multiple-choice and short-answer questions covering the major vascular domains (arterial, venous, dialysis access, trauma, imaging, perioperative care).
Questions will be tagged by topic, cognitive level, and difficulty.
2. Produce Explanations & Teaching Points
For each question, AI will generate:
A stepwise reasoning process
Teaching pearls
References (SVS guidelines, UpToDate, landmark trials, etc.)
3. Expert & Trainee Validation System
Vascular surgery faculty and trainees will score each question on:
Accuracy
Educational quality
Clinical fidelity
Appropriateness for VSITE preparation
Questions with high average ratings will form the final curated bank.
Low-performing items will be refined iteratively using reinforcement prompts.
4. Future Expansion: Adaptive Learning Platform
After establishing the validated bank, we plan to develop:
Adaptive question sequencing based on trainee weaknesses
Benchmarked learning curves
AI-generated dashboards for faculty to monitor progress
A longitudinal study comparing AI-augmented learning vs standard practice
Impact:
This project aims to create the first AI-validated vascular surgery question bank, improving access to high-quality educational materials and addressing variability in reasoning found in current question banks. The tool will support UCSF trainees and potentially scale nationally to vascular training programs.
This aligns directly with the UCSF AI & Education Symposium mission: to leverage AI for high-value, learner-centered innovation in medical education.
Contact
Becky Long, [email protected]