Directives to improve patient outcomes and enhance safety within the healthcare system have led to development of technologies to assist practitioners in a variety of activities. The purpose of this study was to explore and evaluate a method for examining the effect of computer-assisted decision making (CADM) using a handheld device on the accuracy (ie, correct diagnosis and treatment) and speed of problem solving by Certified Registered Nurse Anesthetists (CRNAs) during simulated critical patient-care events. A randomized crossover design with matched-pair sampling was used. In a high-fidelity human simulation environment, 4 CRNAs participated in 2 plausible critical anesthesia case scenarios. CRNA performance with and without CADM technology, environmental authenticity, and reliability and validity of data collection tools and simulated case scenarios were evaluated. Time to correct diagnosis and treatment varied by scenario, taking less time with CADM for one but more with CADM for the other, likely due to differences in pace, intensity, and conduct of the 2 scenarios. We believe this study supports further exploration and application of CADM in complex patient scenarios involving anesthesia practitioners. Affirmation of environmental authenticity also validates the high-fidelity human simulation environment as an appropriate setting to conduct research in this area.