“Unlocking the Future of Operative Dictation through Generative AI: a Comparative Pilot Study using a Whisper based model architecture on Operative Report Dictation"

Introduction: The advent of ChatGPT and Generative AI has significantly impacted neurosurgical research and patient care. An often-overlooked partner in this AI synergy is WhisperAI, an open-source generative speech-to-text model trained on extensive datasets. WhisperAI uniquely combines generative AI and speech recognition to flawlessly transcribe intricate speech into text, pushing the boundaries of what was previously possible. The current study provides a proof of concept utilizing a novel generative AI architecture based on the Whisper model, achieving near-perfect dictation results comparable to industry-standard software, all without the need for additional fine-tuning or validation. The study's overarching aim is to propose an innovative open-source framework, empowering neurosurgeons with tools on par with industry leaders, freely accessible to all.

Methods: Ten randomly selected operative reports (5 spine, 5 other) from a single institution were obtained for analysis. Each report was dictated using voice recording software and further run through our model for speech-to-text transcription. Word-Error-Rate (WER) was further calculated using python programming language and an algorithm plugin for each of the three dictations. Each recording was further transcribed through Dragon speech-to-text algorithm for comparison.

Results: Results of the calculated WER are shown for each dictation run in Table 1. Average WER for our algorithm was 7.08% with a median of 6.86%. When comparing results to Dragon, the average WER was worse (9.08%) suggesting on-par performance of generative AI methods to traditional text to speech. WER for spine dictations performed slightly worse than other general neurosurgical procedures (WER of 7.7% vs 6.8% respectively), possibly related to the increased propensity for niche instruments, names and tools in spine surgery. Both subsets performed better than Dragon ( WER of 7.7% vs 10.8% for Spine, 6.8% vs 7.9% for other), with Spine showing the biggest increase in accuracy over Dragon (2.52%). Further recording length for each dictation and words-spoken per minute (WSPM) are noted in Table 1.

Conclusion : Whisper AI based generative text-to-speech algorithms provide promising results comparable, if not superior, to traditional commercial speech-to-text models without any additional fine-tuning. Further exploration is necessary in order to best understand the capabilities and abilities to improve clinical practice with such new technology.