Go with the flow: optimizing voice recognition to streamline workflow Jeffrey Chenoweth MD Saint Louis VAMC Kim Wilson MD Tucson VAMC
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Go with the flow: optimizing voice recognition to streamline
workflow
Go with the flow: optimizing voice recognition to streamline
workflow
Jeffrey Chenoweth MDSaint Louis VAMC
Kim Wilson MDTucson VAMC
Jeffrey Chenoweth MDSaint Louis VAMC
Kim Wilson MDTucson VAMC
Voice recognition to streamline workflow
• Jeffrey Chenoweth MD– Saint Louis VAMC
• Kim Wilson MD– Tucson VAMC
• Jeffrey Chenoweth MD– Saint Louis VAMC
• Kim Wilson MD– Tucson VAMC
Voice recognition to streamline workflow
• Driving forces behind VR
• Pushback – VR controversy
• VR development• Case study – VR
implementation• PACS setup and
reporting with VR– Kim Wilson MD
• Practical points for improving Radiologist workflow
• Future development of VR
• Driving forces behind VR
• Pushback – VR controversy
• VR development• Case study – VR
implementation• PACS setup and
reporting with VR– Kim Wilson MD
• Practical points for improving Radiologist workflow
• Future development of VR
Radiology reporting
• Basics unchanged in last century
• Product not images but report
• Communication to improve patient care
• Penultimate step in Radiology process
• Final step -- clinician action
• Basics unchanged in last century
• Product not images but report
• Communication to improve patient care
• Penultimate step in Radiology process
• Final step -- clinician action
If goals of reporting are unchanged why do we need VR
now?
Why VR?
• Absence of skilled transcriptionists?
• Transcription cost?– Probably not
• Absence of skilled transcriptionists?
• Transcription cost?– Probably not
Why VR?
• Improved report turnaround time
• Fewer report errors
• Improved report turnaround time
• Fewer report errors
Why VR?
• Transcription turnaround time
– Cassette tapes: week – 10 days
– Digital dictation: hours – 3 days
– VR: minutes• Decreases calls for
preliminary read• Clinicians expect
immediate report availability
• Transcription turnaround time
– Cassette tapes: week – 10 days
– Digital dictation: hours – 3 days
– VR: minutes• Decreases calls for
preliminary read• Clinicians expect
immediate report availability
Why VR?
• Improved report turnaround time
• Improved patient care
• Makes the Radiology report relevant
• Improved report turnaround time
• Improved patient care
• Makes the Radiology report relevant
Why VR?
• Problems with the traditional report correction editing process– Outside normal Radiologist workflow– Disruptive– Time consuming
• Problems with the traditional report correction editing process– Outside normal Radiologist workflow– Disruptive– Time consuming
Why VR?
• Error prone traditional report correction – editing process
– Time lag forgetfulness• Grammar checking vs.
content errors– Right – left errors– Date errors– DHCP blue screen daze
• After 50 + reports, how closely are you reading the report?
• Error prone traditional report correction – editing process
– Time lag forgetfulness• Grammar checking vs.
content errors– Right – left errors– Date errors– DHCP blue screen daze
• After 50 + reports, how closely are you reading the report?
Why VR?
• Report completion while image is in front of Radiologist
• Immediate error correction
• Once you’re done, you’re done
• Immediate report availability
• Report completion while image is in front of Radiologist
• Immediate error correction
• Once you’re done, you’re done
• Immediate report availability
VR controversy
VR controversy – Radiologist’s view
• Increased dictation time
• Increased error rate vs. good transcriptionist
• Removes focus on images
• Increased dictation time
• Increased error rate vs. good transcriptionist
• Removes focus on images
VR controversy – accuracy rate
• Is 95% acceptable?
• 90% of all reports have errors prior to sign off
• 10 % of reports have errors with transcriptionists
– J Digit Imaging Jun 2007
• Is 95% acceptable?
• 90% of all reports have errors prior to sign off
• 10 % of reports have errors with transcriptionists
– J Digit Imaging Jun 2007
VR economically justified?
• Decreased Radiologist productivity
– 50% longer dictation time– 24% shorter reports
– J Digit Imaging Jun 2007
• Decreased Radiologist productivity
– 50% longer dictation time– 24% shorter reports
– J Digit Imaging Jun 2007
VR – economically justified?
• Replacing lower paid transcriptionists with highly paid physicians
– Greater Radiologist productivity transcriptionists more cost effective than VR
– “… which course of action makes the most economic sense… is not always obvious.”
– JACR 2007; 4: 890
• Replacing lower paid transcriptionists with highly paid physicians
– Greater Radiologist productivity transcriptionists more cost effective than VR
– “… which course of action makes the most economic sense… is not always obvious.”
– JACR 2007; 4: 890
VR – two decades of controversy
• “Has considerable potential in the future… at present has limited function and definitely needs more technical improvement.”
– Radiology Nov 1988; 169: 580
• “… voice recognition systems are currently not ready for prime time.”
– JACR 2007; 4: 667
• “Has considerable potential in the future… at present has limited function and definitely needs more technical improvement.”
– Radiology Nov 1988; 169: 580
• “… voice recognition systems are currently not ready for prime time.”
– JACR 2007; 4: 667
VR – two decades of controversy
• “Speech recognition systems are used today in more than 1,000 radiology departments and are experiencing a growth rate typical of modern enabling technology.”
– JACR 2007; 4:670
• “Speech recognition systems are used today in more than 1,000 radiology departments and are experiencing a growth rate typical of modern enabling technology.”
– JACR 2007; 4:670
History of VR – a quarter century + of progress despite persistent
controversy
Evolution of Radiology reports
• Paper reports Electronic reports• Paper reports Electronic reports
Transcriptionist model – 1
• Tapes– Batch transcription– Batch correction,
signature
• Tapes– Batch transcription– Batch correction,
signature
Transcriptionist model – 2
• Digital transcription pool– In-line transcription– Batch correction,
signature
• Digital transcription pool– In-line transcription– Batch correction,
signature
Computer data acquisition systems
• Mark-sense forms
• GE RAPORT– AJR 1977; 128: 825
• Mark-sense forms
• GE RAPORT– AJR 1977; 128: 825
Computer data acquisition systems
• Microcomputers – CLIP Harvard
– Numeric codes for reporting– Radiology 1979; 133: 349
– Recognition of spoken numeric codes– Radiology 1981; 138: 585
• Microcomputers – CLIP Harvard
– Numeric codes for reporting– Radiology 1979; 133: 349
– Recognition of spoken numeric codes– Radiology 1981; 138: 585
True VR – Kurzweil system 1987
• Reported by several New England hospitals (including Boston VAMC)
• 1,000 word lexicon
• 5 sections by anatomy or subspecialty
– Radiology 1987; 164: 569.
• Reported by several New England hospitals (including Boston VAMC)
• 1,000 word lexicon
• 5 sections by anatomy or subspecialty
– Radiology 1987; 164: 569.
True VR – Kurzweil system 1987
• Able to dictate a report 88% of the time– 12% beyond scope of lexicon
• Use of macros
• Dictation time 20% longer
• Able to dictate a report 88% of the time– 12% beyond scope of lexicon
• Use of macros
• Dictation time 20% longer
True VR – Kurzweil system 1987
• Drawbacks
– Time and attention diverted from film analysis towards interaction with a monitor
– Increased dictation time proportional to degree of abnormality on film
– Problems with background noise– Problems with repeated interruptions– “Has considerable potential in the future… at
present has limited function and definitely needs more technical improvement.”
» Radiology Nov 1988; 169: 580
• Drawbacks
– Time and attention diverted from film analysis towards interaction with a monitor
– Increased dictation time proportional to degree of abnormality on film
– Problems with background noise– Problems with repeated interruptions– “Has considerable potential in the future… at
present has limited function and definitely needs more technical improvement.”
» Radiology Nov 1988; 169: 580
VR – state of the art 1999
– Error rate 30%
– Misrecognition of words
– Increased dictation timeRadioGraphics 1999; 19: 2.
– Error rate 30%
– Misrecognition of words
– Increased dictation timeRadioGraphics 1999; 19: 2.
VR – today
• Web architecture• Integration with
PACS– Improved efficiency– Decreased errors
• Improved recognition rates
• Decreased turnaround time
• Web architecture• Integration with
PACS– Improved efficiency– Decreased errors
• Improved recognition rates
• Decreased turnaround time
VR case study: Saint Louis VAMC
VR drivers
• PACS implementation– Fewer lost films – More reports
required
• PACS implementation– Fewer lost films – More reports
required
VR drivers
• CPRS implementation
• Universal availability of patient chart
• Clinical demand for faster reports
• CPRS implementation
• Universal availability of patient chart
• Clinical demand for faster reports
VR drivers
• Problem of preliminary reports– Clinical demand– Error correction– Legal issues
• Problem of preliminary reports– Clinical demand– Error correction– Legal issues
VR drivers
• Transcription problems– New contractor (low
bidder)– Cut and paste
errors– Variable
transcriptionist quality
• Transcription problems– New contractor (low
bidder)– Cut and paste
errors– Variable
transcriptionist quality
VR drivers
• Turn around time mandate– 90% completion in 48 hours
successful• 95% completion in 48 hours excellent
– Actual far less
• Turn around time mandate– 90% completion in 48 hours
successful• 95% completion in 48 hours excellent
– Actual far less
Analysis of options
• Hire more Radiologists– Full-time– Part-time
• Retired Radiologists• Fellows
– Recruiting difficulties
• Pay• Vacation
• Hire more Radiologists– Full-time– Part-time
• Retired Radiologists• Fellows
– Recruiting difficulties
• Pay• Vacation
Analysis of options
• Improve efficiency of reporting cycle
VR
• Improve efficiency of reporting cycle
VR
Proposal for VR system
• Strong administration support
• Support for VISN-wide solution– Some centers opted out
• Strong administration support
• Support for VISN-wide solution– Some centers opted out
System evaluation
• Radiologist input• Administration
– ADPAC– PACS coordinator– IT
• Literature review
• Radiologist input• Administration
– ADPAC– PACS coordinator– IT
• Literature review
System evaluation
• Vendor demonstrations– Radiologist trials
• Evaluation of administrator functions
• Vendor demonstrations– Radiologist trials
• Evaluation of administrator functions
Survey existing users
• Most sites only have experience with one system
• Hard to get good comparisons
• Your mileage may vary– Differences in
technical, administrative support for system
• Most sites only have experience with one system
• Hard to get good comparisons
• Your mileage may vary– Differences in
technical, administrative support for system
License issues
• Per unique user• Per workstation• Simultaneous users
vs. individual user
• Per unique user• Per workstation• Simultaneous users
vs. individual user
Vendor recommendation and selection
Planning
• Documentation review
• Site planning
• Documentation review
• Site planning
Results
• Report turnaround 90 – 95 % within 48 hours
• Cost savings
• Report turnaround 90 – 95 % within 48 hours
• Cost savings
VR implementation: lessons learned
• Plan, plan, plan• Plan, plan, plan
Lessons learned – project team
• Identify members– PACS administrator– Transcription
administrator– Editors– IT– Radiologist
• Dedication essential• Time consuming• Work closely with vendor• Read documentation
closely
• Identify members– PACS administrator– Transcription
administrator– Editors– IT– Radiologist
• Dedication essential• Time consuming• Work closely with vendor• Read documentation
closely
Lessons learned – conference calls
• Weekly calls
• Need everyone involved– IT– Administrators– Editors– Radiologist– Vendor
• Weekly calls
• Need everyone involved– IT– Administrators– Editors– Radiologist– Vendor
Lessons learned – conference calls
• Write questions in advance
• Keep minutes– Serves as a
resource– Complex project,
can’t remember everything
– Document to prevent misunderstandings
• Write questions in advance
• Keep minutes– Serves as a
resource– Complex project,
can’t remember everything
– Document to prevent misunderstandings
Lessons learned – installation issues
• Administrator training critical
• Get administrator manuals before vendor rep shows up
• Write questions in advance
• Take notes
• Administrator training critical
• Get administrator manuals before vendor rep shows up
• Write questions in advance
• Take notes
Lessons learned – test, test, test
• Test everything – don’t even think of implementation until this is done
• Vendor supplied checklist
• Test everything – don’t even think of implementation until this is done
• Vendor supplied checklist
Lessons learned – test, test, test
• Test system and test accounts
• Register procedure names and CPT codes
• Enter orders into Vista– Check request entry
into VR system
• Test system and test accounts
• Register procedure names and CPT codes
• Enter orders into Vista– Check request entry
into VR system
Lessons learned – test, test, test
• Dictate test reports– Check for proper
upload– Test addendums
and corrections– Input every type of
diagnostic code– Check parent and
descendants
• Dictate test reports– Check for proper
upload– Test addendums
and corrections– Input every type of
diagnostic code– Check parent and
descendants
Lessons learned – test, test, test
• Change orders• Minimum of 100 test patients• Test every Radiologist
– Include residents• Test sending to editor• Test telephony
• Change orders• Minimum of 100 test patients• Test every Radiologist
– Include residents• Test sending to editor• Test telephony
Lessons learned – Radiologist champion
• Change resistance• Promote system,
convince others that this is way to go
• Upfront buy-in from Radiologists
• Must see as improving patient care– vs. mandate from
administration
• Change resistance• Promote system,
convince others that this is way to go
• Upfront buy-in from Radiologists
• Must see as improving patient care– vs. mandate from
administration
Lessons learned – Radiologist champion
• Overcome objections– “I’m a physician not a transcriptionist!”
• Help others as one professional to another– Keep people going thru rough spots
• Need close communication with remainder of implementation team– Get feedback
• Overcome objections– “I’m a physician not a transcriptionist!”
• Help others as one professional to another– Keep people going thru rough spots
• Need close communication with remainder of implementation team– Get feedback
Lessons learned – Radiologist training
• Radiologists that have problems generally did not get good training
• Radiologists that have problems generally did not get good training
Lessons learned – Radiologist training
• Must have training schedule for every Radiologist
• Everyone has dedicated blocks for training, including follow-up– Minimum 4 hours with
trainer• Some may need more
attention
– Follow-up session
• Must have training schedule for every Radiologist
• Everyone has dedicated blocks for training, including follow-up– Minimum 4 hours with
trainer• Some may need more
attention
– Follow-up session
Lessons learned – Radiologist training
• Once trained, go cold turkey• Continuing support• Dealing with non-native English
speakers• Dealing with poor dictation styles• Refresher training
• Once trained, go cold turkey• Continuing support• Dealing with non-native English
speakers• Dealing with poor dictation styles• Refresher training
Lessons learned – site trainer training
• Must learn to train new users
• Critical if residents involver
• Individual training• Sit in on user
training sessions
• Must learn to train new users
• Critical if residents involver
• Individual training• Sit in on user
training sessions
Lessons learned – continual QC
• Continual effort and vital for long-term success
• Test plan• Test telephony• Dummy orders uploading
• Continual effort and vital for long-term success
• Test plan• Test telephony• Dummy orders uploading
Lessons learned – continual QC
• Pull real reports and monitor for errors
• Intervention if needed
• Retraining of dictator
• Rebuild voice model
• Pull real reports and monitor for errors
• Intervention if needed
• Retraining of dictator
• Rebuild voice model
Lessons learned – continual QC
• Look in CPRS– Report text ok– E-signature ok– Diagnostic codes
• Look in CPRS– Report text ok– E-signature ok– Diagnostic codes
Lessons learned – continual QC
• Monitor continually– Uploads– Orphan dictations– Exams without
reports
• Monitor continually– Uploads– Orphan dictations– Exams without
reports
Lessons learned – continual QC
• Need support contract
• Keep contacts handy
• Know who to call
• Need support contract
• Keep contacts handy
• Know who to call
Lessons learned – trouble log
• Take notes for every trouble call to vendor
• Resource to fix problems on your own– Record
• Day• Ticket #• Who spoke to• Problem• How it was resolved• Note recurring problem
• Take notes for every trouble call to vendor
• Resource to fix problems on your own– Record
• Day• Ticket #• Who spoke to• Problem• How it was resolved• Note recurring problem
Lessons learned – backup plan
• Backup VR server?
• Utilize another transcription contract?
• Other medical center?
• Backup VR server?
• Utilize another transcription contract?
• Other medical center?
PACS setup and reporting with VR
• Kim Wilson MD– Tucson VAMC
• Kim Wilson MD– Tucson VAMC
Radiologist workflow: practical points
Goals
• Increase dictation efficiency
• Maximize eyes on image
• Increase dictation efficiency
• Maximize eyes on image
Transcription models – read, edit, done
• Highly recommended
• Minimize turnaround• Make corrections
while image is in front of you
• Once it’s gone you don’t have to deal with it again
• Highly recommended
• Minimize turnaround• Make corrections
while image is in front of you
• Once it’s gone you don’t have to deal with it again
Transcription models – batch correct, sign
• Most efficient work flow state?
• Longer turnaround
• Error correction more difficult– Right – left– Dates
• Most efficient work flow state?
• Longer turnaround
• Error correction more difficult– Right – left– Dates
Transcription models – editor• Not recommended
• Transforms transcriptionist model correctionist
• Inefficient, expensive• Maximum turnaround
time• Must remember to
correct and sign reports
• Editor errors• When is it helpful?
– Non-native English speakers?
– Poor dictation technique
• Not recommended
• Transforms transcriptionist model correctionist
• Inefficient, expensive• Maximum turnaround
time• Must remember to
correct and sign reports
• Editor errors• When is it helpful?
– Non-native English speakers?
– Poor dictation technique
Transcription styles
• Free dictation
• Templates and macros
• Free dictation
• Templates and macros
Free dictation
• Advantage– Keeps eyes on
image
• Disadvantage– More time with
editing and corrections
• Advantage– Keeps eyes on
image
• Disadvantage– More time with
editing and corrections
Templates – advantages
• Improved time savings
• Improved report accuracy
• Consistent report structure– Personally– Across department– Need agreement among radiologists
• Facilitates structured reporting– BIRADS
• Improved time savings
• Improved report accuracy
• Consistent report structure– Personally– Across department– Need agreement among radiologists
• Facilitates structured reporting– BIRADS
Templates – disadvantages
• Takes eyes of the image
• May forget to delete non-relevant text
• Takes eyes of the image
• May forget to delete non-relevant text
Templates
• Especially useful for repetitive boilerplate– Biopsy– Angiography
• Especially useful for repetitive boilerplate– Biopsy– Angiography
• The patient was placed on the CT table in [<supine> ] position.
• Initial scans were obtained to localize the [ ].
• An appropriate site at the [ ] was marked.
• The patient was prepped and draped in the usual sterile manner. Local anesthesia was achieved with infiltration of 1% Xylocaine.
• The patient was placed on the CT table in [<supine> ] position.
• Initial scans were obtained to localize the [ ].
• An appropriate site at the [ ] was marked.
• The patient was prepped and draped in the usual sterile manner. Local anesthesia was achieved with infiltration of 1% Xylocaine.
Template approaches
• Few general reports– Fill in the blanks– Default fill in the
blanks
• Many specific reports
• Few general reports– Fill in the blanks– Default fill in the
blanks
• Many specific reports
• Case [ ]. [ ]• There is no evidence
of fracture, dislocation, or bony destruction.
• [<The joint spaces are within the limits of normal.>]
• [ < >]• Impression:• [<Negative
examination.>]
• Case [ ]. [ ]• There is no evidence
of fracture, dislocation, or bony destruction.
• [<The joint spaces are within the limits of normal.>]
• [ < >]• Impression:• [<Negative
examination.>]
Templates– itemized reports
– Lungs: [<normal.>]– Pleura: [<normal.>]– Mediastinum [<normal.>]– Hila: [<normal.>]– Other: [< >]– Comparison: [<None.>]
– Impression: [<normal>]
– Lungs: [<normal.>]– Pleura: [<normal.>]– Mediastinum [<normal.>]– Hila: [<normal.>]– Other: [< >]– Comparison: [<None.>]
– Impression: [<normal>]
Template tricks
• Standardize template naming convention
• Modality body part side, technique
• Standardize template naming convention
• Modality body part side, technique
Template tricks
• Make template easy to change on the fly
• Liberal use of paragraphs
• Make template easy to change on the fly
• Liberal use of paragraphs
• Case [ ].• Ultrasound abdominal aorta.
• Real-time ultrasound examination of the abdominal aorta was obtained in transverse and longitudinal projections.
• The patient [<does not have an>] abdominal aortic aneurysm.
• The abdominal aorta measures [ ] cm in maximal diameter.
• [< >]
• Impression:
• [<The patient does not have an abdominal aortic aneurysm.>]
• Case [ ].• Ultrasound abdominal aorta.
• Real-time ultrasound examination of the abdominal aorta was obtained in transverse and longitudinal projections.
• The patient [<does not have an>] abdominal aortic aneurysm.
• The abdominal aorta measures [ ] cm in maximal diameter.
• [< >]
• Impression:
• [<The patient does not have an abdominal aortic aneurysm.>]
Dictation technique
• Fast ok• Must be clear and
distinct– Think before
speaking– Know what you
want to say– No filler sounds
• Fast ok• Must be clear and
distinct– Think before
speaking– Know what you
want to say– No filler sounds
Dictation technique
• Speak in phrases– Get a flow– Correct in phrases rather than individual
words
• Use complete sentences• Use paragraphs liberally
• Speak in phrases– Get a flow– Correct in phrases rather than individual
words
• Use complete sentences• Use paragraphs liberally
Dictation technique
• Consistent style
• Keep reports short
• Don’t number items in impression
• Consistent style
• Keep reports short
• Don’t number items in impression
Dictation technique
• Dictate – then correct– Keep eyes on image
• Read and correct reports carefully before signing
• Dictate – then correct– Keep eyes on image
• Read and correct reports carefully before signing
Microphones
• Proper location
• Headsets?
• Proper location
• Headsets?
Environment
• Noise control
• Bullpen disruption
• Noise control
• Bullpen disruption
Monitor layout
• Open window in admin monitor– Don’t continually
check transcription– Dictate then edit
• Pop-up in admin monitor
• Open window in admin monitor– Don’t continually
check transcription– Dictate then edit
• Pop-up in admin monitor
Monitor layout
• Separate monitor?– VR– CPRS– Internet window –
Google– Decision support
software?– Teaching file
software?
• Separate monitor?– VR– CPRS– Internet window –
Google– Decision support
software?– Teaching file
software?
Training for problem words
• Case number December
• Pulmonary bony
• Adrenal no renal
• Case number December
• Pulmonary bony
• Adrenal no renal
Gotchas
• Impression:
• Dictate case number in every report– Troubleshooting
• Impression:
• Dictate case number in every report– Troubleshooting
Gotchas
• How reports look in VR may not be how report looks in PACS, Vista, or CPRS– Line spacing– New lines vs. paragraphs
• How reports look in VR may not be how report looks in PACS, Vista, or CPRS– Line spacing– New lines vs. paragraphs
Residents
• Pre-dictation by resident
• Make corrections and finalize report at time of checking
• Easy sign-off by attending
• Drawbacks – templating– May not learn elements of
a good report
• Pre-dictation by resident
• Make corrections and finalize report at time of checking
• Easy sign-off by attending
• Drawbacks – templating– May not learn elements of
a good report
Success rules for VR
• You must want system to work
• Training the VR vs. VR training you
• Rule of thirds
• You must want system to work
• Training the VR vs. VR training you
• Rule of thirds
Future development of VR
Improved recognition engines
• Better accent recognition
• Better recognition of small words
• Better accent recognition
• Better recognition of small words
Improved integration of PACS, HIS-RIS
• Too easy to mark case as read when not• Too easy to mark case unread when is
read• Too easy to hang up report
– Impression:
• Too easy to forget to sign off on report
• Too easy to mark case as read when not• Too easy to mark case unread when is
read• Too easy to hang up report
– Impression:
• Too easy to forget to sign off on report
Improved grammar checking
• There• Their• They’re
• Two• Too• To
• Capitalization
• There• Their• They’re
• Two• Too• To
• Capitalization
Structured reporting
• Standard lexicons
• Universal framework for reports– Improve readability– Minimize style
variation between Radiologists
– Data mining
• BIRADS
• Standard lexicons
• Universal framework for reports– Improve readability– Minimize style
variation between Radiologists
– Data mining
• BIRADS
Seamless integration of communication
• Clinical alerts
• Paging for critical findings
• Feedback to technologist, QA supervisor
• Clinical alerts
• Paging for critical findings
• Feedback to technologist, QA supervisor
Ultimate VR goal: universal recognition
Outlook Mailgroup
• VHA Radiology Voice Recognition
• VHA Radiology Voice Recognition