Is our assessment of resuscitation skills robust? Examining the validation process

Seymour-Walsh, Amy;Worley, Paul;Vnuk, Anna;Grantham, Hugh

Abstract

Purpose: Just as a chain is only as strong as its weakest link, so too is assessment only as robust as the tool used to examine performance. We sought to assess paramedic students insert a laryngeal mask airway (LMA) and intraosseous (IO) device in a simulated setting in a randomised controlled trial. We searched the literature for validated assessment tools for these two skills, and after finding none suitable to this context, we developed one. Methods: We invited expert pre-hospital clinicians (retrievalists and intensive care paramedics) to participate in a modified Delphi study1 to investigate expert consensus for LMA and IO application in the pre-hospital setting. Two rounds of data collection were performed. Results: Using this procedure as reported in the literature, consensus was achieved for IO (Cronbach's α = 0.725) but not for LMA (Cronbach's α = 0.494). Conclusions: A critique of the Delphi method challenges (1) Cronbach's α as the accepted coefficient2 in reporting data agreement, (2) the current understanding of “consensus”, (3) whether a Delphi method is the best approach to determine consensus for all types of skills, and (4) the task of validation as a single, finite activity.3

Journal

Resuscitation

Publication Name

Resuscitation

Volume

106

ISBN/ISSN

1873-1570

Edition

N/A

Issue

Supplement 1

Pages Count

Location

N/A

Publisher

Elsevier

Publisher Url

N/A

Publisher Location

N/A

Publish Date

N/A

Url

N/A

Date

N/A

EISSN

N/A

DOI

10.1016/j.resuscitation.2016.07.144