Is our assessment of resuscitation skills robust? Examining the validation process

Purpose: Just as a chain is only as strong as its weakest link, so too is assessment only as robust as the tool used to examine performance. We sought to assess paramedic students insert a laryngeal mask airway (LMA) and intraosseous (IO) device in a simulated setting in a randomised controlled trial. We searched the literature for validated assessment tools for these two skills, and after finding none suitable to this context, we developed one. Methods: We invited expert pre-hospital clinicians (retrievalists and intensive care paramedics) to participate in a modified Delphi study1 to investigate expert consensus for LMA and IO application in the pre-hospital setting. Two rounds of data collection were performed. Results: Using this procedure as reported in the literature, consensus was achieved for IO (Cronbach's α = 0.725) but not for LMA (Cronbach's α = 0.494). Conclusions: A critique of the Delphi method challenges (1) Cronbach's α as the accepted coefficient2 in reporting data agreement, (2) the current understanding of “consensus”, (3) whether a Delphi method is the best approach to determine consensus for all types of skills, and (4) the task of validation as a single, finite activity.3

Resuscitation

Resuscitation

106

1873-1570

N/A

Supplement 1

1