Chapter 4: Quality Assurance of Objective Structured Clinical Examinations

CHAPTER HIGHLIGHTS

ORIENTATION TO THE CHAPTER

Objective Structured Clinical Examinations (OSCEs) are widely used to assess clinical, communication, and practical skills in medical education, especially in high-stakes contexts. The quality assurance of an OSCE is therefore of critical importance to ensure that such tests are fair, robust, and defensible. OSCEs are highly complex to design and implement, and many different aspects have to be scrutinized. We have taken a specific approach to quality assuring OSCEs, using what we refer to as the Kane Validity Framework, because this provides a holistic and systematic framework to use when evaluating the quality of an OSCE. This chapter begins by setting the context of OSCEs within a learning and competence taxonomy, then moves onto describing the features of the Kane Validity Framework. The next section of the chapter describes the application of each section of the Kane Validity Framework in relation to OSCEs. Templates are provided for use when working through the different aspects of quality assuring OSCEs. Other quality assurance mechanisms and issues are described, including the special circumstances that the COVID-19 pandemic has placed on the conduct of OSCEs, at the time of writing. The chapter concludes with a summary of how quality assurance of OSCEs should be conducted in a systematic and holistic manner to ensure that defensibility, credibility, and fairness are maintained and signposts show opportunities for further assessment enhancement.

INTRODUCTION

Within a frequently used learning and competence taxonomy that is applied to clinical competence [1], OSCEs align with the “shows how” level of clinical competence in Miller’s pyramid. (See Figure 4-1.) As such, OSCEs are an assessment method fit for purpose when assessing a student’s or trainee’s (or a candidate’s) ability to perform particular clinical skills. As a consequence, OSCEs are currently regarded as the best format for integrated, high-stakes summative assessment of clinical competence using a single test format. OSCEs typically assess across a range of clinical skill domains—for example, physical examination, consultation and communication skills such as consent, clinical reasoning, and technical or procedural skills [2].

Although there is extensive literature about OSCEs, there is limited work that provides practical advice on how to quality assure an OSCE. When approaching quality assurance of an OSCE, several factors should be scrutinized in a systematic manner so that evaluation of the quality of the test or examination in terms of validity, reliability, and fairness is a consistent and coherent exercise. This more holistic approach has evolved ...