Evaluation of mini-clinical evaluation exercise (mini-CEX) as a method for assessing clinical skills in anaesthesia postgraduate education

Abstract

INTRODUCTION Clinical skills are most essential for patient care; however, the assessment of clinical competencies can be challenging and complex. Currently, most traditional assessment methods for postgraduate students (PGS) used in most medical colleges in India (yearly written essay type questions and summative ratings including oral case-based discussions and table viva voce) are summative and try to find out how much the PGS gained knowledge rather than their clinical ability.[1] Nowadays, there is a trend toward competency-based training with emphasis on formative rather than summative assessment. In India, the new undergraduate (UG) curriculum is competency- based. Workplace-based assessment (WPBA) is integral to programmatic assessment in a competency-based curriculum.[2] WPBA involves two things: (1) Direct observation of clinical and non-clinical skills in real situations especially the workplace.(2) Proper feedback by evaluator to trainee. Timely feedback contributes to student learning and improvement and is the heart of medical education. It is said that 80% PGS in India do not receive any feedback. There are several methods of WPBA. Mini-clinical evaluation exercise (mini-CEX) is one method of formative assessment and WPBA. In mini-CEX, an expert, usually a faculty member observes the actual encounter of the trainees with the patient at the workplace, rates their history taking, physical examination and counselling skills and provides feedback to them.[3] There have been some studies in western countries and a few from India on the use of mini-CEX in PG medical education. Few non-anaesthesia related studies on this topic have been published from India.[456] The mini-CEX has been adopted into anaesthesia training programmes in some countries.[278] We conducted a study on PGS in anaesthesia with an aim to evaluate mini-CEX as a method for assessing clinical skills in anaesthesia PG education. Our primary objectives were to assess the perceptions of PGS and faculty members regarding mini-CEX as an assessment/learning tool in Anaesthesiology and Critical Care. The secondary objectives were to introduce, sensitise, and train our faculty and PGS in the use of mini-CEX. METHODOLOGY Institutional ethics committee approval was obtained. The cross-sectional study took place from November 2019 to March 2020 in the Department of Anaesthesiology at a government medical college in our country. The study population included PGS and faculty members. Written informed consent was taken from the participants. Initially, the students and faculty were sensitised by us regarding mini-CEX with the help of a short talk and a video session on how the exercises would be conducted and what was expected out of them. Trainee-patient encounters with different topics and scenarios were planned in anaesthesiologist-specific workplaces : pre-anaesthesia examination (PAE) clinic, the pain and palliative care clinic, postoperative ward, and the surgical intensive care unit (ICU) [Table 1]. The lists of faculty and student names were displayed on the notice-board and in faculty and PGS Whatsapp groups. In an encounter, the PGS was given an exercise on history taking, physical examination, and counselling. The student was instructed beforehand that he or she was being observed by the faculty member, that his performance was being scored and the time allotted to him. Observation by the faculty was done for 15 min and data was entered in the mini-CEX form [Annexure A]. This was followed by an oral feedback for 5 min by the observer faculty to the student. The assessment was thus done in routine working conditions which is not usually done at assessment during standard viva examinations. At the end of the encounter, a prevalidated questionnaire was given to the student regarding the conduct of mini-CEX [Annexure B]. At the end of the study, the participating faculty and PGS were given questionnaires dealing with their perceptions on the value and acceptance of mini-CEX as an assessment tool in anaesthesiology. The questionnaires were based on existing studies investigating faculty and student views on mini-CEX.[5910] The PGS maintained anonymity while filling the responses. The questionnaires were prevalidated by subject experts as well as experts in medical education from the authors’ institutes.Table 1: Examples of clinical encounters that were used for mini-clinical evaluation exercisesQuantitative data from the responses to the questionnaires were analysed by using the Statistical Package for the Social Sciences (SPSS) version 20 (International Business Machine, United States). RESULTS Twenty-nine faculty-PGS encounters including 20 faculty members and 27 PGS (11: 1st year, 14:2nd year, 2: 3rd year) took place. The total number of PGS exhibiting satisfactory, superior, and unsatisfactory scores for different clinical skills was calculated. As regards the conduct of mini-CEX, 100% PGS felt that the topics selected were useful, pre-encounter instructions given were clear and adequate, useful feedback was given and the exam was well- coordinated and organised. 52% PGS felt that the examiner was intimidating, 15% felt that personality and gender did affect the conduct of mini-CEX, 2% encountered multiple disturbances during the exercise, 2% felt that the time allocated for the exercise was not sufficient, and 8% felt that the questions asked were not relevant. A majority of the students had positive perceptions regarding various aspects of mini-CEX as an assessment tool in anaesthesia [Figure 1a]. The questionnaire on faculty perceptions regarding mini-CEX as an assessment tool revealed positive responses for a majority of the questions except for the question asking whether mini-CEX eliminates examiner bias [ [Figure 1b]. The mean satisfaction score with mini-CEX was 7.28 (range 5-9) for faculty and 8.08 (range 6-10 ) for PGS. Scores were superior in 60% PGS for clinical judgement, 62% PGS for counselling skills, 44% PGs for organisation and efficiency, 62% PGS for professionalism and 60% PGS for overall clinical competence.Figure 1: Perceptions regarding mini-CEX as an assessment tool in anaesthesiology (a) Student perceptions, (b) Faculty perceptionsDISCUSSION The responses and scores show that a majority of the PGS and faculty in our study were satisfied with the use and conduct of mini-CEX as an assessment tool in anaesthesia. This shows that they perceived mini-CEX as an effective and efficient learning tool in anaesthesiology. A meta-analysis has demonstrated the positive effects of mini-CEX on trainee performance.[11] The one-to-one interaction with faculty in authentic clinical settings, case diversity, and multiple encounters with different assessors in mini-CEX has already been welcomed by residents and faculty in specialities other than anaesthesia.[456] Mini-CEX has been perceived in another study to have very positive educational impacts and relative feasibility among anaesthesia trainees.[12] The Mini-CEX assessment tool has several weaknesses too including creation of teacher-student friction, demoralisation of students, variable assessor stringency, assessor subjectivity, and case-specificity.[12] The students in our study were a bit nervous, anxious, and hesitant when they came to know that their chance had come for the encounters; however, this was only for the first few encounters. Joshi et al. have also reported nervousness while examining cases in some of their study population.[10] Castanelli et al. found some difficulties in finding time to schedule assessments and deliver timely feedback in busy clinical workplaces.[2] We had planned and organised the encounters in such a way that no busy work schedule was hampered. Our study has some limitations. The number of encounters is less and it is a single-centre study; nevertheless, based on this pilot study, we propose to conduct a larger study in the future. We conclude that mini-CEX is an effective formative assessment tool of potential value to assess clinical skills in anaesthesiology. It can be incorporated into anaesthesia PG education curriculums in India and elsewhere. Financial support and sponsorship Nil. Conflicts of interest There are no conflicts of interest.

Topics & Concepts