Background/context
Assessment and feedback on practical clinical skills is a key aspect of education and training for graduates in allied health. However, these assessments are inherently subjective and moderation of quantitative marking is currently poorly achieved. Subtle differences in techniques may be attributed importance and lead to different marking of outcomes, and students often express frustration with multiple markers using ‘different’ standards. Attempts to increase reliability use standards based assessment, but the written communication of their practical meaning, and examiners customary approaches provide barriers to the success of these interventions.
Research/evaluation method
In a large, first-year undergraduate unit we created a multi-aspect, multimedia marking rubric to guide the multiple assessors (>10) responsible for practical skills examination. Videos were shot of ‘exemplar’ student achievement in exams, with current recommended teaching of techniques demonstrated. Editing highlighted the moments which achieved marks delineated in the written marking rubric, and included notation on set up and variations which were acceptable. In addition, an info-graphic of common errors was produced. This resource acted as a reminder to examiners of expectations, as well as a method of feedback for students.
Outcomes
To our knowledge this is the first example of multimedia rubrics in the University, and potentially in the country. Written rubrics, even when combined with staff discussion are open to individual interpretation. In brief questionnaires to examiners before and after watching the videos 30% of staff stated they were ‘completely confident’ to assess after reading the written rubric, but indicated they would not have given full-marks to the student in the ‘exemplar exam’ of the video. This is the precise problem that previously was undetected in moderation pre-exam and emerged when students complained about disparities. The video rubrics were successful in improving consistency and expectation between markers in a notoriously difficult exam to moderate.
Addressing the theme/s of the Conference (up to 200 words recommended)
Exploiting emerging technologies to enable employability.
In allied health, confidence in effective and correct technique of practical clinical skills is critical to graduate employability. Higher education teaching must therefore attend to reliable teaching and assessment of such skills. Yet the inherent variability of marking and teaching from examiners own experience, knowledge and values adds uncertainty and frustration to student learning experience. By exploiting technology to create multimedia marking rubrics and infographic feedback sheets, we are stepping towards a more stable and consistent teaching and assessment experience, which will lead to greater confidence in our graduates. This project is especially important in large junior units which often involve high numbers of different assessors for the same skills. Critically, this early experience for students must be as consistent as possible to allow confident progression.