Pecha Kucha Higher Education Research and Development Society of Australasia 2015

Filling the learning analytics gap for high-stakes exams (#319)

Mathew Hillier 1
  1. University of Queensland, St Lucia, QLD, Australia


Drawing on the education, information technology and analytics literature, we argue that a combination of e-exams and learning analytics could be a powerful tool in enhancing our understanding of student behaviour, strategies and performance in examinations.

The rapid rise in the use of learning analytics has been highlighted by the work of Dawson and McWilliam (2008) and the recent report by Siemens, Dawson and Lynch (2013) that is based on the 'big data' available from a plethora of sources within the institution. The potential for insight by drawing in data from student information and learning management systems mean that a great deal can be learnt about the way students are engaging and performing on formative assessments during the semester. A wide range of digital learning activities are now available for formative assessment that includes digital patients (Newby et al. 2011), virtual microscopy (Kumar et al. 2009), simulated conversations (Nelson & Dawson 2014), virtual history excursions (Matthews & Agutter 2014), simulated practicums (Gregory, et al. 2013) and virtual immersive foreign language learning (Grant et al 2013) that can provide data for analysis. However, when it comes to the typical paper based exam there is very little in the way of data points that can contribute to such analysis. Further it is currently rare for students to receive comprehensive feedback on how they have performed in examinations beyond a 'pass or fail' and similarly very little analytics of the exam questions themselves is performed. E-exams offer a way 'fill the data gap' with respect to our knowledge of student performance and behaviour in high stakes assessments.



The potential for learning analytics and computerised high stakes exams is currently an untapped resource in assessing, evidencing and evaluating graduate capabilities. The contemporary workplace and social sphere is characterised by the high availability of information and a sophisticated range of ICT tools. Contemporary ICT enhanced formative learning activity does provide a means approximate the 'real world' however we are still left with the problem of authentication. The paper-based, high stakes exam remains as the primary means by which universities can be confident that the work submitted belongs to student being assessed. However a paper-based medium no longer reflects the problem solving environment that graduates will encounter in professional practice. This results in a disconnect between the desired set of attributes graduates will be expected to hold and our ability to validate and assess their capabilities. Further, the use of learning analytics in higher education is increasing as a means for understanding student learning behaviour, progress, engagement and success (or otherwise) as well as a means to improving learning delivery. However, the paper-based exam represents a significant information gap with regard to the data required to complete a more comprehensive picture.

  1. Dawson, S. & McWilliam, E. (2008) Investigating the application of IT generated data as an indicator of learning and teaching performance, Australian Learning and Teaching Council. Retrieved from
  2. Siemens, G., Dawson, S. & Lynch, G. (2013), Improving the quality and productivity of the higher education sector, Australian Government Office for Learning and Teaching, Retrieved from
  3. Newby, D., Jin, J., Summons, P., Athauda, R., Park, M., Schneider, J., Kable, S., Marriott, J., Duncan, G., Simpson, M. & Xu, R. (2011) Final report: Development of a computer-generated digital patient for teaching and assessment in pharmacy [CG7-431], Australian Learning and Teaching Council. Retrieved from
  4. Kumar, R., Velan, G. de Permentier, P., Adam, P., Bonser, S. & Beal, M. (2009) Final Report: Virtual microscopy for enhancing learning and teaching [CG7-398]. Australian Learning and Teaching Council. Retrieved from
  5. Nelson, R. & Dawson, P. (2014) Final Report: Assessment as learning through conversation simulation [SD12-2258], Australian Government Office for Learning and Teaching. Retrieved from
  6. Matthews, C. & Agutter, K. (2014) Final Report: Getting A MUVE On: Developing Web 2.0 Curricula in the Australian Humanities [PP9-1392], Australian Government Office for Learning and Teaching. Retrieved from
  7. Gregory, S., Dalgarno, B., Crisp, G., Reiners, T., Masters, Y., Dreher, H. & Knox (2013) Final report: VirtualPREX: Innovative assessment using a 3D virtual world with pre-service teachers [PP10-177], Australian Government Office for Learning and Teaching. Retrieved from
  8. Grant, S., Huang, H., Pasfield-Neofitou, S. & Lan, Y. (2013) Enhancing tertiary Chinese foreign language learning through task - based language learning in 3D multiuser virtual environments, [SD13-3445] in "2013 Successful Grants Round 2", Australian Government Office for Learning and Teaching. Retrieved from
Download Presentation