INBETA updated

The one about Pass Marks

Hi there

I started working as an academic in 2009 and from day 1 I was hooked on the idea that I could do things in the classroom that might help students to learn better. I spent years working on learning tasks, experimenting with different tools, changing my assessment plans, reading the literature, attending conferences, and talking to like-minded colleagues.

I paid attention and I thought about pedagogical problems all the time but it took me almost 10 years to question the fundamental assumption that a passing grade should be a fixed percentage (i.e. 50%) across all assessment tasks.

If you give this just a few seconds' thought you'll recognise the following problems:
  • Not all assessments are the same.
  • Not all assessors are the same.
  • Not all patients are the same.
How then, is it possible that all pass marks can be the same? In this short newsletter we share two resources that will help you approach your assessment tasks in a different way. If nothing else they should get you thinking about how seriously we should be taking the completely arbitrary pass mark of 50% (hint: not seriously at all).

If you enjoy these newsletters please share them with friends and colleagues, mention them on the interwebs, or print massive posters and put them up at work. And, it's always nice to know that they're not simply being destroyed by your spam filter, so let us know what you think.

FOR LISTENING

Challenges with setting standards setting

Schoeman, S. (2017). Standard setting. SAAHE podcast.


In this episode I discuss the issue of standard setting with Scarpa Schoeman. Scarpa has published a variety of peer-reviewed articles and presented at international conferences on the topic of medical education and assessment. His research interests include assessment and standard setting, as well as the educational environment for medical students. His clinical interests and practice focus on Emergency Medicine. He also acts as Assessment consultant to the Colleges of Physicians, Obstetrics and Gynaecology, Paediatricians and Anaesthetists of South Africa. He is a Fellow of the Higher Education Academy in the United Kingdom and is a part-time tutor in Assessment and Standard setting for the CME at Dundee University.

FOR READING

An overview of standard setting methods

Norcini, J. J. (2003). Setting standards on educational tests. Medical Education, 37(5), 464–469.

Objective: This instalment in the series on professional assessment provides an introduction to methods of setting standards. Method: A standard is a special score that serves as a boundary between those who perform well enough and those who do not. The practical steps in selecting it include: deciding on the type of standard; deciding the method for setting it; selecting the judges; holding the meeting; calculating the cutpoint, and deciding what to do afterwards. Four of the more popular methods are illustrated for both written and clinical examinations. Results: The most important criteria for selecting a method for setting standards are whether it is consistent with the purpose of the test, based on expert judgement, informed by data, supported by research, transparent, and requires due diligence. The credibility of the standard will rely largely on the nature of the standard setters and the selection of a broadly representative and knowledgeable group is essential. After the standard has been set, it is important to ensure that stakeholders view the results as credible and that the pass rates have sensible relationships with other markers of competence. Conclusions: A standard is an expression of professional values in the context of a test’s purpose and content, the ability of the examinees, and the wider social or educational setting. Because standards are an expression of values, methods for setting them are systematic ways of gathering value judgements, reaching consensus and expressing that consensus as a single score on a test.
Why is it not 52% or 55% or 45%? What is the scientific rationale for using a score of 50%? Unfortunately, the answer to all these questions is the same: We don’t know.

- Scarpa Schoeman
Thanks for reading. If someone passed this email on to you and you enjoyed it you might like to sign up here.

Enjoy the rest of your day,
Ben and Michael
MailPoet