Nigel discusses recent changes to assessments for students and how he prepared students for these new approaches.
Amy: Hi, everyone. I’m joined this morning by Professor Nigel Savery from the School of Biochemistry and he’s going to talk to us a little bit about assessment. So Nigel, can you just tell us a little bit about your assessments and how they’ve changed over the past year or so?
Nigel: Yes. So when I knew I was going to talk to you Amy, I thought about which unit would be the best one to talk about. So there’s some units in the second year that we teach about 300 students on and the way it used to be assessed before everything changed, was we had two different practical assessments, we had two essays that they wrote as coursework that were worth like two and a half percent, we had a little data handling thing that was worth 5%. And then we had an exam. So the exam overall was 80% of the unit. A third of it was 50 multiple choice questions. Yeah, you’re looking like this, so there’s a lot of assessments right? So a third of it was 50 multiple choice questions. And then they wrote two essays as well in in the exam. What happened when we went online was we consolidated that a lot. So, the thing that went completely was the multiple choice questions. So we couldn’t find a way to do that securely online.
We reduced the number of coursework essays from two down to one, and we gave a greater weighting for it. The data interpretation coursework also had a greater weighting, the practicals were consolidated so there was a single write up for it. And then the final assessment went from that big 80%, with all those different things in it, to a single essay that they wrote that was worth 60% of the unit mark, and that they had a week to write.
Nigel: So they went from two and a whole bunch of MCQs in two and a half hours to a whole week for one essay. So, some quite big changes.
Amy: Big changes, but much simpler it sounds like.
Nigel: Yes, yeah.
Okay, so how did you – that is quite a huge change -how did you prepare the students for that then?
Nigel: So the support that they used to get during the term was was basically two tutorials. So after they’d written each essay, they would have a tutorial where you discuss the essay, and that was pretty much it. We replaced that with fortnightly skills workshops. So they had five or six, I think of the skills workshops throughout the unit. And there were no marks associated with them but they all lead in some way towards preparation for the new assessments.
Nigel: So to prepare them for the essays, we got them – so they were working in groups, it was facilitated, we had several staff, this was all done on Zoom. They had sessions where they just worked as a group and wrote an introduction, not a whole essay, just an introduction. And then they read the introductions that the other groups had written, and they left comments on them. And we picked out and discussed, you know, what was good about this one, what was bad about that one. And then we’d have another one that was about how to draw a figure or another one or how to write a conclusion.
Nigel: One of them we gave them two essays from last year to mark, we gave them the marking criteria, we explained how that would work in this new open book, sort of situation where recall wasn’t being given as much credit. And they each group marked them and then they, you know, we put everybody’s marks that they compared what marks it got. And it turned out for the good essays, they were really accurate. So we gave them a range of different ones, one on the not quite so good essays, they were really harsh. So they were quite pleased to find out that we were less harsh on them. But the idea of that was to get them reading an essay, they didn’t know how good it was, discussing the marking criteria. And then they could start to see that there were lots of different ways to get to the same mark, that it wasn’t a straightforward model answer. There was a whole set of ideas, and they could put them in different ways.
Nigel: And then some of the other skills workshops were about the data handling thing that was one of the other coursework assessments. So I think that when they got into the assessment, they had a lot more discussion and writing. Lots of different bits and pieces to put them together to go through that assessment.
Amy: And how do you how do you think the impact of the learning because I would say for a STEM subject, that’s not really a traditional way to assess students.
Nigel: It did change things because… so we’d had to change the nature of the essay titles, because a lot of our essay titles were very factually, you know, they would get a choice and it’s like, write me an essay about this one specific biological process. And that can’t be challenging enough for something that’s open book and given quite a lot of time. So our essay titles were roaming over the whole unit. So the thing would be, you know, okay, you know, all of this biochemistry or this molecular genetics, how can you pick examples to talk about the importance of this concept, which, as you say, is quite different way of thinking and studying.
Nigel: And that’s what we were trying to nurture with these skills workshops. It’s like it’s not about remembering the facts with trying to get the students to think about how can you put them together? How do they fit together? How could you use them to make a hypothesis or make an argument?
Amy: So, what impact do you think the workshops had on the students?
Nigel: So one thing was when we looked at the marks at the end of the unit, the marks the overall marks, the median marks were very similar to the previous years. So I think that in putting students into a new environment, we prepared them to the point that they were able to do as well as the previous cohorts had done. So numerically, it worked. In that we got the mark distribution that we were looking for.
Taking a step back from that, when you read the work, you could see where you could clearly see where things have been taken on board. So for example, the introductions were much better when we’d had a discussion about what an introduction should do. And you could see that they’ve taken that on board. But it was much harder, I think, for the students to make this shift from an essay being a way of saying all these facts, you know, to leaving some facts out, and the importance being how they strung them together. So you could see that some students did that very well. But I think we hadn’t completely, you know, in 12 weeks, we haven’t completely converted people to thinking about things..
Amy: Yeah it’s a mindset change.
Nigel: Yes, yes. So there’s still work to do I think on that about if we stay with more open book assessments, all of us need to sort of change a bit.
Amy: Okay, so one last question. Are you going to stick with this type of assessment? Or do you think you’ll revert back to them was the ‘multiple choice, two essays’?
Nigel: I think, I think, a mixture of probably, so I think everybody that I’ve spoken to sees the advantages, the essays are better, they’re better prepared. If you want students to think it’s nice to give them time to think and not do the time pressured stuff. I think there is still some argument that says if you take out multiple choice all together, you’re not testing the- you’re not making sure you’ve tested the breadth of knowledge. So I think there’s still a debate to be had about how and where we test that. And probably being allowed in being able to do some secure assessment. Maybe MCQs, maybe short answers, but you know, the right places in the programmes, I think that will still be important, I think going forward.
Amy: Okay. All right. Really interesting. Well, thanks for taking the time to talk to us Nigel.
Nigel: Welcome. Thanks, Amy.