A case study by Becky Selwyn produced as part of a Education Development Project 2022/23.
The Unit
Engineering by Investigation is a 20-credit ‘lab unit’, delivered to 500-600 1st year undergraduate engineering students per year. The unit learning outcomes require students to be able to demonstrate the skills required to successfully undertake practical activities. These outcomes include engagement with risk assessments, development of coding skills, use of electronic principles, and written communication (Unit Description). The summative assessment is a written lab report, following guidelines provided throughout the unit.
The Problem
Engineering undergraduates find writing difficult. It is often something they have actively chosen not to study in their prior studies (having focused on maths and sciences), so they have limited experience when they arrive at university. However, the ability to communicate is a core competence required by engineering careers, and additionally forms a significant part of assessment of 40-credit research projects in 3rd and 4th years of study.
Repeated cycles of practice, feedback, and opportunities to close the feedback loop by using feedback in future practice have been demonstrated to help students learn, but feedback must also be timely and relevant to the next task [1]. The Engineering by Investigation unit provides four formative lab report assessments throughout the year building from only Methods and Results sections up to a full report by the final assessment. This gives students plenty of practice and chance to act on feedback after each stage. However, it is not feasible to provide staff feedback on this volume of reports in a timely manner, so we introduced peer feedback.
The Practice
Peer feedback has been shown to result in better student engagement with feedback, internalisation of assessment criteria, and better student assessment outcomes. It encourages students to become more independent learners, and prepares them for giving and receiving feedback in their later studies and their careers [2, 3].
Students uploaded their formative lab reports to Turnitin and were randomly allocated another report to review using Turnitin’s PeerMark functionality. Review questions were provided through PeerMark, as shown in Figure 1. The structured questions used for each formative lab report were directly linked to the summative assessment criteria, and there were several questions for each section of the report. The number of questions increased with each formative report as the reports included more sections.
Figure 1: Student view of Turnitin’s PeerMark system, with anonymous report in main window and review questions on right-hand side.
Figure 2 shows the three steps involved for each formative assessment. Repetition maximised the opportunity for students to incorporate feedback into later reports. Peer reviews were completed during in-person timetabled sessions held in Ivy Gate. Students on the unit were split into four groups of ~125 students to fit within the room capacity, hence each session was repeated four times per fortnight. Self-selected teams of 4-5 students used the pod computers to work through the review questions for one report at a time, meaning that all students saw multiple other reports, and groups combined their experience and understanding to provide each review. Three staff facilitated the sessions by prompting and questioning student teams while they completed the reviews.
Figure 2: Peer review process steps. This was repeated four times (once for each formative assessment).
The Impact
Monitoring student completion of formative tasks and comparing with outcome in the pass/fail summative assessment showed that students who completed more formative tasks were more likely to pass the summative assessment. Repeated practice and enforced interaction with the assessment criteria seemed to help clarify expectations, and the constructive alignment between formative and summative tasks was useful to students.
Staff perceived a reluctance from students to engage with peer reviews. Student surveys and focus groups highlighted that the main source of this reluctance was lack of confidence in a student’s own ability (or trust in a peer’s abilities), and a desire for the ‘expert’ to provide feedback.
It may be difficult to produce an end of year lab report that I know is good enough to pass because my work has only ever been reviewed by my peers and not a member of staff who understands the passing criteria better
I feel that students aren’t in a very good position to provide expert advice on my reports
Quotes from students about their expectations of the peer review process.
After experiencing the peer review process once, students were more positive about both giving and receiving peer reviews, but particularly recognised the additional benefits to themselves of generating feedback for someone else.
It made me realise what I had missed
Usually they are about to point out something you would not have noticed.
It is quite fascinating how much you gain unintentionally just to equip yourself with enough knowledge to give peer reviews.
Quotes from students about their experience of engaging with peer reviews.
Next Steps
Peer review is now embedded as a core aspect of this unit, and with careful expectation setting from staff throughout the process, student engagement has improved. Elements of the peer review process have also been added to units in later years.
Five Top Tips
If you are considering using peer feedback in your own teaching, these are the five top tips that we would suggest to maximise the likelihood of successful implementation.
- Check that the software works the way you want it to, and be ready to show students how to use it before they start.
- Be clear about expectations, both for the submissions and for the reviews:
- For lab reports, this meant being specific about what points should be addressed in each section of the report, and how data should be presented correctly. Give students training in this before the submission.
- For peer reviews this meant emphasising that the benefits come from providing peer reviews, not receiving them. Highlight the importance of students engaging with the assessment criteria for the summative assessment, and how the formative assessment criteria fit with the summative assessment criteria.
- Get students to work in groups to share their knowledge while providing reviews.
- Provide structured questions to guide the reviews.
- Objective yes/no questions work well for the basics (e.g. does the figure have axis labels with units included?).
- For more complex areas, asking about the reader’s interpretation can reduce the reviewer’s fear of getting it wrong, as well as focusing the author’s mind on clarity for their audience (e.g. what did you, as the reader, understand to be the main source of error in this work?).
- Use staff to engage in dialogue with reviewers rather than giving definitive answers – if a student has a question, send it back to them and their team and help them find the argument towards the right answer (if there is one).
Contact
If you would like to discuss our experiences further, or share your own experiences and tips, please contact Becky Selwyn – r.selwyn@bristol.ac.uk
References
[1] Carless, D. (2019.) Feedback loops and the longer-term: towards feedback spirals. Assessment & Evaluation in Higher Education, 44(5), 705-714. DOI: 10.1080/02602938.2018.1531108
[2] Moore, C. & Teather, S. (2013). Engaging students in peer review: Feedback as learning. In Special issue: Teaching and learning in higher education: Western Australia’s TL Forum. Issues in Educational Research, 23(2), 196-211. http://www.iier.org.au/iier23/moore.html
[3] Nicol, D., Thomson, A. & Breslin, C. (2014). Rethinking feedback practices in higher education: a peer review perspective. Assessment & Evaluation in Higher Education, 39(1), 102-122, DOI: 10.1080/02602938.2013.795518
