feedback – Bristol Institute for Learning and Teaching Blog

The Office: Episode 6

October 31, 2019October 31, 2019 BILT

‘All kinds of feedback’

A few months ago, I was sat in a conference when I got an email from one of the heads of department asking me if I was around. Without thinking, I replied that I was working but not in the office. The subsequent email asked me, in broken English, whether I could purchase £500 in Amazon vouchers and send them back to them. My suspicions were raised so I checked the email address – and, lo and behold, it had come from a scammer.

I spent the rest of the conference not thinking about the topics of discussion but how the scammer could improve the scam – how they might increase their chances of catching me out – and the feedback I should give them. I can’t help it. I love feedback (although I have learnt to keep feedback on the precise science of loading a dishwasher to a minimum over the years).

In today’s episode, I want to talk about feedback. It’s amazing. For me, it is one of the biggest reasons I am in education – to give kind, constructive, thought-provoking and applicable feedback. If you want a great lecture on concrete I am sure there are thousands of YouTube videos just waiting to be discovered (or maybe not) but getting personal feedback really is gold. Being able to present your design to an engineer who can gently ask questions and help a student to realise both what works and what maybe doesn’t is really important.

So when I design a unit ‘feedback’ is one of the items I really focus on. How can students get feedback? From whom? How can they apply it in the future on other units? And more importantly how will they apply it when they go into industry and act as a professional engineer?

Feedback mapping

I love to draw. And so rather than list the types of feedback I will use, I map them. For ‘Timber 4’ the feedback map looks like this:

Open a larger version of the image here.

So, what on earth is going on here? Well I have tried to show also sorts of different ‘feedback’ mechanisms.

Feed-in and Feed-out

On the left we have the ‘feed-in’ what have they learnt previously – and what feedback did the students receive which will help them on this unit.

On the right we have the ‘feed-out’ what happens to the feedback I give the students after the unit – this looks forward both to other units (which at this point in their degree is quite limited as they only have one term to go after this unit) but it also looks beyond this – to their life as a practising engineer – and the skills they will need and the experiences they will have.

The feedback on the final project is designed to support them in another project – their 40 credit Masters design project. I use the same marking pro forma and will provide feedback so that they can learn for this next project.

Different Types of Feedback

In the middle is all the formative feedback which occurs within the unit. We might call this feed forward or feedback cycles, I’m not sure what the exact pedagogic term is, but in my mind, it is where much of the learning occurs. And it’s where I can bring real value to the students by being involved. I have tried to build in a number of different mechanisms.

Firstly, I sit in the office and discuss questions that students might have. Some people call this feedback, I actually don’t like this term… I prefer ‘conversation’ or just plain old ‘teaching’! I think it’s useful to differentiate the two, feedback should be focused and specific not just a conversation. This of course doesn’t mean that these conversations are not important, they really are, it’s just I think that if we call them feedback its confusing.

Secondly, students are required to review each other’s work. Every group has a checking log which records the feedback students have produced, every calculation page has a checking box – which should only be signed once the page is checked, and every drawing has the same box. The aim is to get students to support each other’s learning whilst also learning from each other.

Next, students are required to submit their drawings from the first project and these are reviewed both by me (who will provide some generic feedback) and much more importantly, by a timber fabricator who will attempt to cost the students designs based on the information they have provided.

Finally, there is a ‘Quality Assurance Review’. This will involve sitting together with each team and reviewing their progress on all four projects. Three should be complete, and one will be in progress. The three complete projects will be reviewed to ascertain whether they can competently design a number of key components. It will also ensure they have checked each other’s work (a checking log is provided to students beforehand so they can clearly see what they need to do). Once we have reviewed the three projects we will then discuss project 4 (the Quality Assurance Review). This is the summative project which they will be about a third of the way through. The aim of the review is to give some technical feedback (based on projects 1-3) but also provide some feedforward on the project they are working on. This review is not credit bearing, but if I am not convinced that they are competent in certain areas of design I will ask for them to include these again within their final project submission.

Formative feedback – Myth busting

I don’t remember how many times people have said to me – ‘if the assessment is formative students won’t do it’ – but it’s a lot. I don’t agree. I think it is much more complex than this. Take the week 3 project for example. The assessment is formative – but ten out of ten groups submitted drawings. That’s 100%. Or everyone. So maybe they will do it if they have a good reason? I like to think that there are lots of good reasons for doing formative assessment including (but not limited to) it’s fun, it’s interesting, it will help build a portfolio of work I can show other people, it will help me develop as I work towards my summative assessment, it helps me to know what I do and don’t know (although I appreciate it’s rarely that simple). Much of this is described in detail in ‘Formative assessment and self-regulated learning: a model and seven principles of good feedback practice’.

Feedback on the unit

So finally I thought I’d let you know how the unit is going. I don’t have any formal feedback, yet. But I am writing a reflective diary every week so I don’t forget anything. Highlights to date have included:

Some really interesting external talks – including one on timber gridshells by Jonathan Roynon of BuroHappold and one on timber architecture by Fergus Feilden who’s Yorkshire Sculpture Park project was shortlisted for the Stirling prize – the highest honour in British architecture.
Taking the students to the Old Vic for a tour – this had two purposes – the Old Vic have agreed to be the client and they had a brand new entrance built from timber I wanted the students to see – I loved hearing their conversations as they noticed specific details.
The buzz of the office – every week it’s busy – people come and go – but there are always more people in than out (lot’s of students have other commitments through the day) and the conversation reminds me of when I used to work in industry – a mix of what you did the day before and technical discussion.
Students turning up in work attire (for the most part) every week.
Students not taking work out the office to continue working on it in their own time (as far as I am aware) – some students started to raise concerns that they might need to do this – but rather than pursue that option we reviewed what they were doing and why they had concerns.

And so at this midpoint in the project (and blog series) it seems to be going well.

Next week – funding and the student BILT fellows will be coming to visit!

Note 1: I decided – for ethical reasons – not to give feedback to the scammer in the end.

Note 2: My son recently discovered a TED talk by James Veitch on replying to scammers which we all watched and laughed to – a lot – If you have ten minutes and need a good laugh I can recommend – https://www.youtube.com/watch?v=C4Uc-cztsJo.

Note 3: I just made up the phrase feed-in and feed-out. I was trying to think of fun names for the episode and I was trying different variants and they seem to make sense to me. If you have seen them used before please let me know so I can reference them in future.

Note 4: Full reference is Nicol, D. J., & Macfarlane-Dick, D., Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in higher education, 31, 2006.

Note 5: Timber gridshells are incredible structures – Jonathan spoke about the Savill Building – for which he was the engineer – and you can find more information here: https://www.burohappold.com/projects/savill-building/

Note 6: For lot’s of beautiful photos of the Yorkshire Sculpture Park building go here: https://www.architecture.com/awards-and-competitions-landing-page/awards/riba-regional-awards/riba-yorkshire-award-winners/2019/the-weston-yorkshire-sculpture-park

Computer-based Diagnostic Assessment of Young Learners with Automated Feedback – an International Trial

This event is part of the School of Education’s ‘Bristol Conversations in Education’ seminar series. These seminars are free and open to the public.

Speaker: Tony Clark, Cambridge Assessment English

An effective diagnostic test can play a key role in the language learning process, allowing specific strengths and weaknesses in students’ linguistic development to be identified and then addressed (Jang, 2012). This paper describes the development of an online diagnostic test by Cambridge Assessment English that assesses English grammatical knowledge at A2 level.

As most language tests to date have been proficiency or achievement tests, there has been relatively little research done in the field of diagnostic language assessment and there is no real agreement on exactly what it entails (Alderson, 2005; Alderson, Brunfaut, & Harding, 2015; Davies, 1999; Lee, 2015). It was decided to create something based on the learning-oriented assessment framework (Jones & Saville, 2016) and in response to this lack of research in the field of diagnostic testing. Another aim was to trial a faster, more iterative way of working to better respond to the continuous rapid changes in technology by producing an initial prototype to trial which we could later improve based on the trial results.

Aimed at learners of approximately 15 years old, the test provides detailed diagnostic feedback on seven grammar categories at both individual and class levels, aiming to improve curriculum and lesson planning and accommodate students’ learning needs. The test was trialled internationally and surveys and focus groups were designed to investigate student and teacher perspectives. As well as discussing the results of the trial, the paper also outlines planned modifications for the next version of the Diagnostic Grammar Test and the implications of this research for wider pedagogical practice.

CLICK TO REGISTER

News, Uncategorized

Why I am Making a Zine about Assessment

March 11, 2019March 11, 2019 BILT

Zoe Backhouse is a BILT Student Fellow and fourth-year Liberal Arts student.

Zine [definition]: some sort of publication, usually mass-produced by photocopying (in some cases scanned, put on the net, or copied via fax) on any range of topics, but usually filled with passion, a means of telling one’s story, sharing thoughts, and/or artwork/ comics/ doodles.

I’m making a Zine about assessment at Bristol Uni. This Zine is going to be creative, visually-engaging and, most importantly, fun!

Zines are great ways to bring narratives together from different types of people. I’m talking to students and academics across campus to understand how they experience assessment at the moment and what they want to change for the future. Assessment is important to us on more than just a pedagogical level. Talking to Physicists last month, I learned a culture of self-certifying where students feel so pressured by stacked deadlines that they tactically decide which exams to opt out of and re-sit in summer. At the same time, the Physicists also had more of a sense of community than any students I’ve come across in Arts. Their lab assessments, group projects and tight-knot relationship with alumni – who frequently post help for problems on their giant Physics Facebook group – has brought together a huge Physics family. Assessment can unite and divide us!

I want to understand more about why assessment is so important for how we experience university, both as teachers and learners. What concepts are currently discussed in the Higher Education sector that we should be taking on at Bristol? What good practice is already happening here that more people should know about? And how can we make the most of our student body, campus and vibrant city to improve how we assess and feedback?

The Zine will consist of drawings and paintings submitted by students, snippets from conversations with academics, quotes from student focus groups and easy-to-read articles condensing theory in HE. It will give academics and student reps ideas on what’s currently being debated and what methods we can move as we become a more pedagogically-focussed university.

Hopefully this will also be an opportunity to introduce Zine as a more mainstream method for presenting information and effecting change! We’d be behind the USA where universities are already harnessing Zinemaking as a way to teach – and learn – from their students.

Have some thoughts on assessment you want included in the Zine? Know someone who would be good for me to talk to? Want to contribute a doodle, cartoon, sketch or piece of creative writing responding to the theme of assessment? Email me at zoe.backhouse@bristol.ac.uk to be involved!

“Write my essay for me!”: Rising to the Challenge of Contract Cheating

Abstract
Reported incidents of ‘contract cheating’ or ghost writing are increasing along with the number of ‘essay mills’ – the providers of bought essays, projects and even dissertations – whose marketing is becoming ever more sophisticated. In this seminar, we will explore the extent, nature and responses to this threat to academic integrity before suggesting ways in which we can alert and inform students and staff, counter the threat, and develop ways of minimising the occurrence of contract cheating in assessment.

Bio
Humphrey Bourne is Education Director (PGT) in the Faculty of Social Sciences and Law. He teaches strategy in the School of Economics, Finance and Management and his research focuses on values in and around organizations, extending into research in organizational identity. He chaired the University working group on academic integrity and misconduct, reporting in October 2017, and has since been closely involved in the implementation of its recommendations.

LeapForward Project: Learning for Practice; Feedforward to support transition to the workplace (workshop)

Open to all staff at the University of Bristol who are interested in how to improve feedback conversations in the workplace.

This workshop will be based on the resources created as part of the LeapForward project “Learning for Practice; Feedforward to support transition to the workplace”.

The LeapForward project drew on the experiences of staff and students in healthcare, social work and theatre studies settings at the University of Bristol. Three, one-hour sessions have been developed, along with a resource Toolkit, which can be used to help staff and students understand their role in feedback (feedforward) processes within workplace learning environments. This session will be an opportunity to experience the workshops that have been developed, provide feedback on them as training resources, and consider whether the workshops might be useful for your own teaching.

This workshop is suitable for any staff member working with undergraduate students in workplace settings, and for those with responsibility for training staff/students in workplace assessment processes.
The workshop will be led by Sheena Warman (CHSE assessment lead), Annie Noble (CHSE staff development lead), and Sarah Kelly (LeapForward project Research Associate).

Tea and coffee will be provided for attendees plus lunch (sandwich selection). Therefore please email the chse-admin@bristol.ac.uk mailbox at the time of booking your place if you have any special dietary requirements so that we can ensure you are catered for.

Part of a series of sessions organised by the Centre for Health Sciences Education.

Blackboard Basics: Marking and Providing Feedback to Online Assignments, see the outcome through the Students’ Eyes (DEO Webinar)

Learn how to make the most of the feedback features in Blackboard inline grading and get an insight into how your students will see their results. As well as talking you through the software we will have time to answer your questions about marking in Blackboard.

This course is suitable for academic staff and all levels of experience (some basic knowledge is helpful but not necessary).

This webinar will be presented by Roberta Perli and Roger Gardner from the Digital Education Office.

Giving Feedback in Turnitin: Getting the most out of Feedback Studio (DEO Webinar)

Learn how to make the most of the feedback features in Turnitin Feedback Studio and get an insight into how your students will see their results. As well as talking you through the software we will have time to answer your questions about marking in Turnitin.

This webinar is suitable for any level, though some basic knowledge of Turnitin would be helpful (but isn’t necessary). This webinar is aimed at academic staff.

This webinar will be presented by Naomi Beckett and Suzi Wells from the Digital Education Office.

Managing Student Access to Online Submission Points in Blackboard: Understanding Pre-Submission Quizzes, Groups and Adaptive Release [DEO Webinar]

Whether you just need a refresher or are new to managing submission points in Blackboard, this short webinar will explain how pre-submission quizzes and groups can be used with adaptive release to manage students’ access to submission points and how you can check if a student can see a submission point or not.

This webinar is suitable for all levels though some basic knowledge of Blackboard would be helpful, but not necessary. This webinar would be most suitable for administrative staff.

The room will be available from around 15 minutes before the webinar starts.

This webinar will be presented by David Perkins de Oliveira and Naomi Beckett from the Digital Education Office.

Image of the outside of Dolberry Building in Langford Campus

Education Enhancement Funds

The LeapForward Project

October 12, 2018 BILT

Dr Sheena Warman was the lead academic on her project, ‘LeapForward’, for which she was awarded a Catalyst Fund from BILT for 2017/18. The project summary and conclusions can be found below – if you would like to view the full report please contact the BILT team.

Project Summary

The LeapForward project aimed to evaluate and improve feedback and feedforward practices within undergraduate Health Sciences programmes (BVSc Veterinary Science, MBChB Medicine, BDS Dentistry), MSc Social Work, and BA Theatre and Performance Studies – a truly interdisciplinary approach. We have focused on student transitions, particularly that from classroom to workplace-based learning environments.

The overall aims and objectives of our research were to:

Explore the impact of current feedback and feedforward practices
Identify priorities for improvement in feedforward in supporting students’ self-regulatory workplace skills
Develop a novel feedforward intervention/resource relevant across diverse disciplines
Identify what different disciplines can learn from each other’s practice

In Phase 1 of the project we explored the impact of current feedback and feedforward processes and practices, by collating existing resources and running ten focus groups, talking to groups of students and staff from each discipline (in separate groups) about their experiences. Analysis of the staff and student focus groups enabled us to identify twelve overall themes, clustered into three overarching categories, which illuminate the student and staff experience of feedback and feedforward across the different disciplines, in the Bristol context.

In Phase 2, we built on the earlier findings to design and develop a modular training package for both students and staff which is intended to support students in their development of self-regulatory workplace skills and provide new feedforward interventions which have the potential to be applicable more widely across the university.

Conclusions

Interdisciplinary approach: there are similarities as well as differences in practices and experiences of feedback and feedforward and both are instructive in understanding feedback and feedforward processes at the University of Bristol
The interdisciplinary nature of the project means that both Phase 1 findings and Phase 2 training packages developed can potentially be applied widely across the University to both Students and Staff, and that there are consistent messages for both parties in the feedback /feedforward dialogue.
Student and Staff ‘Feedback Literacy’ is similar, however, for maximum overall benefit, both groups can be supported to move through ‘literacy’, via ‘capacity’ and ‘managing affect’ to ‘action’ (Carless and Boud, 2018) on a consistent and agreed trajectory.

Education Enhancement Funds

Building Confident Engaged Researchers Through Active Partnership and Problem Based Learning

October 12, 2018 BILT

A Teaching Innovation Grant was awarded to Dr Chris Kent and Dr Jess Fielding for the academic year 2017/18 – you can find a summary of the project they undertook with their grant below. If you would like to read the full report, please contact bilt-info@bristol.ac.uk.

Project Summary

This project was designed to assist the redevelopment of our research methods provision at Year 1.

The newly developed course will focus on active learning in small groups and continuous, low-risk, assessment. Specifically, it will address four main aims in our Year 1 teaching:

1. Enable students to programme and conduct their own experiments within TB1

2. Provide weekly continuous formative feedback on knowledge and understanding

3. Enable effective small group peer support via ‘homework clubs’

4. Embed a culture of student participation in lectures via interactive smart-phone response systems (SRS).

The BILT grant facilitated the redesign and comprised two main work packages

(WP). WP1: development and evaluation of a self-contained series of lab sessions designed to deliver design, programming and analysis skills to Y1 psychology students (meeting aims 1 and 3). WP2: developed a set of home and class activities for summative and formative assessment (meeting aims 2 and 4).

Conclusions

We feel the grant was extremely helpful to us. We have meet our aims and are in a much better place to implement our redesigned research method training courses.

Students, who undertook the pilot, showed a keen interest and disposition to research methods once they were exposed to a hands-on approach to building experiments and understanding why programming and statistics were important (via problem-based learning and active participation in work shops). The students left with a positive disposition towards further exploring research methods and we highly positivetowards the way in which the material was delivered. All students managed to develop their own independent research question, programme an experiment to test it, collect and analysis the data; this is quite impressive given the short six week time scale!

The research skills we developed by actively engaging students in research from the get-go, by showing them they could develop their own simple experiments and analysis with a few simple tools. The problem-based approach to introducing new software was successful and students felt confident in exploring the software.