Bristol Institute for Learning and Teaching Blog
We are BILT, and we're here to provide information and resources to promote educational excellence in the University and beyond. This is our blog site, where you can also find resources, videos and our discussion forum!
Introduction to online marking using Blackboard ‘the Marks and Feedback extension’.
See the Eventbrite page for further details about this webinar and to register.
Recordings
If you want to view any recordings of the webinars you can access them in the DEO Staff Development Course in the ‘Session content and webinar recordings’ section.
Contact information
If you have any questions, please contact the the DEO by emailing digital-education@bristol.ac.uk
This webinar is aimed at colleagues who will be involved in marking and moderating assignments submitted through Blackboard using the Marks and Feedback extension.
Areas covered: accessing assignments on Blackboards, using online marking features to annotate assignment, provide feedback and mark, check student view of the marked assignment.
See the Eventbrite page for further details about this webinar and to register.
Contact information
If you have any questions, please contact the the DEO by emailing digital-education@bristol.ac.uk
This webinar is aimed at colleagues who will be involved in marking and moderating assignments submitted through Blackboard using the Marks and Feedback extension.
Areas covered: accessing assignments on Blackboards, using online marking features to annotate assignment, provide feedback and mark, check student view of the marked assignment.
See the Eventbrite page for further details about this webinar and to register.
Contact information
If you have any questions, please contact the the DEO by emailing digital-education@bristol.ac.uk
So, I will be honest, I have been dwelling on this blog idea for a while now, and the reason I haven’t written it is because I was stuck. I was struggling to come up with the perfect name for my idea. I hope the name I have come up with makes sense. But first some background…
I have been dwelling on the idea of Authentic Learning for a very long time now, probably as far back as 2003 when I started teaching, having worked for a few years as a practicing engineer. I have developed ideas and strategies, based on my own experience, that I have tried across a number of units. Then, at some point last year I read Marilyn Lombardi’s paper on Authentic Learning (2007). It was such a beautiful moment as it summarised my own practice so clearly and succinctly. She articulated what I had innately known. I made a matrix of the 10 facets of authentic learning and mapped my own units against them. With the exception of reflection (and more on that in another blog post I hope) I had been doing everything she listed for years.
Note: If you would like a further explanation of authentic learning I wrote a blog post on the subject last term as part of my “The Office” project, which you can read here.
However I also noticed a gap. An 11th facet of authentic learning, if you will. Providing feedback whilst staying ‘in role’. I started to call it authentic feedback. But a quick internet search of the term ‘authentic feedback’ shows that the term was already taken, by another idea on feedback. And so I floundered and my ideas paused. Until now.
And so here it is, my idea. Providing feedback in an authentic context. I know it’s not as snappy as authentic feedback, but I think it says what it does on the tin. I don’t need lots of paragraphs explaining what I actually mean.
So how have I (and in-fact we in engineering) been providing feedback in an authentic context. Below are just a few examples.
The Design Team Meeting
A few years back I created a unit called Understanding Architecture. It teaches Civil Engineers to understand what the architect is trying to achieve by placing them in the architect’s shoes. The unit is very practical and includes the students developing a conceptual design for a building. I wanted to create a formative feedback point within the unit to help students as they developed their ideas. However rather than just ask them to submit their ideas up to that point I put it into the context of professional practice and asked them to lead a design team meeting (known, rather unimaginatively, in industry as a DTM).
A design team meeting is a staple of the building design process, all the different disciplines come together, with the client, and discuss their progress, problems and conflicts. It is an interactive design space where the team then solve the problems moving the design forward.
So, we created this context. We invited engineers, architects and client representatives to be part of the meeting, and our students had to both present their ideas and chair the meeting. It creates a space for constructive feedback, where the design can be pushed and pulled. The client can confirm if the brief is right, the engineers can challenge some of the practical aspects of the design and the clients architect can question some of the design decisions. This way students are given feedback whilst staying in role and in an authentic manner.
The Quality Assurance Review
In Timber Engineering – a unit I blogged about obsessively last term (see https://bilt.online/the-office-episode-0-trailer/) I carried out a similar exercise to the above Design Team Meeting, but took it in a different direction. This time I recast the formative feedback as a Quality Assurance Review. Every project I worked on was subjected to internal reviews as part of our practice. These ensured the design was safe, was fulfilling the brief, but also looked for opportunities, how could we do this better, how can we learn from this project and share these ideas etc. The review was carried out by a director not directly involved in the project and there was a checklist of items which we had to ensure we had covered.
I used the same approach for my fourth year timber engineering unit. I created a series of Quality Assurance forms and a procedure. Students then presented the different projects they had worked on and I was able to provide feedback across a number of facets. One of the strengths of this approach was that all work presented should have been reviewed by another member of the students team, this form of peer review is both helpful for learning, and normal practice in industry. The Quality Assurance Review then checks has this has been carried out and what can we learn from this process?
The Stakeholder Presentation
At the other end of our programme, in the first year, my colleague Jeff Barrie runs a project in our design unit, where students must come up with an engineering solution to an authentic brief. The only problem is that there are three stakeholders, with conflicting interests. It is therefore very difficult to create a solution that satisfies all three stakeholders. This is brought to life when students present their schemes (including fantastic models) to the stakeholders (three assessors each play the role of a different stakeholder). Some stakeholders are delighted with the design, others not happy that their needs have been met or their concerns have not been heard. The aim is not to create a solution that works for everyone but to be able to articulate why the solution is the most suitable when there are conflicting requirements.
The Green Pen
Finally, in industry, people red pen everything! Every drawing I drew, every report I wrote, would reappear on my desk a few days later covered in red pen. Taking in drafts and writing comments on them is actually incredibly authentic. However, I would like to suggest going a step further. An ex colleague of mine used to work for a practice called Alan Baxter’s. As was common practice everywhere else people would red pen each other’s work as a way of checking and providing feedback. But in Alan Baxter’s no one was allowed to use a green pen. No one, that is, except Alan Baxter. When Alan reviewed a drawing or report he wrote in green!
What I like about this idea is that we can, and should, encourage students to red pen each other’s work, to support each other’s learning (and learn themselves in the process) but we should also provide feedback, and we can differentiate our feedback from thier’s by simply using a different colour pen. This way we can create feedback in an authentic context.
What feedback in an authentic context have you done?
I would love to hear from other authentic learning practitioners who have stayed in character to provide feedback. You could email me, or even better, tell the world by adding it to the comments below. I think there is so much space for innovation and creativity in this area and I would love to explore it further.
Reference
Lombardi M., “Authentic Learning for the 21st Century: An Overview”, Educase, 2007.
Spotlight on ‘Voicing Vulnerability’
It’s lunchtime on Woodland Road. The autumn skylight floods in through the bay window at the Multifaith Chaplaincy. The meeting space is bustling with a few members of staff and dozens of students all giving friendly greetings and catching up over complimentary tea, coffee, and today’s affordably priced soup: Thai Style Pea, Mint & Coconut.
I weave through groups of students immersed in conversation and try to capture a few snippets of student conversations, thoughts, feelings, and perspectives centered around dis/connection, failure, and feedback that make up our experiences of vulnerability whilst at the University. The approach of this Humans of Bristol University feature is to turn towards community spaces at the University and the people bringing these spaces to life.
What brings you to the Multifaith Chaplaincy?
Emily: I love this space. I love the soup. I love what these women are doing here; affordable soup is such an incentive to meet up with friends and grab lunch on campus. The meeting room has a calm and relaxing atmosphere.
Tom: Yeah. I feel like it is a much better working environment than some of the larger libraries across the campus with clinical lighting and intimidating atmospheres. For me, the Arts and Social Science Library might be a good spot if you are doing work at 3 AM and want to stay awake. But I find the space quite clinical. In often feels like a sad place in the daytime, so I tend to come to the Multifaith Chaplaincy to study in a more relaxing ‘Living Room’ environment.
Do you think University staff and students could benefit from more of these community-oriented spaces and the services and support they offer?
Maya: Yes! Especially if staff are also involved. Some of us have so few interactions with staff members because of our limited contact hours.
Tom: Also, I feel like there is a demand at the University for spaces like this one. I mean look at the popularity of the SU Living Room… it is so busy there now. In a way, the space has become a bit too busy, so I still think the Multifaith Chaplaincy is the place for me. We definitely need more community hubs on campus to offset the demand of the SU Living Room and to not run the risk of our social and community spaces quickly becoming overcrowded.
What are your thoughts on the growing importance of the ‘Ways to Well-being’ strategy at the University? What do you think is working and where do you think the University needs to improve?
Emily: This year I know where the well-being advisers are in our department; we receive a lot of e-mails about this. I think the University has done a lot more than people tend to give them credit for. The University is getting better at preventative strategies despite the wait-time for counselling remaining rather disappointing.
Tom: I think overseeing student attendance at lectures would be nice. And it does seem to be working for the courses that already do this. The University should grow from this strength. It’s important to check up on how students are doing, whether they are faring well, especially those who do not feel up for coming into University.
Emily: It would be nice knowing the university actually cares about us as people beyond our academic production.
Maya: Also, I think the fact that we do not meet our personal tutors very often is quite detrimental to student well-being. I mean my personal tutor meets with me like once a term officially. Me and so many of my friends feel like we do not know what we are doing most of the time. Then we get grades and feedback returned and feel confused as to how we ended up with the grade: good or bad.
In terms of negative feedback, how do you feel reading back on comments from markers?
Emily: Most of us enter University with optimism and high expectation, we often feel the pressure to make the most out of the experience and excel in the best way we can: whether that is socially, in our extracurricular activities, or in our academic grade. Sometimes, given the random collection of factors and unexpected events, we do not succeed in our personal aspirations at University – this can unsettle us emotionally.
Tom: I guess most of us don’t feel well-equipped to cope with failure. University needs to prepare students for failure and educate us on mechanisms for coping and reflecting on that failure. A disappointing mark is never just an academic failure, but it can feel like a personal failure as well.
Where do you draw energy and support when you are feeling vulnerable or a little lost at University?
Maya: I think course mates have become so important for me. Actually, without them I would feel so lost. We have created group chats and can help each other out with notes and support each other in both the administrative and academic sense.
Emily: Yeah, I am lucky because biology is quite a friendly course.
Tom: Oh really? What? Does everyone really get on with everyone? My course feels so cliquey.
I point out how the opportunities to forge connections across our academic cohort and to develop a sense of belonging should not be left to mere chance and luck. Instead, the ‘importance of course mates’ should be part of the University Well-being Strategy and we ought to think about how much our teaching and learning spaces are conducive to forging personable connections.
Do you recall memories of a time where you had positive engagement with academic staff and how you benefited from it?
Tom: I actually remember a time where the absolute inverse happened. I remember a time where I was snubbed by a member of staff. I was sort of following him after a lecture and I went over and said “I am really interested in (X) you presented and (Y) in the slide, could you tell me more about how (Z) might fit into what you are talking about?”
He replied by saying I should go and research this myself and find it all out for myself. But, you see, I was trying to do that, but I was confused. Despite expressing interest and showing engagement I seemed to hit a wall. I felt like this particular staff member really did not care about me. I think the overemphasis on ‘independent learning’ makes me feel frequently deflated.
Emily: I agree. I find the whole ‘learn by yourself’ style of teaching quite isolating. If I am trying to engage with staff after a lecture or in consultation hours, then I think we are within our right to ask for a bit more personable support and guidance from staff rather than relying on their signposts to research papers. For me the learning is in the process, and staff should be contributing to that learning process. Sometimes I feel like the only recognizable outcome of our academic pursuits is the grade, but what about the learning process required to construct the essay argument itself? I guess a 2000-word essay can’t really encompass all the intellectual growth spurts we feel throughout the term. Nor can all of our learning be neatly certified in a 60 or 69. Yet we still feel like a failure if we do not receive the numerical grade we hoped for.
Tom: Yeah, failing has so many negative connotations to it. But sometimes our failures can create moments of learning. It could be cool for us to reorder the popular narratives around failure and success. At the end of the day we are all imperfect and we could use this attribute to transform how we respond to challenging experiences of disappointment and inadequacy.
Emily: Instead of saying, ‘What grade did you get?’ me and my friends ask, ‘Are you happy with the grade you got?’. We then start to talk about our feelings around expectation and disappointment rather than ending our conversations with a numerical grade.
Owen Barlow, BILT Student Fellow, December 2019
A few months ago, I was sat in a conference when I got an email from one of the heads of department asking me if I was around. Without thinking, I replied that I was working but not in the office. The subsequent email asked me, in broken English, whether I could purchase £500 in Amazon vouchers and send them back to them. My suspicions were raised so I checked the email address – and, lo and behold, it had come from a scammer.
I spent the rest of the conference not thinking about the topics of discussion but how the scammer could improve the scam – how they might increase their chances of catching me out – and the feedback I should give them. I can’t help it. I love feedback (although I have learnt to keep feedback on the precise science of loading a dishwasher to a minimum over the years).
In today’s episode, I want to talk about feedback. It’s amazing. For me, it is one of the biggest reasons I am in education – to give kind, constructive, thought-provoking and applicable feedback. If you want a great lecture on concrete I am sure there are thousands of YouTube videos just waiting to be discovered (or maybe not) but getting personal feedback really is gold. Being able to present your design to an engineer who can gently ask questions and help a student to realise both what works and what maybe doesn’t is really important.
So when I design a unit ‘feedback’ is one of the items I really focus on. How can students get feedback? From whom? How can they apply it in the future on other units? And more importantly how will they apply it when they go into industry and act as a professional engineer?
Feedback mapping
I love to draw. And so rather than list the types of feedback I will use, I map them. For ‘Timber 4’ the feedback map looks like this:
So, what on earth is going on here? Well I have tried to show also sorts of different ‘feedback’ mechanisms.
Feed-in and Feed-out
On the left we have the ‘feed-in’ what have they learnt previously – and what feedback did the students receive which will help them on this unit.
On the right we have the ‘feed-out’ what happens to the feedback I give the students after the unit – this looks forward both to other units (which at this point in their degree is quite limited as they only have one term to go after this unit) but it also looks beyond this – to their life as a practising engineer – and the skills they will need and the experiences they will have.
The feedback on the final project is designed to support them in another project – their 40 credit Masters design project. I use the same marking pro forma and will provide feedback so that they can learn for this next project.
Different Types of Feedback
In the middle is all the formative feedback which occurs within the unit. We might call this feed forward or feedback cycles, I’m not sure what the exact pedagogic term is, but in my mind, it is where much of the learning occurs. And it’s where I can bring real value to the students by being involved. I have tried to build in a number of different mechanisms.
Firstly, I sit in the office and discuss questions that students might have. Some people call this feedback, I actually don’t like this term… I prefer ‘conversation’ or just plain old ‘teaching’! I think it’s useful to differentiate the two, feedback should be focused and specific not just a conversation. This of course doesn’t mean that these conversations are not important, they really are, it’s just I think that if we call them feedback its confusing.
Secondly, students are required to review each other’s work. Every group has a checking log which records the feedback students have produced, every calculation page has a checking box – which should only be signed once the page is checked, and every drawing has the same box. The aim is to get students to support each other’s learning whilst also learning from each other.
Next, students are required to submit their drawings from the first project and these are reviewed both by me (who will provide some generic feedback) and much more importantly, by a timber fabricator who will attempt to cost the students designs based on the information they have provided.
Finally, there is a ‘Quality Assurance Review’. This will involve sitting together with each team and reviewing their progress on all four projects. Three should be complete, and one will be in progress. The three complete projects will be reviewed to ascertain whether they can competently design a number of key components. It will also ensure they have checked each other’s work (a checking log is provided to students beforehand so they can clearly see what they need to do). Once we have reviewed the three projects we will then discuss project 4 (the Quality Assurance Review). This is the summative project which they will be about a third of the way through. The aim of the review is to give some technical feedback (based on projects 1-3) but also provide some feedforward on the project they are working on. This review is not credit bearing, but if I am not convinced that they are competent in certain areas of design I will ask for them to include these again within their final project submission.
Formative feedback – Myth busting
I don’t remember how many times people have said to me – ‘if the assessment is formative students won’t do it’ – but it’s a lot. I don’t agree. I think it is much more complex than this. Take the week 3 project for example. The assessment is formative – but ten out of ten groups submitted drawings. That’s 100%. Or everyone. So maybe they will do it if they have a good reason? I like to think that there are lots of good reasons for doing formative assessment including (but not limited to) it’s fun, it’s interesting, it will help build a portfolio of work I can show other people, it will help me develop as I work towards my summative assessment, it helps me to know what I do and don’t know (although I appreciate it’s rarely that simple). Much of this is described in detail in ‘Formative assessment and self-regulated learning: a model and seven principles of good feedback practice’.
Feedback on the unit
So finally I thought I’d let you know how the unit is going. I don’t have any formal feedback, yet. But I am writing a reflective diary every week so I don’t forget anything. Highlights to date have included:
And so at this midpoint in the project (and blog series) it seems to be going well.
Next week – funding and the student BILT fellows will be coming to visit!
Note 1: I decided – for ethical reasons – not to give feedback to the scammer in the end.
Note 2: My son recently discovered a TED talk by James Veitch on replying to scammers which we all watched and laughed to – a lot – If you have ten minutes and need a good laugh I can recommend – https://www.youtube.com/watch?v=C4Uc-cztsJo.
Note 3: I just made up the phrase feed-in and feed-out. I was trying to think of fun names for the episode and I was trying different variants and they seem to make sense to me. If you have seen them used before please let me know so I can reference them in future.
Note 4: Full reference is Nicol, D. J., & Macfarlane-Dick, D., Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in higher education, 31, 2006.
Note 5: Timber gridshells are incredible structures – Jonathan spoke about the Savill Building – for which he was the engineer – and you can find more information here: https://www.burohappold.com/projects/savill-building/
Note 6: For lot’s of beautiful photos of the Yorkshire Sculpture Park building go here: https://www.architecture.com/awards-and-competitions-landing-page/awards/riba-regional-awards/riba-yorkshire-award-winners/2019/the-weston-yorkshire-sculpture-park
This event is part of the School of Education’s ‘Bristol Conversations in Education’ seminar series. These seminars are free and open to the public.
Speaker: Tony Clark, Cambridge Assessment English
An effective diagnostic test can play a key role in the language learning process, allowing specific strengths and weaknesses in students’ linguistic development to be identified and then addressed (Jang, 2012). This paper describes the development of an online diagnostic test by Cambridge Assessment English that assesses English grammatical knowledge at A2 level.
As most language tests to date have been proficiency or achievement tests, there has been relatively little research done in the field of diagnostic language assessment and there is no real agreement on exactly what it entails (Alderson, 2005; Alderson, Brunfaut, & Harding, 2015; Davies, 1999; Lee, 2015). It was decided to create something based on the learning-oriented assessment framework (Jones & Saville, 2016) and in response to this lack of research in the field of diagnostic testing. Another aim was to trial a faster, more iterative way of working to better respond to the continuous rapid changes in technology by producing an initial prototype to trial which we could later improve based on the trial results.
Aimed at learners of approximately 15 years old, the test provides detailed diagnostic feedback on seven grammar categories at both individual and class levels, aiming to improve curriculum and lesson planning and accommodate students’ learning needs. The test was trialled internationally and surveys and focus groups were designed to investigate student and teacher perspectives. As well as discussing the results of the trial, the paper also outlines planned modifications for the next version of the Diagnostic Grammar Test and the implications of this research for wider pedagogical practice.
Zoe Backhouse is a BILT Student Fellow and fourth-year Liberal Arts student.
Zine [definition]: some sort of publication, usually mass-produced by photocopying (in some cases scanned, put on the net, or copied via fax) on any range of topics, but usually filled with passion, a means of telling one’s story, sharing thoughts, and/or artwork/ comics/ doodles.
I’m making a Zine about assessment at Bristol Uni. This Zine is going to be creative, visually-engaging and, most importantly, fun!
Zines are great ways to bring narratives together from different types of people. I’m talking to students and academics across campus to understand how they experience assessment at the moment and what they want to change for the future. Assessment is important to us on more than just a pedagogical level. Talking to Physicists last month, I learned a culture of self-certifying where students feel so pressured by stacked deadlines that they tactically decide which exams to opt out of and re-sit in summer. At the same time, the Physicists also had more of a sense of community than any students I’ve come across in Arts. Their lab assessments, group projects and tight-knot relationship with alumni – who frequently post help for problems on their giant Physics Facebook group – has brought together a huge Physics family. Assessment can unite and divide us!
I want to understand more about why assessment is so important for how we experience university, both as teachers and learners. What concepts are currently discussed in the Higher Education sector that we should be taking on at Bristol? What good practice is already happening here that more people should know about? And how can we make the most of our student body, campus and vibrant city to improve how we assess and feedback?
The Zine will consist of drawings and paintings submitted by students, snippets from conversations with academics, quotes from student focus groups and easy-to-read articles condensing theory in HE. It will give academics and student reps ideas on what’s currently being debated and what methods we can move as we become a more pedagogically-focussed university.
Hopefully this will also be an opportunity to introduce Zine as a more mainstream method for presenting information and effecting change! We’d be behind the USA where universities are already harnessing Zinemaking as a way to teach – and learn – from their students.
Have some thoughts on assessment you want included in the Zine? Know someone who would be good for me to talk to? Want to contribute a doodle, cartoon, sketch or piece of creative writing responding to the theme of assessment? Email me at zoe.backhouse@bristol.ac.uk to be involved!
Abstract
Reported incidents of ‘contract cheating’ or ghost writing are increasing along with the number of ‘essay mills’ – the providers of bought essays, projects and even dissertations – whose marketing is becoming ever more sophisticated. In this seminar, we will explore the extent, nature and responses to this threat to academic integrity before suggesting ways in which we can alert and inform students and staff, counter the threat, and develop ways of minimising the occurrence of contract cheating in assessment.
Bio
Humphrey Bourne is Education Director (PGT) in the Faculty of Social Sciences and Law. He teaches strategy in the School of Economics, Finance and Management and his research focuses on values in and around organizations, extending into research in organizational identity. He chaired the University working group on academic integrity and misconduct, reporting in October 2017, and has since been closely involved in the implementation of its recommendations.
Open to all staff at the University of Bristol who are interested in how to improve feedback conversations in the workplace.
This workshop will be based on the resources created as part of the LeapForward project “Learning for Practice; Feedforward to support transition to the workplace”.
The LeapForward project drew on the experiences of staff and students in healthcare, social work and theatre studies settings at the University of Bristol. Three, one-hour sessions have been developed, along with a resource Toolkit, which can be used to help staff and students understand their role in feedback (feedforward) processes within workplace learning environments. This session will be an opportunity to experience the workshops that have been developed, provide feedback on them as training resources, and consider whether the workshops might be useful for your own teaching.
This workshop is suitable for any staff member working with undergraduate students in workplace settings, and for those with responsibility for training staff/students in workplace assessment processes.
The workshop will be led by Sheena Warman (CHSE assessment lead), Annie Noble (CHSE staff development lead), and Sarah Kelly (LeapForward project Research Associate).
Tea and coffee will be provided for attendees plus lunch (sandwich selection). Therefore please email the chse-admin@bristol.ac.uk mailbox at the time of booking your place if you have any special dietary requirements so that we can ensure you are catered for.
Part of a series of sessions organised by the Centre for Health Sciences Education.
