We are BILT, and we're here to provide information and resources to promote educational excellence in the University and beyond. This is our blog site, where you can also find resources, videos and our discussion forum!
So, I will be honest, I have been dwelling on this blog idea for a while now, and the reason I haven’t written it is because I was stuck. I was struggling to come up with the perfect name for my idea. I hope the name I have come up with makes sense. But first some background…
I have been dwelling
on the idea of Authentic Learning for a very long time now, probably as far
back as 2003 when I started teaching, having worked for a few years as a
practicing engineer. I have developed ideas and strategies, based on my own
experience, that I have tried across a number of units. Then, at some point
last year I read Marilyn Lombardi’s paper on Authentic Learning (2007). It was
such a beautiful moment as it summarised my own practice so clearly and
succinctly. She articulated what I had innately known. I made a matrix of the
10 facets of authentic learning and mapped my own units against them. With the
exception of reflection (and more on that in another blog post I hope) I had been
doing everything she listed for years.
Note: If you would like a further explanation of authentic learning I wrote a blog post on the subject last term as part of my “The Office” project, which you can read here.
However I also noticed
a gap. An 11th facet of authentic learning, if you will. Providing
feedback whilst staying ‘in role’. I started to call it authentic feedback. But
a quick internet search of the term ‘authentic feedback’ shows that the term
was already taken, by another idea on feedback. And so I floundered and my
ideas paused. Until now.
And so here it is, my
idea. Providing feedback in an authentic
context. I know it’s not as snappy as authentic feedback, but I think it
says what it does on the tin. I don’t need lots of paragraphs explaining what I
So how have I (and in-fact
we in engineering) been providing feedback in an authentic context. Below are
just a few examples.
The Design Team Meeting
A few years back I
created a unit called Understanding Architecture. It teaches Civil Engineers to
understand what the architect is trying to achieve by placing them in the architect’s
shoes. The unit is very practical and includes the students developing a
conceptual design for a building. I wanted to create a formative feedback point
within the unit to help students as they developed their ideas. However rather
than just ask them to submit their ideas up to that point I put it into the
context of professional practice and asked them to lead a design team meeting (known,
rather unimaginatively, in industry as a DTM).
A design team meeting
is a staple of the building design process, all the different disciplines come
together, with the client, and discuss their progress, problems and conflicts.
It is an interactive design space where the team then solve the problems moving
the design forward.
So, we created this
context. We invited engineers, architects and client representatives to be part
of the meeting, and our students had to both present their ideas and chair the
meeting. It creates a space for constructive feedback, where the design can be
pushed and pulled. The client can confirm if the brief is right, the engineers
can challenge some of the practical aspects of the design and the clients
architect can question some of the design decisions. This way students are
given feedback whilst staying in role and in an authentic manner.
The Quality Assurance Review
In Timber Engineering
– a unit I blogged about obsessively last term (see https://bilt.online/the-office-episode-0-trailer/) I carried out a similar exercise to the above
Design Team Meeting, but took it in a different direction. This time I recast
the formative feedback as a Quality Assurance Review. Every project I worked on
was subjected to internal reviews as part of our practice. These ensured the
design was safe, was fulfilling the brief, but also looked for opportunities,
how could we do this better, how can we learn from this project and share these
ideas etc. The review was carried out by a director not directly involved in
the project and there was a checklist of items which we had to ensure we had
I used the same
approach for my fourth year timber engineering unit. I created a series of
Quality Assurance forms and a procedure. Students then presented the different
projects they had worked on and I was able to provide feedback across a number
of facets. One of the strengths of this approach was that all work presented
should have been reviewed by another member of the students team, this form of
peer review is both helpful for learning, and normal practice in industry. The
Quality Assurance Review then checks has this has been carried out and what can
we learn from this process?
The Stakeholder Presentation
At the other end of our programme, in the first year, my colleague Jeff Barrie runs a project in our design unit, where students must come up with an engineering solution to an authentic brief. The only problem is that there are three stakeholders, with conflicting interests. It is therefore very difficult to create a solution that satisfies all three stakeholders. This is brought to life when students present their schemes (including fantastic models) to the stakeholders (three assessors each play the role of a different stakeholder). Some stakeholders are delighted with the design, others not happy that their needs have been met or their concerns have not been heard. The aim is not to create a solution that works for everyone but to be able to articulate why the solution is the most suitable when there are conflicting requirements.
The Green Pen
Finally, in industry,
people red pen everything! Every drawing I drew, every report I wrote, would
reappear on my desk a few days later covered in red pen. Taking in drafts and writing
comments on them is actually incredibly authentic. However, I would like to
suggest going a step further. An ex colleague of mine used to work for a
practice called Alan Baxter’s. As was common practice everywhere else people
would red pen each other’s work as a way of checking and providing feedback.
But in Alan Baxter’s no one was allowed to use a green pen. No one, that is,
except Alan Baxter. When Alan reviewed a drawing or report he wrote in green!
What I like about this
idea is that we can, and should, encourage students to red pen each other’s
work, to support each other’s learning (and learn themselves in the process)
but we should also provide feedback, and we can differentiate our feedback from
thier’s by simply using a different colour pen. This way we can create feedback
in an authentic context.
What feedback in an authentic context have you
I would love to hear from other authentic learning practitioners who have stayed in character to provide feedback. You could email me, or even better, tell the world by adding it to the comments below. I think there is so much space for innovation and creativity in this area and I would love to explore it further.
“Authentic Learning for the 21st Century: An Overview”, Educase, 2007.
It’s lunchtime on Woodland Road. The autumn skylight floods in through the bay window at the Multifaith Chaplaincy. The meeting space is bustling with a few members of staff and dozens of students all giving friendly greetings and catching up over complimentary tea, coffee, and today’s affordably priced soup: Thai Style Pea, Mint & Coconut.
I weave through groups of students immersed in conversation and try to capture a few snippets of student conversations, thoughts, feelings, and perspectives centered around dis/connection, failure, and feedback that make up our experiences of vulnerability whilst at the University. The approach of this Humans of Bristol University feature is to turn towards community spaces at the University and the people bringing these spaces to life.
What brings you to
the Multifaith Chaplaincy?
Emily: I love this space.
I love the soup. I love what these women are doing here; affordable soup is
such an incentive to meet up with friends and grab lunch on campus. The meeting
room has a calm and relaxing atmosphere.
Tom: Yeah. I feel like it is a much better working environment than some of the larger libraries across the campus with clinical lighting and intimidating atmospheres. For me, the Arts and Social Science Library might be a good spot if you are doing work at 3 AM and want to stay awake. But I find the space quite clinical. In often feels like a sad place in the daytime, so I tend to come to the Multifaith Chaplaincy to study in a more relaxing ‘Living Room’ environment.
Do you think
University staff and students could benefit from more of these community-oriented
spaces and the services and support they offer?
Maya: Yes! Especially if
staff are also involved. Some of us have so few interactions with staff members
because of our limited contact hours.
Tom: Also, I feel like there is a demand at the University for spaces like this one. I mean look at the popularity of the SU Living Room… it is so busy there now. In a way, the space has become a bit too busy, so I still think the Multifaith Chaplaincy is the place for me. We definitely need more community hubs on campus to offset the demand of the SU Living Room and to not run the risk of our social and community spaces quickly becoming overcrowded.
What are your thoughts on the growing importance of the ‘Ways to Well-being’ strategy at the University? What do you think is working and where do you think the University needs to improve?
Emily: This year I know where the well-being advisers are in our department; we receive a lot of e-mails about this. I think the University has done a lot more than people tend to give them credit for. The University is getting better at preventative strategies despite the wait-time for counselling remaining rather disappointing.
Tom: I think overseeing student attendance at lectures would be nice. And it does seem to be working for the courses that already do this. The University should grow from this strength. It’s important to check up on how students are doing, whether they are faring well, especially those who do not feel up for coming into University.
Emily: It would be nice
knowing the university actually cares about us as people beyond our academic
Maya: Also, I think the
fact that we do not meet our personal tutors very often is quite detrimental to
student well-being. I mean my personal tutor meets with me like once a term
officially. Me and so many of my friends feel like we do not know what we are
doing most of the time. Then we get grades and feedback returned and feel
confused as to how we ended up with the grade: good or bad.
In terms of negative feedback, how do you feel
reading back on comments from markers?
Emily: Most of us enter University with optimism and high expectation, we often feel the pressure to make the most out of the experience and excel in the best way we can: whether that is socially, in our extracurricular activities, or in our academic grade. Sometimes, given the random collection of factors and unexpected events, we do not succeed in our personal aspirations at University – this can unsettle us emotionally.
Tom: I guess most of us
don’t feel well-equipped to cope with failure. University needs to prepare
students for failure and educate us on mechanisms for coping and reflecting on
that failure. A disappointing mark is never just an academic failure, but it can
feel like a personal failure as well.
Where do you draw energy and support when you
are feeling vulnerable or a little lost at University?
Maya: I think course mates have become so
important for me. Actually, without them I would feel so lost. We have created
group chats and can help each other out with notes and support each other in
both the administrative and academic sense.
Emily: Yeah, I am lucky because biology is quite a
Tom: Oh really? What? Does everyone really get
on with everyone? My course feels so cliquey.
I point out how the opportunities to forge connections across our academic cohort and to develop a sense of belonging should not be left to mere chance and luck. Instead, the ‘importance of course mates’ should be part of the University Well-being Strategy and we ought to think about how much our teaching and learning spaces are conducive to forging personable connections.
Do you recall memories of a time where you had positive engagement with academic staff and how you benefited from it?
Tom: I actually remember
a time where the absolute inverse happened. I remember a time where I was
snubbed by a member of staff. I was sort of following him after a lecture and I
went over and said “I am really interested in (X) you presented and (Y) in the slide,
could you tell me more about how (Z) might fit into what you are talking about?”
He replied by saying
I should go and research this myself and find it all out for myself. But, you
see, I was trying to do that, but I was confused. Despite expressing interest
and showing engagement I seemed to hit a wall. I felt like this particular
staff member really did not care about me. I think the overemphasis on ‘independent
learning’ makes me feel frequently deflated.
Emily: I agree. I find the whole ‘learn by yourself’ style of teaching quite isolating. If I am trying to engage with staff after a lecture or in consultation hours, then I think we are within our right to ask for a bit more personable support and guidance from staff rather than relying on their signposts to research papers. For me the learning is in the process, and staff should be contributing to that learning process. Sometimes I feel like the only recognizable outcome of our academic pursuits is the grade, but what about the learning process required to construct the essay argument itself? I guess a 2000-word essay can’t really encompass all the intellectual growth spurts we feel throughout the term. Nor can all of our learning be neatly certified in a 60 or 69. Yet we still feel like a failure if we do not receive the numerical grade we hoped for.
Tom: Yeah, failing has
so many negative connotations to it. But sometimes our failures can create
moments of learning. It could be cool for us to reorder the popular narratives
around failure and success. At the end of the day we are all imperfect and we
could use this attribute to transform how we respond to challenging experiences
of disappointment and inadequacy.
Emily: Instead of saying, ‘What grade did you get?’ me and my friends ask, ‘Are you happy with the grade you got?’. We then start to talk about our feelings around expectation and disappointment rather than ending our conversations with a numerical grade.
A few months ago, I was sat in a conference when I got an email from one of the heads of department asking me if I was around. Without thinking, I replied that I was working but not in the office. The subsequent email asked me, in broken English, whether I could purchase £500 in Amazon vouchers and send them back to them. My suspicions were raised so I checked the email address – and, lo and behold, it had come from a scammer.
I spent the rest of the conference not thinking about the topics of discussion but how the scammer could improve the scam – how they might increase their chances of catching me out – and the feedback I should give them. I can’t help it. I love feedback (although I have learnt to keep feedback on the precise science of loading a dishwasher to a minimum over the years).
In today’s episode, I want to talk about feedback. It’s amazing. For me, it is one of the biggest reasons I am in education – to give kind, constructive, thought-provoking and applicable feedback. If you want a great lecture on concrete I am sure there are thousands of YouTube videos just waiting to be discovered (or maybe not) but getting personal feedback really is gold. Being able to present your design to an engineer who can gently ask questions and help a student to realise both what works and what maybe doesn’t is really important.
So when I design a unit ‘feedback’ is one of the items I really focus on.
How can students get feedback? From whom? How can they apply it in the future
on other units? And more importantly how will they apply it when they go into
industry and act as a professional engineer?
I love to draw. And so rather than list the types of feedback I will use, I map them. For ‘Timber 4’ the feedback map looks like this:
So, what on earth is going on here? Well I have tried to show also sorts of different ‘feedback’ mechanisms.
On the left we have the ‘feed-in’ what have they learnt previously – and
what feedback did the students receive which will help them on this unit.
On the right we have the ‘feed-out’ what happens to the feedback I give
the students after the unit – this looks forward both to other units (which at
this point in their degree is quite limited as they only have one term to go
after this unit) but it also looks beyond this – to their life as a practising
engineer – and the skills they will need and the experiences they will have.
The feedback on the final project is designed to support them in another
project – their 40 credit Masters design project. I use the same marking pro
forma and will provide feedback so that they can learn for this next project.
In the middle is all the formative feedback which occurs within the
unit. We might call this feed forward or feedback cycles, I’m not sure what the
exact pedagogic term is, but in my mind, it is where much of the learning occurs.
And it’s where I can bring real value to the students by being involved. I have
tried to build in a number of different mechanisms.
Firstly, I sit in the office and discuss questions that students might have. Some people call this feedback, I actually don’t like this term… I prefer ‘conversation’ or just plain old ‘teaching’! I think it’s useful to differentiate the two, feedback should be focused and specific not just a conversation. This of course doesn’t mean that these conversations are not important, they really are, it’s just I think that if we call them feedback its confusing.
Secondly, students are required to review each other’s work. Every group has a checking log which records the feedback students have produced, every calculation page has a checking box – which should only be signed once the page is checked, and every drawing has the same box. The aim is to get students to support each other’s learning whilst also learning from each other.
Next, students are required to submit their drawings from the first project and these are reviewed both by me (who will provide some generic feedback) and much more importantly, by a timber fabricator who will attempt to cost the students designs based on the information they have provided.
Finally, there is a ‘Quality Assurance Review’. This will involve sitting together with each team and reviewing their progress on all four projects. Three should be complete, and one will be in progress. The three complete projects will be reviewed to ascertain whether they can competently design a number of key components. It will also ensure they have checked each other’s work (a checking log is provided to students beforehand so they can clearly see what they need to do). Once we have reviewed the three projects we will then discuss project 4 (the Quality Assurance Review). This is the summative project which they will be about a third of the way through. The aim of the review is to give some technical feedback (based on projects 1-3) but also provide some feedforward on the project they are working on. This review is not credit bearing, but if I am not convinced that they are competent in certain areas of design I will ask for them to include these again within their final project submission.
feedback – Myth busting
I don’t remember how many times people have said to me – ‘if the
assessment is formative students won’t do it’ – but it’s a lot. I don’t agree.
I think it is much more complex than this. Take the week 3 project for example.
The assessment is formative – but ten out of ten groups submitted drawings.
That’s 100%. Or everyone. So maybe they will do it if they have a good reason?
I like to think that there are lots of good reasons for doing formative
assessment including (but not limited to) it’s fun, it’s interesting, it will
help build a portfolio of work I can show other people, it will help me develop
as I work towards my summative assessment, it helps me to know what I do and
don’t know (although I appreciate it’s rarely that simple). Much of this is
described in detail in ‘Formative
assessment and self-regulated learning: a model and seven principles of good
Feedback on the unit
So finally I thought I’d let you know how the unit is going. I don’t
have any formal feedback, yet. But I am writing a reflective diary every week
so I don’t forget anything. Highlights to date have included:
Some really interesting
external talks – including one on timber gridshells by Jonathan Roynon of
BuroHappold and one on timber architecture by Fergus Feilden who’s Yorkshire
Sculpture Park project was shortlisted for the Stirling prize – the highest
honour in British architecture.
Taking the students to
the Old Vic for a tour – this had two purposes – the Old Vic have agreed to be
the client and they had a brand new entrance built from timber I wanted the
students to see – I loved hearing their conversations as they noticed specific
The buzz of the office
– every week it’s busy – people come and go – but there are always more people
in than out (lot’s of students have other commitments through the day) and the
conversation reminds me of when I used to work in industry – a mix of what you
did the day before and technical discussion.
Students turning up in
work attire (for the most part) every week.
Students not taking
work out the office to continue working on it in their own time (as far as I am
aware) – some students started to raise concerns that they might need to do
this – but rather than pursue that option we reviewed what they were doing and
why they had concerns.
And so at this midpoint in the project (and blog series) it seems to be
Next week – funding and the student BILT fellows will be coming to visit!
Note 1: I decided – for ethical reasons – not to give feedback to the
scammer in the end.
Note 2: My son recently discovered a TED talk by James Veitch on
replying to scammers which we all watched and laughed to – a lot – If you have
ten minutes and need a good laugh I can recommend – https://www.youtube.com/watch?v=C4Uc-cztsJo.
Note 3: I just made up the phrase feed-in and feed-out. I was trying to
think of fun names for the episode and I was trying different variants and they
seem to make sense to me. If you have seen them used before please let me know
so I can reference them in future.
Note 4: Full reference is Nicol, D. J., & Macfarlane-Dick, D., Formative
assessment and self-regulated learning: a model and seven principles of good
feedback practice. Studies in higher
education, 31, 2006.
An effective diagnostic test can play a key role in the language learning process, allowing specific strengths and weaknesses in students’ linguistic development to be identified and then addressed (Jang, 2012). This paper describes the development of an online diagnostic test by Cambridge Assessment English that assesses English grammatical knowledge at A2 level.
As most language tests to date have been proficiency or achievement tests, there has been relatively little research done in the field of diagnostic language assessment and there is no real agreement on exactly what it entails (Alderson, 2005; Alderson, Brunfaut, & Harding, 2015; Davies, 1999; Lee, 2015). It was decided to create something based on the learning-oriented assessment framework (Jones & Saville, 2016) and in response to this lack of research in the field of diagnostic testing. Another aim was to trial a faster, more iterative way of working to better respond to the continuous rapid changes in technology by producing an initial prototype to trial which we could later improve based on the trial results.
Aimed at learners of approximately 15 years old, the test provides detailed diagnostic feedback on seven grammar categories at both individual and class levels, aiming to improve curriculum and lesson planning and accommodate students’ learning needs. The test was trialled internationally and surveys and focus groups were designed to investigate student and teacher perspectives. As well as discussing the results of the trial, the paper also outlines planned modifications for the next version of the Diagnostic Grammar Test and the implications of this research for wider pedagogical practice.
Zoe Backhouse is a BILT Student Fellow and fourth-year Liberal Arts student.
Zine [definition]: some sort of publication, usually mass-produced by photocopying (in some cases scanned, put on the net, or copied via fax) on any range of topics, but usually filled with passion, a means of telling one’s story, sharing thoughts, and/or artwork/ comics/ doodles.
I’m making a Zine about assessment at Bristol Uni. This Zine is going to be creative, visually-engaging and, most importantly, fun!
Zines are great ways to bring narratives together from
different types of people. I’m talking to students and academics across campus
to understand how they experience assessment at the moment and what they want
to change for the future. Assessment is important to us on more than just a
pedagogical level. Talking to Physicists last month, I learned a culture of
self-certifying where students feel so pressured by stacked deadlines that they
tactically decide which exams to opt out of and re-sit in summer. At the same
time, the Physicists also had more of a sense of community than any students I’ve
come across in Arts. Their lab assessments, group projects and tight-knot
relationship with alumni – who frequently post help for problems on their giant
Physics Facebook group – has brought together a huge Physics family. Assessment
can unite and divide us!
I want to understand more about why assessment is so
important for how we experience university, both as teachers and learners. What
concepts are currently discussed in the Higher Education sector that we should
be taking on at Bristol? What good practice is already happening here that more
people should know about? And how can we make the most of our student body,
campus and vibrant city to improve how we assess and feedback?
The Zine will consist of drawings and paintings submitted by
students, snippets from conversations with academics, quotes from student focus
groups and easy-to-read articles condensing theory in HE. It will give
academics and student reps ideas on what’s currently being debated and what
methods we can move as we become a more pedagogically-focussed university.
Hopefully this will also be an opportunity to introduce Zine
as a more mainstream method for presenting information and effecting change! We’d
be behind the USA where universities are already harnessing Zinemaking as a way
to teach – and learn – from their students.
Have some thoughts on
assessment you want included in the Zine? Know someone who would be good for me
to talk to? Want to contribute a doodle, cartoon, sketch or piece of creative
writing responding to the theme of assessment? Email me at email@example.com to be
Reported incidents of ‘contract cheating’ or ghost writing are increasing along with the number of ‘essay mills’ – the providers of bought essays, projects and even dissertations – whose marketing is becoming ever more sophisticated. In this seminar, we will explore the extent, nature and responses to this threat to academic integrity before suggesting ways in which we can alert and inform students and staff, counter the threat, and develop ways of minimising the occurrence of contract cheating in assessment.
Humphrey Bourne is Education Director (PGT) in the Faculty of Social Sciences and Law. He teaches strategy in the School of Economics, Finance and Management and his research focuses on values in and around organizations, extending into research in organizational identity. He chaired the University working group on academic integrity and misconduct, reporting in October 2017, and has since been closely involved in the implementation of its recommendations.
Open to all staff at the University of Bristol who are interested in how to improve feedback conversations in the workplace.
This workshop will be based on the resources created as part of the LeapForward project “Learning for Practice; Feedforward to support transition to the workplace”.
The LeapForward project drew on the experiences of staff and students in healthcare, social work and theatre studies settings at the University of Bristol. Three, one-hour sessions have been developed, along with a resource Toolkit, which can be used to help staff and students understand their role in feedback (feedforward) processes within workplace learning environments. This session will be an opportunity to experience the workshops that have been developed, provide feedback on them as training resources, and consider whether the workshops might be useful for your own teaching.
This workshop is suitable for any staff member working with undergraduate students in workplace settings, and for those with responsibility for training staff/students in workplace assessment processes.
The workshop will be led by Sheena Warman (CHSE assessment lead), Annie Noble (CHSE staff development lead), and Sarah Kelly (LeapForward project Research Associate).
Tea and coffee will be provided for attendees plus lunch (sandwich selection). Therefore please email the firstname.lastname@example.org mailbox at the time of booking your place if you have any special dietary requirements so that we can ensure you are catered for.
Part of a series of sessions organised by the Centre for Health Sciences Education.