Tag: inclusive assessment

News

Moving Assessment Online: Key Principles for Inclusion, Pedagogy and Practice

BILT

This AdvanceHE webinar was chaired by Patrick Baughan with presenters David Carless, Jess Moody and Jess Stokes discussing different aspects. The format of the webinar was that each presenter gave a 10-15 minute presentation (some followed these guidelines more closely than others) and at the end questions were taken and the panel had a discussion.

Screengrab of the three speakers and panel chair on Zoom.

David Carless was the first to speak, covering assessment and feedback in online learning environments. His recent tweets (@CarlessDavid) cover a lot of the material discussed below, but I’ve summarised the main points he addressed below.

Assessment principles:

  • Flexibility and choice to enable – we want to give students opportunity to show best knowledge and performance.
  • Assessment needs to be a partnership with students, rather than something that is done to students.
  • Assessment during this period should be of ‘no detriment’ to our students. We need to provide alternative assessments that can meet the learning outcomes we are looking for – David offered examples of these but you can see a similar list on this DEO page.

Feedback principles:

  • Pedagogy even more than technology should guide planning feedback.
  • Students need to be active in the feedback processes, making meaning from, and acting on, feedback
  • There needs to be a social and interpersonal and relational aspect to feedback, which is even more pertinent at the moment.
  • There also needs to be opportunities for acting upon feedback. Proof of feedback pudding is in the eating! Timing of feedback needs to allow for opportunities – think about peer feedback and internal self-evaluation.

Suggested practices for doing this:

  • Audio and video peer feedback;
    • enables students to make academic judgements and they can compare their own work with peers. In this climate, it can also help develop a sense of community (Filius et al, 2019). In research done with peer assessment in MOOCs, it was shown that multiple peer reviews aligned with self-evaluation of own work were most effective. It can be a really rich process in the composing of peer review.
  • Collaborative writing, e.g. Google Docs – multiple sources of feedback and action works in process.
  • Online quizzes with automated feedback
  • Teaching screencast or give video feedback to students via online conferencing tools. Allows us to build rapport, nuance, trust and builds social presence. Also encourages students to take action and helps develop shared responsibilities.

Workload needs to be wisely deployed – we need to reduce teacher commentary at times when it cannot be taken up.

To summarise:

  • Pedagogy drives technology use
  • We need students to have active involvement in assessment and feedback
  • Social presence, care and trust is of upmost importance
  • Support and coaching for feedback literacy should be available.

Jess Moody then went on to deliver her short presentation on inclusion and online assessment in the Covid-19 pandemic.

She identified the key aspects of the challenge:

  • Decisions about assessment must ensure that all students are equally enabled to demonstrate their learning.
  • The key factors in decision making are changing or unknown (both delivery and health concerns, economic distress).
  • The danger of compounding existing structural inequities – award, progression, grants and careers.

Jess then went on to discuss some priority issues:

  • Digital equity – students do not have equal access to home to both learning materials and access to feedback. Things like internet at home, space and a place to work, privacy at home, access to resources. We need to enable software and hardware for students at home they would normally use on campus.

Safeguarding – Not all of our students are safe at home, need to think of stress of that on top of assessment. Also the online spaces present different challenges (gendered/ racialised issues) for our students.

  • Temporal equity – students are craving normalcy but time is not available equally to students. There are issues around caring responsibilities, health religious observance and access. Students need option to disengage where they can not prioritise assessments. Not all days are equal – students may have part time jobs etc that means they need extra time to complete tasks. We also need to consider how we check in with students wellbeing during this time.

This is not a binary switch from assessment ‘A’ to assessment ‘B’. We need to understand the diversity and uncertainty of individual needs and we have to support their informed choices about things like delaying exams, taking assessments in a different format, etc. We need to give agency and sense of control to student who may otherwise be feeling powerless.

Policy, procedure and impact analysis – we should embed equality impact analysis in decisions about change. Priorities are changing and we need to ensure we have more streamlined extenuating circumstances, resits, progressions rules and deferral and interruption procedures. Certain groups are likely to apply for these more than others, so be prepared for this. Built into all of this needs to be a commitment to reviewing the impact of decision on different groups.

Key principles

  • No one should be left behind – 0identiy our most vulnerable groups
  • Do no more harm – don’t compound existing inequalities in the crisis
  • Be transparent and flexible
  • Support should be first
  • Make sure you understand the impact of your decisions.

Geoff Stoakes – special advisor in advance HE and close involvement in TEF

At this point in the webinar my neighbour came round to drop off some shopping he had picked up for us so I missed the first part of Geoff’s presentation. When I rejoined the webinar, Geoff almost immediately lost his connection to the internet so all I can do is post the slide we were on! Please speak to the AQPO about any quality questions you have.

We then started the discussion element of the webinar while Geoff sorted out his internet connection.

(Geoff did then go on to finish his presentation but at this point I had been listening and writing notes for 50 minutes and was finding it very difficult to concentrate. There was a great deal of text on his slides and he was going through them too fast for me to take good notes. You can see all of his slides on the AdvanceHE website, which provide a good enough summary of what he was saying.)

I walked away from my computer for a couple of minutes to get a drink and have a quick conversation with my husband. This seemed to reset my concentration ready for the final discussion/ questions part of the presentation.

Discussion following the talks covered:

How lecturers could minimise their own bias when marking online – Jess talked about how bias impacts our decisions more when we are stressed, tired, hungry, etc – which is more evident now at the moment. Institutionally how do we support out staff, deadline for markings could be extended, as well as when and how anonymisation is helpful, how you design assessment mitigates bias and continuous monitoring to ensure that we minimise bias where possible. David discussed evaluative judgements and what we can learn from art and design communities and make professional judgements, it is part of their subject to discuss this and so we need to bring it into other discipline conversations.

How to make it easier to record video feedback – David says that sometimes hard-working staff do too much with feedback (and students can find it overwhelming!) – less is more. We need to train students to self-evaluate and make use of peer feedback.

Resources for students for peer feedback – David has covered this is his previous writings (Carless and Winstone, 2019 – ‘meaty’ chapter on it) – we need to train and coach students in how to do it, model our own experiences, sell the benefits, negotiate with them how to tackle the challenges.

Increase in student anxiety with the flexibility offered in assessments – students are worried they might make wrong choice – how do we mitigate this? David has seen this in his research and encountered this – the more choice, the more confusing for students! We need to negotiate choices with them and asking them to think it through. Jess discusses informed choices and how we communicate in different ways – how can we make things as clear as possible? And consider – are there certain choices that may impact on certain groups more than others?  We also need a space where people can come and have that conversation. Why and how are people making certain choices in these times too?

Issues with internet connections – can’t give feedback online – is responsibility of HEI to provide internet access or they need to provide alternative feedback and resources? Jess starts the conversation and says there are legal requirements here that need to be considered depending on where you are in the world. There are moral questions – who are we leaving behind? Other institutions are making funds available for students but internet access is a really difficult one – there are things around proportionality in implementing the Equality Act. Geoff adds that some universities are partnering with a company to ensure students have laptops. We also need to consider alternative forms of assessments that allow for students that do not have internet access.

Recording of this webinar is in the Advance HE Connect membership benefit series, also in Teaching and Learning forum. Advance HE Connect is available as an app on iOS and Android.

If you’re thinking of a doing a webinar, make it shorter than an hour unless you build in long enough breaks for people to have a concentration reset!

Amy Palmer

News

Insights from attending UWE’s Festival of Learning for an afternoon

BILT

The following post was written by Emilie Poletto-Lawson, a BILT Fellow and Educational Developer in Academic Staff Development.

The Academic Practice Directorate at the University of the West of England (UWE) is the equivalent of Academic Staff Development[  team at the University of Bristol. They organised a week-long winter festival of learning [] after the success of their one day Learning and Teaching Conference which started in 2011. This year, they aimed to “create a buzz about Learning and Teaching to coincide with the NSS survey”. I attended one afternoon but it was fantastic to see students and staff come together to share their enthusiasm for learning and teaching.

The first half of the afternoon was entitled “Fresh approaches to T&L – A session in our new laundry space to get you inspired” led by Dr Laura Bennett (Associate director – academic practice directorate). The session included members of staff that delivered sessions in the new ‘laundry room’ as well as students that were attending sessions there but who were also using the room for extra-curricular activities.

I valued the opportunity to visit the Glenside campus of the University of the West of England to discover the laundry  room. At a time when a lot of thinking is going into teaching spaces in our University (BILT symposium June 2018 , BILT fellows working on space and design of Temple Quarter) it is always enriching to see what colleagues are experimenting with.

Before telling you about the presentation, let me tell you about the room. When we came in, it was a big empty space. I must say it had a medical feel to it, very white, sink at the back, metal shelves, not a warm atmosphere but perfect for its intended purpose: “a practical learning space for trainee optometrists, paramedics and occupational therapy students”. The facilitators were coming straight from another session on the other side of the campus so we had to build our classroom which was in itself a nice way to feel like we belonged and it was our space.

The room can be described as a “connected classroom”. There are four screens on both side walls, connected to a keyboard that enables students to use the screen as a group and it is also possible for the facilitator to show the main screen on all screens or to display the students’ screen on the main screen/all the screens. If you were able to attend a session during the digital classroom roadshow two years ago (June 2017) the set up was very similar apart from the fact that the tables were not fixed to the floor.

As the idea was to experience the technical aspect of the room we built our on wheels foldable table next to the screen and sat on high stools (not very easy if you have short legs like me) ready to roll. As I managed to sit down I realised my bag was quite far down from me on the floor and I had nowhere to put my coat. I was also quite far from the front as the room can open on both sides to create an even bigger space so the screens are at the back. Having moved the weekend before it did not take long for my back to start hurting but I was not quite sure what to do when another lady voiced the same issue and was given the option to grab a heavy chair instead of the stool. It was good to have an option but the chair was considerably lower creating some difficulties if you wanted to work from the table. Final hurdle for me as a non-native speaker, a fan covered the voices of speakers that did not use a microphone and it was a real strain to keep up.

However, despite all that, I still think it was a great workshop because it was about possibilities, about teaching differently and the space supporting your approach and ideas rather than limiting you. If you came into the room and lectured for three hours just talking at students, you would be missing the huge opportunities the toom has to offer to make your students more active, to encourage and facilitate group work, peer learning etc.

Laura Bennett introduced the aim of the session and presented key ideas from the literature regarding space and concluded that “Space should be what you need it to be”. The next speaker was Liz Reilly (Senior lecturer, social work) whose presentation “The Laundry in action – pitfalls and possibilities” gave a very engaging insight into the use of the room. Liz was very positive regarding the possibilities the room offered for learning and teaching:

  • Create groups based on theme 
  • Carousel approach – screens act as flipchart 
  • Moving from one table to the next made the students were very active 

However, she also picked up on the inclusivity issues I mentioned earlier and some other practical aspects.

  • Inclusion: comfort, high tables are a problem for people who cannot spent too much time on a stool and for wheelchairs, far away so lip read or hearing impairment 
  • Booking of the room, paperwork involved
  • Groups complained they could not hear what the lecturer said to specific groups 
  • Finally, being faced with one of her students lying on the floor to do back exercises despite the active approach she had in place was definitely not an outcome she expected.

Here are her pieces of advice:

  • Play around in the room 
  • Play around with what you are doing 
  • Log in ahead of your session and test everything: screens, keyboards, etc. 
  • Have a conversation with people managing the room 
  • Get feedback from students 
  • Get someone to observe you 

The following presentation, “Simulation: the Laundry as Emergency Room” byAimee Hilton (Senior Lecturer, Adult Nursing) took the original idea behind the design of the room and took it quite a few steps further. She transformed the Laundry into an emergency room treating the victims of a mass casualty event for paramedics, radiographers and nurses students. Drama students joined students paramedics, radiographers and nurses from other years to play the roles of patients. She also involved journalists students whose aim was to get as much in the way as possible journalists would should such an event take place. The university security team, fire brigade and ambulance crew also joined in to add to the realism of the situation. Did I mention professional make-up? Now, I must admit I would have loved to be a fly on the wall. The feedback from the students was extremely positive. It was very interesting that the hardest part of the planning was recruiting enough actors. I particularly liked the multi-disciplinary approach of the project.

The last presentation was by three students from the pre-hospital simulation society who study in the room but also used it for one of their events. The society provides “student led learning with the aim to facilitate realistic quality simulations to improve clinical competency and confidence within student Paramedics”. The Laundry is only one example of location, they have created simulations in a car, outside, during freshers’ fair etc. The idea is to design simulations of rare situations so that students are better prepared should it ever happen to them in their professional life. Each simulation is followed by a debrief at the end looking at what went well, what the literature says about such a situation etc. Their enthusiasm and commitment were exemplary.

Finally, Laura Bennet concluded the session with a tour of the side rooms and suggestions of technology to use to make your teaching more interactive. If you have attended CREATE workshops, you will recognize a few of those:

I am very grateful for the opportunity to attend the event and I feel I have learnt a lot. For me the main take away is that we need to make the space work for us and to be mindful of who will be in the room and how accessible out teaching as well as the room are.

News

Event Summary of ‘Making IT* Happen: from strategy to action’ at the University of Leicester

BILT

*Inclusive teaching

The following post was written by Emilie Poletto-Lawson, who has been a BILT Fellow since January 2018. 

I am a BILT fellow (based in Academic Staff Development), working on the BILT theme of inclusive assessment. On the 14th of November 2018 I attended a two-part workshop lead by Pete Quinn (Pete Quinn Consulting) and Dr Mike Wray (inclusininhe.com) that explored inclusion from theory to practice at University of Leicester.

The first part of the day focused on inclusivity from a disability perspective and the session looked at the current situation in universities regarding inclusivity and reflected on where we are at 8 years after the Equality Act was introduced.  Even though all stakeholders agree on inclusivity in principle, “making it happen” can prove somewhat difficult as new initiatives can meet resistance. Lecture Capture (new in some institutions) would be a very good example, it is a very important step for a great number of students and, in particular, students with a disability and yet this practice is still being greatly challenged by lecturers within universities and in the news. The main criticism is that students no longer attend lectures but the USS pensions-strike in 2018 also highlighted issues regarding who owns the rights to the recordings.

While universities must assure legal compliance, it is important to define what it is “reasonable” when it comes to reasonable adjustments we make to support our students learning. Pete Quinn highlighted the risk, stated by the Inclusive Teaching and Learning in Higher Education as a Route to Resilience publication, of “insufficient institutional oversight” which could lead to “unduly influenced [decisions] which are made by what individual members of staff perceive as reasonable” without a proper understanding of what the Equality Act requires or with appropriate emphasis being placed on relevant considerations”. It is therefore essential to break silos of practice within an institution and ensure all the relevant stakeholders work together to assure consistent and appropriate adjustments are in place.

Pete Quinn also presented the overview by Abi James (Assistive Learning Ltd) of public sector website and application regulations that all websites will need to comply with to ensure accessibility for all. Websites and content shared on intranets and extranets (this includes Blackboard) created after 23.09.18 will need to be compliant by September 2019, with anything created before 23.09.18 has September 2020 as a deadline. Finally, the deadline for mobile apps will be September 2021. You can find more information on this on the government’s website.

After looking at the work of the Office for Students, the Office of the Independent Adjudicator and the equality and human rights commission we reflected on our institutions’ situation regarding the following themes:

  • Non-medical help provision
  • Assistive technology
  • Lecture capture
  • Inclusive teaching, learning and assessment policy
  • Inclusive teaching, learning and assessment in practice
  • Placement and internships

 

In the afternoon, Mike Wray presented Inclusive Learning and Teaching and Assessment Framework (ILTAF), an audit framework to help universities improve their level of inclusivity. The framework contains four sections:

  • Quality assurance
  • Before teaching
  • During teaching
  • Assessment

We were also given time to discuss inclusive learning and teaching in our institutions with a view to share good practice, agree goals and take stock in the future.  I am very grateful to colleagues from the Universities of Leeds, Durham, Bath and Edinburgh for very interesting exchanges.

What next?

A number of questions arose from the day and left me wondering how best to initiate change and champion inclusivity within Bristol and what can we do in our practice to support this?  I will be working on developing a self-assessment document that would support unit/programme leads in reviewing inclusive assessment (and teaching) practices across their units and programmes.

If you are interested in finding out more on inclusive practice, you might want to look into the following MOOC, from the University of Southampton, recommended by the speakers: Inclusive Learning Teaching.

Here are a few reading suggestions from the event:

On key actors and texts regarding inclusivity mentioned by the speakers:

http://www.legislation.gov.uk/ukpga/2010/15/pdfs/ukpga_20100015_en.pdf

https://www.officeforstudents.org.uk/

http://www.oiahe.org.uk/

https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/587221/Inclusive_Teaching_and_Learning_in_Higher_Education_as_a_route_to-excellence.pdf

https://www.equalityhumanrights.com/en

 

 

News

Should all assessments be inclusive?

BILT

The following post was written by Emilie Poletto-Lawson, an Educational Developer and BILT Fellow. 

I am a BILT fellow based in Academic Staff Development where I work as an Educational Developer. I have been working on the BILT theme of assessment – focusing on inclusive assessment since February 2018. I am undertaking a literature review with a view to making recommendations around inclusive assessment principles that we can embed into our units and programs at the University of Bristol to work alongside our institutional principles on Assessment and Feedback.

From my reading to date the  main take away is that inclusivity is predominantly discussed as a means for supporting students with disabilities. It is very much viewed as a deficit approach to considering assessment, however, I strongly believe it is far more than that, we want to be inclusive of all learners and for inclusive assessment to actually be more inclusive.

As part of my BILT fellow role I recently attended an event at the University of Leicester called “Making IT* Happen: from strategy to action (*Inclusive Teaching)’, led by Pete Quinn and Mike Wray (blog available here). The focus was very much on supporting disabled students in our institutions and ensuring universities are legally compliant with the Equality Act. In preparation for the event, the experts highlighted good practice in the work we do at Bristol, for example we received positive feedback on our institution website regarding inclusivity (http://www.bristol.ac.uk/disability-services/study-support/reasonable-adjustments/) and in particular videos created by Louise Howson from Academic Staff Development (http://www.bristol.ac.uk/staffdevelopment/academic/resources/learning-and-teaching-resources/learning-and-teaching-videos/ ).

Regarding the literature review I am working on, when researching the key words “inclusive” “assessment” in “higher education”, I obtained 9596 results in ERIC and yet, going through the abstracts not that many articles encompass all three parameters. It appears there might be a gap in the literature here despite inclusivity being key to university strategies in the UK and beyond for a number of years now. So far, the key emerging themes from my searching can be seen below.

Inclusive Assessment in Higher Education map created by Emilie Poletto-Lawson
Created with Mindmeister 21.09.2018

In the US literature the Inclusive aspects of articles relates to the idea of an inclusive campus and looks at inclusivity from the selection process (access) to the students completing their degree (success). In the UK, the literature shows there is an acknowledged need for policies, strategies and processes as well as professional development to bring about inclusive practices.

Initial readings suggest there is a rhetoric of inclusivity as a given good, but it is difficult to identify concrete examples, especially when it comes to assessment. The literature review is the first step to articulate a clear definition before focusing on what inclusive assessment means for us at the University of Bristol.

If you are interested in this topic why not read “Against being Inclusive” by Jeffrey Carlson, interim provost and Vice President for Academic Affairs at Dominican University? I appreciate it might be an odd recommendation since this post advocates that all assessments should be inclusive, but I think this article, published in 2016, does offer food for thought and reinforces the need to clearly define what we mean by “inclusivity” before we move to making recommendations at Bristol.

Professor Debby Cotton and Dr Rebecca Turner at their Education Excellence Seminar
News

Easing the transition of undergraduates through an immersive induction module

BILT

The opening Education Excellence seminar of 2018/19 took place on Thursday 20th September in 43 Woodland Road. Professor Debby Cotton and Dr Rebecca Turner (accompanied by Rebecca’s son, Thomas) came up from Plymouth University’s PedRIO to deliver a seminar on the immersive induction module all undergraduates take at their institution. The seminar was attended by almost fifty members of staff and was a great start to the 18/19 seminar series– there were only two minor hiccups; the first being the hospitalisation of Rebecca’s childminder (cue baby gurgles throughout the lecture) and the fact the RePlay box was still on its summer break (cue this blog post).

After a brief introduction from Alvin, Rebecca introduced the project to the audience. The project was proposed after it was recognised that students were struggling with the transition to university. It was hoped the immersive module would help with social integration, as well as allowing students a transitional period when beginning their university studies.

Following a successful pilot year in 2014, the immersive induction module was rolled out across all undergraduate programmes in the University. The module is a four-week introduction to the degree – students do not undertake any other modules during this time, in which they can focus on getting to grips with self-study, academic skills, the language of their subject. The module also gives students the opportunity to get to know others on their course through the use of group work. Team-building, peer interaction and academic integration are all used to boost motivation and enthuse students. Most students are asked to complete an assessment, designed to be inclusive, at the end of the module, which provides students with early feedback and reduces exam-linked anxiety.

It was hypothesised that this module would improve retention and student attainment – and it did. Initial results from the pilot showed retention approved across the board, with students naming a sense of belonging, academic integration, social integration and strong study skills as being key factors in the improvement. Peer collaboration and networking grows due to the collaborative work that takes place early in the programme. The average grade from first assignments went up from 62% to 67%, despite the fact the individual student needs had not always been recognised at this point. Both genders showed heightened performance, thought the enhancement was greater for males, therefore reducing the attainment gap.

There were a number of challenges that the immersive module has presented. One of the biggest issues caused was that it raised the students expectations to a level where they could not be maintained in modules going forward. Further to this, students felt like a ‘second transition’ had been created, and still struggled to an extent when the immersive module ended. Some students did not want to take part in self-directed study and group work at the beginning of their degree; they expect to be in lecture theatres and have their questions answered by the lecturer. Some lecturing staff were not enthusiastic about changing their way their subject was taught so it could be covered in one module, too. There were a number of operational issues that presented as part of the roll-out. Teaching spaces weren’t always ideal – though the majority of sessions took place in ‘flatbed’ spaces, a number of large lecture theatres had to be used and weren’t viable for interactive teaching as they are too large and become noisy.

Overall the project was very successful, increasing the retention and attainment of first-year students and generally improving the student experience, though this has come with some new challenges. We were left with a number of question to consider when thinking about whether we could implement a similar structure, including:

  • What opportunities could an immersive format offer you?
  • What challenges or concerns would you have?
  • How could an immersive format help create a sense of belonging?
  • How can we manage student expectations of HE on arrival?
  • How can we better prepare students to progress on to subsequent modules?

Look out for the next edition of our ‘An interview with’ series with Debby and Rebecca coming in October.

The full peer-reviewed paper can be found here: https://www.tandfonline.com/doi/abs/10.1080/13562517.2017.1301906

Meet the BILT Fellows

Meet the BILT Fellows: Emilie Poletto-Lawson

BILT

We asked our Fellows to write us a short blog about their background and what they are doing as part of their BILT Fellowship. The following blog is from Emilie Poletto-Lawson, who has been a BILT Fellow since February 2018.

Based in the School of Modern Languages, I specialise in teaching French both in the degree programme and University-Wide Language Programme. My areas of interest are student motivation, feedback and digital enhancement of learning.

I started my BILT fellowship on inclusive assessment on the 1st of February 2018. Looking at how best to design a curriculum and assessments to meet the needs of all our students while maintaining strong academic content is a highly motivating challenge. I strongly believe that education is our greatest asset on the path to equality, and it is with this in mind that I am researching this theme.

Looking at inclusivity, my first task is defining “inclusive assessment”, encompassing all of its dimensions as it has been evolving, and looking at the myth of inclusive assessment as a “dumbing down” of education. My postulate is that acts such as the 1995 Disability Discrimination Act and the 2010 Equality Act were catalysts for this field, but I am still at the beginning of my journey.

If you are interested in this topic, I would recommend the following article, as it raises some very interesting questions and tackles the main criticism of inclusivity, that it is “dumbing down” education:

Haggis, T. (2006) Pedagogies for diversity: retaining critical challenge amidst fears of ‘dumbing down’. Studies in Higher Education, 31(5), pp.521–535. [Accessed 15.03.2018] http://www.storre.stir.ac.uk/bitstream/1893/457/1/Haggis_Pedagogies_for_diversity_2006.pdf

The aim of my research is to learn from the literature to inform our practice at the University of Bristol. At this stage I have many questions. What does an inclusive syllabus look like? What does an inclusive educator look like? What should educational development for the latter look like? What is the first step? And what are the next ones? Who can/should initiate change? Is inclusivity a way to improve engagement? What does all this mean for us at UoB? These are but a few!

If you would like to discuss inclusivity in assessment, do not hesitate to contact me.

You can read my blog post on the ‘Attendance vs Engagement’ debate on the BILT Blog.