News

Moving Assessment Online: Key Principles for Inclusion, Pedagogy and Practice

This AdvanceHE webinar was chaired by Patrick Baughan with presenters David Carless, Jess Moody and Jess Stokes discussing different aspects. The format of the webinar was that each presenter gave a 10-15 minute presentation (some followed these guidelines more closely than others) and at the end questions were taken and the panel had a discussion.

Screengrab of the three speakers and panel chair on Zoom.

David Carless was the first to speak, covering assessment and feedback in online learning environments. His recent tweets (@CarlessDavid) cover a lot of the material discussed below, but I’ve summarised the main points he addressed below.

Assessment principles:

  • Flexibility and choice to enable – we want to give students opportunity to show best knowledge and performance.
  • Assessment needs to be a partnership with students, rather than something that is done to students.
  • Assessment during this period should be of ‘no detriment’ to our students. We need to provide alternative assessments that can meet the learning outcomes we are looking for – David offered examples of these but you can see a similar list on this DEO page.

Feedback principles:

  • Pedagogy even more than technology should guide planning feedback.
  • Students need to be active in the feedback processes, making meaning from, and acting on, feedback
  • There needs to be a social and interpersonal and relational aspect to feedback, which is even more pertinent at the moment.
  • There also needs to be opportunities for acting upon feedback. Proof of feedback pudding is in the eating! Timing of feedback needs to allow for opportunities – think about peer feedback and internal self-evaluation.

Suggested practices for doing this:

  • Audio and video peer feedback;
    • enables students to make academic judgements and they can compare their own work with peers. In this climate, it can also help develop a sense of community (Filius et al, 2019). In research done with peer assessment in MOOCs, it was shown that multiple peer reviews aligned with self-evaluation of own work were most effective. It can be a really rich process in the composing of peer review.
  • Collaborative writing, e.g. Google Docs – multiple sources of feedback and action works in process.
  • Online quizzes with automated feedback
  • Teaching screencast or give video feedback to students via online conferencing tools. Allows us to build rapport, nuance, trust and builds social presence. Also encourages students to take action and helps develop shared responsibilities.

Workload needs to be wisely deployed – we need to reduce teacher commentary at times when it cannot be taken up.

To summarise:

  • Pedagogy drives technology use
  • We need students to have active involvement in assessment and feedback
  • Social presence, care and trust is of upmost importance
  • Support and coaching for feedback literacy should be available.

Jess Moody then went on to deliver her short presentation on inclusion and online assessment in the Covid-19 pandemic.

She identified the key aspects of the challenge:

  • Decisions about assessment must ensure that all students are equally enabled to demonstrate their learning.
  • The key factors in decision making are changing or unknown (both delivery and health concerns, economic distress).
  • The danger of compounding existing structural inequities – award, progression, grants and careers.

Jess then went on to discuss some priority issues:

  • Digital equity – students do not have equal access to home to both learning materials and access to feedback. Things like internet at home, space and a place to work, privacy at home, access to resources. We need to enable software and hardware for students at home they would normally use on campus.

Safeguarding – Not all of our students are safe at home, need to think of stress of that on top of assessment. Also the online spaces present different challenges (gendered/ racialised issues) for our students.

  • Temporal equity – students are craving normalcy but time is not available equally to students. There are issues around caring responsibilities, health religious observance and access. Students need option to disengage where they can not prioritise assessments. Not all days are equal – students may have part time jobs etc that means they need extra time to complete tasks. We also need to consider how we check in with students wellbeing during this time.

This is not a binary switch from assessment ‘A’ to assessment ‘B’. We need to understand the diversity and uncertainty of individual needs and we have to support their informed choices about things like delaying exams, taking assessments in a different format, etc. We need to give agency and sense of control to student who may otherwise be feeling powerless.

Policy, procedure and impact analysis – we should embed equality impact analysis in decisions about change. Priorities are changing and we need to ensure we have more streamlined extenuating circumstances, resits, progressions rules and deferral and interruption procedures. Certain groups are likely to apply for these more than others, so be prepared for this. Built into all of this needs to be a commitment to reviewing the impact of decision on different groups.

Key principles

  • No one should be left behind – 0identiy our most vulnerable groups
  • Do no more harm – don’t compound existing inequalities in the crisis
  • Be transparent and flexible
  • Support should be first
  • Make sure you understand the impact of your decisions.

Geoff Stoakes – special advisor in advance HE and close involvement in TEF

At this point in the webinar my neighbour came round to drop off some shopping he had picked up for us so I missed the first part of Geoff’s presentation. When I rejoined the webinar, Geoff almost immediately lost his connection to the internet so all I can do is post the slide we were on! Please speak to the AQPO about any quality questions you have.

We then started the discussion element of the webinar while Geoff sorted out his internet connection.

(Geoff did then go on to finish his presentation but at this point I had been listening and writing notes for 50 minutes and was finding it very difficult to concentrate. There was a great deal of text on his slides and he was going through them too fast for me to take good notes. You can see all of his slides on the AdvanceHE website, which provide a good enough summary of what he was saying.)

I walked away from my computer for a couple of minutes to get a drink and have a quick conversation with my husband. This seemed to reset my concentration ready for the final discussion/ questions part of the presentation.

Discussion following the talks covered:

How lecturers could minimise their own bias when marking online – Jess talked about how bias impacts our decisions more when we are stressed, tired, hungry, etc – which is more evident now at the moment. Institutionally how do we support out staff, deadline for markings could be extended, as well as when and how anonymisation is helpful, how you design assessment mitigates bias and continuous monitoring to ensure that we minimise bias where possible. David discussed evaluative judgements and what we can learn from art and design communities and make professional judgements, it is part of their subject to discuss this and so we need to bring it into other discipline conversations.

How to make it easier to record video feedback – David says that sometimes hard-working staff do too much with feedback (and students can find it overwhelming!) – less is more. We need to train students to self-evaluate and make use of peer feedback.

Resources for students for peer feedback – David has covered this is his previous writings (Carless and Winstone, 2019 – ‘meaty’ chapter on it) – we need to train and coach students in how to do it, model our own experiences, sell the benefits, negotiate with them how to tackle the challenges.

Increase in student anxiety with the flexibility offered in assessments – students are worried they might make wrong choice – how do we mitigate this? David has seen this in his research and encountered this – the more choice, the more confusing for students! We need to negotiate choices with them and asking them to think it through. Jess discusses informed choices and how we communicate in different ways – how can we make things as clear as possible? And consider – are there certain choices that may impact on certain groups more than others?  We also need a space where people can come and have that conversation. Why and how are people making certain choices in these times too?

Issues with internet connections – can’t give feedback online – is responsibility of HEI to provide internet access or they need to provide alternative feedback and resources? Jess starts the conversation and says there are legal requirements here that need to be considered depending on where you are in the world. There are moral questions – who are we leaving behind? Other institutions are making funds available for students but internet access is a really difficult one – there are things around proportionality in implementing the Equality Act. Geoff adds that some universities are partnering with a company to ensure students have laptops. We also need to consider alternative forms of assessments that allow for students that do not have internet access.

Recording of this webinar is in the Advance HE Connect membership benefit series, also in Teaching and Learning forum. Advance HE Connect is available as an app on iOS and Android.

If you’re thinking of a doing a webinar, make it shorter than an hour unless you build in long enough breaks for people to have a concentration reset!

Amy Palmer

News

Neurodiversity and Digital Accessibility

Last week we hosted the third of our Digital Accessibility events, this time with Dafydd Henke-Reed, Senior Accessibility Consultant with AbilityNet. Dafydd has been diagnosed with Autism and Dyslexia and spoke about his personal experiences of Neurodiversity.

Dafydd speaking at the event.

Dafydd was engaging and open about his experiences growing up, going to University and the technology he uses day to day. From the very start he highlighted that Autism is a spectrum and that we were hearing what Neurodiversity means to him.

From Cognitive Brick Walls to being horrified when friendly lecturers asked him to move forward from the back row of a lecture theatre, we heard about the barriers and obstacles he had faced.

What stood out for me…

“Dyslexia could be solved with tools; Autism was about learning how to thrive in a seemingly hostile culture.”

Dafydd had refused support related to Autism at University. Tactics such as large yellow “appropriate allowance when marking” stickers felt like a brand. This is pertinent; many students may not disclose their “disabilities” due to previous experience or because they find allowances intrusive or counterproductive. In fact, with conditions such as Autism Spectrum Disorder may not consider it a disability in the first case, it’s just the way they are. If we are to be truly inclusive, then we need to design our learning experience to remove barriers and everyone benefits.

“Come over for group study and we’ll get beers and pizza in? Hell no!”

Dafydd spoke about how he found groups and teamwork challenging. He’ll use digital tools like Slack or instant messaging to communicate rather than walking to a colleague’s desk. He also praised electronic tickets (“I won’t lose them”)

He showed us the Speech to Text (STT) and Text to Speech (TTS) systems he uses every day along with the spelling correction functionality.

Do’s and don’ts

The excellent UK Gov “Do’s and Don’ts” guides were given a name check again, this time for Dyslexia and Autism. If you haven’t seen them, check out these lovely visual guide posters. I think they should be printed out in every office!

Designing for users on the autistic spectrum. Do use simple colours; write in plain language; use simple sentences and bullets; make buttons descriptive; build simple and consistent layouts. Don't use bright contrasting colours; use figures of speech and idioms; create a wall of text; make buttons vague and unpredictable; build complex and cluttered layouts. Designing for users with dyslexia. Do use images and diagrams to support text; align text to the left and keep in a consistent layout; consider producing materials in other formats (for example audio or video); keep content short, clear and simple; let users change the contrast between background and text. Don't use large blocks of heavy text; underline words, use italics or write in capitals; force users to remember things from previous pages - give reminders and prompts; reply on accurate spelling - use autocorrect or provide suggestions; put too much information in one place.

Martin Nutbeem

Teaching Stories

Getting started with Experiential Learning

Who doesn’t like an adventure, a dive into the unknown, an unexpected challenge and the possibility of transforming who you are and how you think about the world? That’s what experiential learning can offer students by inviting them to learn through doing.

In my own experience, I’ve taken students on fieldtrips to conduct guided research (interviewing communities or documenting historic spaces) and placed them as leaders in public engagement (designing exhibitions or creating dynamic online content). In one month alone, my students volunteered over 500 hours to engaged with 370 members of the public in person and thousands more online through blogs and social media channels.

I evaluated these efforts and demonstrated that confidence across a range of skills went through the roof. Of the students surveyed, 100% stated that these skills would help their future career, and 100% would recommend the experience to other students.

The qualitative responses to the survey draw out how students think about these experiential learning opportunities:

  • It was a really good opportunity to try something I hadn’t done before and the chance to complete something independently but with good leadership.
  • We were given clear instructions but also given the opportunity to make our own choices and decisions, with support when needed.
  • Allowing people to choose which tasks they wanted to work on kept people motivated and enthusiastic. The varied roles allowed the development of a number of skills.
  • There was a good balance between having an opportunity to be creative and do our own thing as well as having a directive.
  • Clear explanation of goals and transferable skills, enthusiasm for engagement, focus on making the most of individual student’s skill sets, benefiting both the project and the individual.

In this blog, I outline how experiential learning can enhance your teaching practice, where it comes from, and suggest next steps for how you can incorporate it within your teaching practice on or off campus. Handy hints on tackling logistics are included too.

Context

Experiential learning can manifest in a multitude of ways to suit your degree programme. From civic engagement, to project-based research, study abroad opportunities, service learning, internships and laboratory classes. At the heart of this approach is the learner’s experience during the process, rather than the mode of delivery of the experience.

Students love experiential learning. It takes them out of the ordinary and into a new learning space that increases their enjoyment and encourages deep learning (Wurdinger & Renton Allison 2017). It also fosters a self-questioning approach that leads to meaningful personal reflection (Cacciamani 2017 p.28).

Experiential learning can transform the curriculum into one that enhances students’ sense of culture and values (connectedness, capability, resourcefulness, purpose (Pitchford & Hendy 2019). Skills for employability are a common benefit of experiential learning offerings (Rainey 2014).

Theory

Experiential learning draws on the research of Roger Saljo (1979) who found that students internalise their learning best through the experience-based processes. In the 1980s, the concept gathered momentum and was explored further, most notably by David Kolb (1984).

Kolb created four categories of interchangeable learning styles that students might encounter through experiential learning: activists, reflectors, pragmatics and theorisers. He also imagined the cognitive processes of learning: a cycle of experience, critical reflection, active experimentation, and abstraction (see also Zull 2002).

Kolb’s theories were subsequently criticised for not taking account of wider pedagogic concerns (e.g. by Rogers 1996 p.108) but are still an influential reference point for teaching and learning developers (e.g. Tomkins & Ulus 2015).

Campus learning

Campus-based teaching can invite external stakeholders (community, commercial, city-based) to offer lectures or practicals in the classroom (Cacciamani 2017). This provokes students to consider new perspectives drawn from real-world contexts and community knowledge (on civic and eservice learning see Strait & Nordyke 2015).

The concept of a “Living Lab” takes advantage of University-run spaces to invite tangible and visible interaction with the campus. Living labs are low cost and have less red tape than other types of experiential learning opportunities. They promote the idea of University spaces as a learning resource and take a holistic approach to learning whereby students link their learning to practice action. Contact External Estates to explore options for on-campus learning.

Off-campus learning

Learning can take place off-campus in sites across the city and further afield (see Urban Spaces. Civic University blog). Domestic and international travel, either independent research trips or group fieldwork, can offer additional routes to engage with authentic problems, interdisciplinarity, exploration, engagement and concepts of global citizenship (see Hull et al. 2016).

Logistical barriers to any excursions include financial overheads, access permissions, and health and safety considerations (see Munge, Thomas & Heck 2017). This latter concern can be mitigated by using existing resources and risk assessment templates prepared by the University Safety and Health Services Team. The International Office can advice and support international opportunities offered to our undergraduate students. Additionally, you will need to ensure adequate insurance cover (contact the Secretary’s Office for further information).

For some programmes, internships and work placements may offer suitable routes for off-campus experiential learning. The LeapForward Project is a good example of how educational initiatives can support transitions into workplace-based learning environments, see https://bilt.online/the-leapforward-project/.

References

Austin, M.J. &  Rust, D.Z., 2015 Developing an Experiential Learning Program: Milestones and Challenges. International Journal of Teaching and Learning in Higher Education 27(1) pp.143-153 https://eric.ed.gov/?id=EJ1069800

Cacciamani, S. 2017 Experiential learning and knowledge building in higher education: An application of the progressive design method. Journal of e-Learning and Knowledge Society 13(1) pp. 27-37 https://doi.org/10.20368/1971-8829/1245

Hull, R.B., Kimmel, C., Robertson, D.P. & Mortimer, M. 2016 International field experiences promote professional development for sustainability leaders. International Journal of Sustainability in Higher Education 17(1) pp.86-104 https://doi.org/10.1108/IJSHE-07-2014-0105

Kolb, D. 1984 “Experiential Learning: Experience as a Source of Learning” (Prentice Hall, New

York)

Munge, B., Thomas, G., & Heck, D. 2018 Outdoor Fieldwork in Higher Education: Learning From Multidisciplinary Experience. Journal of Experiential Education 41(1) pp. 39-53 https://doi.org/10.1177/1053825917742165

Pitchford, A. & Hendy, J. 2019 Embracing the university: Experiential solutions for effective transitions. Teaching and Learning Conference 2019, AdvanceHE https://www.advance-he.ac.uk/knowledge-hub/teaching-and-learning-conference-2019-embracing-university-experiential-solutions

Rainey, B. 2014 Teaching for the real world: creating materials for experiential learning: The law in action. Briefing paper for EvidenceNet, Learning and Teaching Enhancement Fund, Wales; AdvanceHE https://www.advance-he.ac.uk/knowledge-hub/teaching-real-world-creating-materials-experiential-learning-law-action

Rogers, A. 1996 “Teaching Adults” (2nd ed.) (Open University Press: Buckingham)

Saljo, R. 1979 Learning in the learner’s perspective: I. Some common-sense conceptions. Reports from the Institute of Education (76) University of Gothenberg https://eric.ed.gov/?id=ED173369

Strait, J. R. & Nordyke, K. 2015 “eService-Learning: Creating Experiential Learning and Civic Engagement Through Online and Hybrid Courses” (Stylus Publishing: Sterling, Virginia) [available via Google Books]

Tomkins, L. & Ulus, E. 2015 ‘Oh, was that “experiential learning”?!’ Spaces, synergies and surprises with Kolb’s learning cycle. Management Learning 47(2) pp. 158-178 https://doi.org/10.1177/1350507615587451

Wurdinger, S., & Allison, P. R. 2017 Faculty perceptions and use of experiential learning in higher education. Journal of E-Learning and Knowledge Society 13(1) pp. 15-26 https://doi.org/10.20368/1971-8829/1309

Zull, J. E. 2002 “The art of changing the brain: Enriching teaching by exploring the biology of learning” (Sterling, BA: Stylus)

Further reading

Kolb, A. Y. & Kolb, D. A. 2005 Learning Styles and Learning Spaces: Enhancing Experiential Learning in Higher Education. Academy of Management Learning & Education 4(2) pp. 193-212 https://www.jstor.org/stable/40214287

Liedtke, C., Jolanta Welfens, M., Rohn, H. & Nordmann, J. 2012 LIVING LAB: user‐driven innovation for sustainability. International Journal of Sustainability in Higher Education 13(2) pp. 106-118 https://doi.org/10.1108/14676371211211809

Plumpton, H. 2010 ‘Bridging the gap’ between theory and practice – situative learning and experiential techniques in the lecture theatre. EvidenceNet, University of Wales Institute; AdvanceHE https://www.advance-he.ac.uk/knowledge-hub/bridging-gap-between-theory-and-practice-situative-learning-and-experiential

Roberts, J.W. 2015 “Experiential education in the college context: What it is, how it works, and why it matters” (Routledge: New York)

Wurdinger, S. & Allison, P., 2017 Faculty perceptions and use of experiential learning in higher education. Journal of e-learning and Knowledge Society 13(1) https://doi.org/10.20368/1971-8829/1309

Dr Aisling (Ash) Tierney – a.tierney@bristol.ac.uk  

News

ABC Learning Design: presentations and Q&A at UCL

Emilie Poletto-Lawson is an Educational Developer (based in Academic Staff Development) and a BILT fellow working on the BILT theme of inclusive assessment. 

This blog of a follow up from the blog post “ABC Learning design: workshop at UCL” which presented how the ABC Learning Design approach works. In this post, we will explore how colleagues at other institutions are using the kit.

First of all, many thanks to Clive Young and Nataša Perović for giving away a complete kit to all participants. It was extremely useful when reflecting back on the day. It is worth noting that all the ABC resources are available on line under a Creative Commons licence Attribution-NonCommercial-ShareAlike.

Here is what the kit looks like:

This photo shows the content of the kit: “Tweet and Share” document, set of cards, document with additional activities, blue tack, stickers, a blank action plan, a guide on how to run the workshop and a recap document.

I personally find the material very inviting and a great testimony to the hard work of all involved. After our hands-on session, Clive and Nataša opened the second half of the morning with a history of the project and update on what it is now and where it is being used. This was then followed by presentations by colleagues from other institutions who shared their take on the method.

Gill Ritchie and Ben Audley from Queen Mary, University of London

First of all, Gill Ritchie from Queen Mary, University of London (QMUL), presented how the ABC for Learning Design has become part of their PGCert Academic Practice programme. In the module “Teaching with learning technologies ”, participants are introduced to the technology wheel and a set of amended cards that contain the technology available at QMUL. They are then expected to design an online activity by December, that they then try out between January and April before writing up their reflection on how it went for their PGCAP.

The image is the technology wheel created by the ABC team based on the 6 learning types. Available here.

The updated version of the wheel by QMUL aims at highlighting what is available and supported by experts within the institution while being less daunting than the pedagogy Wheel Model  developed by Allan Carrington based on Bloom’s taxonomy that can be seen as offering an overwhelming amount of options. The University of Reading also created its own version (link  here ).

The wheel and activity types cards from the ABC kit are used with participants to discuss possibilities within their teaching leading to what sounded like fruitful conversations. If you are interested in finding out which technological tools the University of Bristol supports, you can contact the Digital Education team .

Gill’s presentation was then followed by her colleague’s, Ben Audsley, dental electronic resources manager in the School of Dentistry at QMUL. Ben supported lecturers with the transition of a module on dental public health to be fully online for distance learning. His approach was to look at the topics for each week and to then think about the technology that could be introduced to support learning. He used the kit focusing on the online suggestion of activities. It was interesting to note that his biggest challenge was to keep staff on track.

Luke Cox from the London School of Economics

Luke Cox, from the London School of economics, introduced a very interesting element in the process: using a critical friend. His presentation was on designing distance learning process and the way he approached it was to request having the course designer and a critical friend together to work and reflect on the design. He identified, actually, getting that critical friend in the same room at the same time as the designer as the biggest challenge.

Arthur Wadsworth, Moira E Sarsfield, Shireen Lock and Jessica Cooper from Imperial College London

The presentation by colleagues from Imperial College London was a great follow up to Luke’s as further to the critical friend, they suggested involving graduate teaching assistant (GTA). I believe this would be a fantastic opportunity to give GTAs a voice and to make them feel more strongly part of the community so long as their time is compensated and at an appropriate point of their studies. Colleagues at Imperial identified that lecturers and teaching fellows are not ready for the 25% module transformation in engineering objective they have. They also added a “fixed approach to teaching and learning” as a key issue. Their solution is to show a sign of remission from leadership around the area of workload and availability.   

Peter Roberts from Goldsmiths, University of London

Peter Roberts from Goldsmiths, University of London, adapted the cards so that instead of names of activities they list verbs. He then adapted the concept to an online activity on Trello , creating a deck with the learning activities (acquisition, collaboration…) to then drag and drop to create their design online.

This image shows an example of Trello to plan your weeks of teaching. Thank you to Peter Roberts for sharing this screen capture.

He also recommended the use of “Learning Designer ” developed by Laurillard at UCL, originally for school teachers.

Another online approach was mentioned in the questions following the presentation. The University of Lincoln has developed “Digital Learning Recipes” to support staff with the technological side of the design. The website gives examples of activity for each learning type and it is then followed by extra resources on the tools available and guides to use them.

And those were just lessons learnt from the morning!

Many thanks to Clive Young and Nataša Perović for sharing the ABC learning design and providing a kit to take home as well as inviting colleagues from other institutions to share their take on the method. It was a very insightful day and I look forward to finding out what’s next.

News

ABC Learning Design: Workshop at UCL

Emilie Poletto-Lawson is an educational developer (based in Academic Staff Development) and a BILT Fellow working on the BILT theme of inclusive assessment. 

If you are at the stage of reviewing how the year went or planning next years teaching, the ABC Learning Design  approach might be for you.

This blog post will share how the ABC Learning Design method works. A second blog post will share how colleagues at other institutions are already using the approach.

An only slightly delayed train journey got me right on time to start a workshop on the ABC Learning Design kit designed by Clive Young and Nataša Perović at University College London. No time to sit back, our two hosts, full of energy, guided us through a 90-minute session in just an hour. Clive and Nataša ran through the different steps of the process at the speed of light to ensure we would have time to try it out. Each table chose a programme lead wanting to design or review their course and off we all went!

First step

The first step of the method is to complete the “Tweet and Shape” document.

This is a photo of the ‘Tweet and Shape’ document described below.

You start with completing information regarding your programme. Your first challenge is to fit the description of your module/unit in the size of a tweet (140 characters). Your students should be able to understand what your module is about by just reading this and ideally wanting to sign up for the course if it is optional. This was the hardest part for our group!

You then reflect on where you are/want to be when it comes to the different learning activities. To help you, you can look at the cards, on the front, there is an explanation of what the type means and, on the back, examples of activities. It is worth noting that the activities are listed according to their digital or non-digital nature supporting your reflexion about developing a blended approach.

This is a photo of an example of the cards found in the kit as described below.

Finally, you need to reflect on how blended your course is/will be. How much is taking place online and how much face-to-face. You then put this aside and look at your course week by week and populate it with the different learning types activity cards. For example: week 1 could be Acquisition followed by Discussion; week 2 could be Investigation, Collaboration, Production and so on and so forth.

This is a photo showing the different cards lined up to represent the content of week 1 and week 2 as described above.

Once you are happy with the shape of your weeks, you can turn the cards over and look more precisely at the types of activities you would do. You then tick the relevant box(es) and you can also add your own.

This is a photo showing the different cards lined up on the other side, listing types of learning activities.

Nothing is set in stone for your redesign and you can make as many changes as you see fit.

Last step

Your last step for the design is to think of assessment. Do you have any formative assessments? If so, you can stick a silver star next to the ticked box. You will do the same with summative assessment, but the star will be gold. At this stage it is worth taking a step back to reflect on the student’s experience. What is the timing like? Will they have other assessments at the same time as yours? Will they have enough time to use feedback to improve if you have formative assessments built in your course? Are the activities and assessments aligning with your learning outcomes?

Once you have done all this, you go back to your “Tweet and Shape” that you completed in red originally and go through all the steps again with a blue pen to identify which changes, if any, you have made.

Optional step

Clive and Nataša added another stage which we did not have time to do in the workshop but that I find extremely valuable. During that last stage you could use more stickers to identify when the different learning outcomes are being achieved throughout the weeks. They also suggested identifying how your module/unit fits in with the university education strategy.

Conclusion

To me, this approach is extremely valuable as it gives you a very practical tool to design/review your unit/module/programme making sure you include activities that will be varied and encompass the different learning types that are key to students’ success. It is also a good way to reflect on the place of assessment on your course and more generally on your programme.

I can see real value in using the ABC learning design method with your colleagues during an away day to gain an overview of what your students experience is throughout the different modules they attend. A nice way of getting that overview, as suggested by our facilitators, is for colleagues to “promenade” in the room looking at all the designs. I also believe it would go extremely well with the TESTA project.

Many thanks to Clive Young and Nataša Perović for sharing the ABC learning design and providing a kit to take home. It was a very insightful workshop and I look forward to trying the kit out.

News, Uncategorized

Teaching Space as a Teaching Lab

The following post was written by James Norman, a BILT Fellow and programme director for Civil Engineering.

Today I continue my physical journey into the research of space as I embark on my third road trip of the year. I am back in Winchester, where I spent so many hours, ten years ago, when working on the Oxford Brookes project I discussed in my previous blog.

The reason for my visit is to revisit the architecture practice I was collaborating with to ask them their views on pedagogy informed design in higher education. Before I go any further I need to come clean, I am a huge fan of what they do. I really enjoyed working with them on Oxford Brookes and I have a great respect for their work more generally. And I am not the only one, they have been short listed three times in the last four years as Education Architect of the Year.

Photo provided by Design Engine Architects

I was expecting our conversation to be simple, straight forward and pedagogy-focussed. Instead it was wide-ranging, chaotic, with ideas flying everywhere. I tried to keep up typing away. But my notes are so wide-ranging it’s hard to know what exactly to say. So, I will do my best to summarise two different overlapping conversations.

The first is around pedagogy informed design, at some point about one and a half hours into our conversation I asked, “When you design a building do you bring a pedagogy or do you respond to the clients pedagogy?” to which Richard Jobson, one of the directors, replied, “it’s a bit of both and we look for common meeting ground. Our job is to challenge people. You can learn and talk to people and move your own thoughts on”.

This led to a much richer discussion about not just pedagogy but all the different competing stakeholders on a university project and how each one comes with an agenda, each one has set requirements and also a vision for the future. And each one is constrained by time, money, but also the needs of other stakeholders. And that the challenge to these ideas by the architect was robust, sometimes fierce and charged with emotion. We discussed how, in our collective experience, pedagogy can be discussed and agreed before a project starts (which the literature suggests is ideal), as a project starts, or some point further down the process, even sometimes after the physical building has started to be constructed.

This led to the discussion that unlike for other stakeholders like library services there is often not a dedicated group of people who are already engaged in conversations around pedagogy and space waiting for the next large building project, that these groups need to be assembled ad hoc (or even post hoc) to try and engage with the design process. As a result, it is hard to have pedagogy before a project and too often the pedagogy comes at some later point in the projects development.

Which of course leads to a bigger discussion, and one we will hopefully be able to respond to in time. Why don’t we have a group who are interested in pedagogy and space who are constantly active? Not waiting for the next project but creating their own. Who are trialling and developing teaching methods in different spaces not as a one-off event but as an ongoing discourse in pedagogy. Maybe the BILT fellowships in space are the start of this. But it strikes me this needs to be a long-term question. Buildings takes years (Oxford Brookes took 7) from idea to completion and we need conversations which understand this and develop with both the buildings and pedagogy.

John Ridgett, the project architect on Oxford Brookes, thought aloud “why not have a teaching lab? A space dedicated to trialling new teaching, both physical and digital. It could be a large warehouse with internal partitions which is designed to be constantly reconfigured”. This strikes me as a fantastic idea which I would like to explore further.

I headed out of Design Engine to walk along the road to their neighbour Winchester University. Here I can see Design Engines work in action. I am currently sitting and typing in one of their spaces. The campus is compact and vibrant with a multitude of lovely design touches. As I am shown round campus by Mat Jane of estates I am introduced to a number of people including Dave Mason who is literally in the middle of looking at furniture layouts. He describes how they, at a smaller scale, do what Design Engine were just suggesting. They trial room layouts, they play and see what works. They notice which rooms are popular and which are not, and they carry out surveys with both staff and students on which spaces they enjoy learning in. The teaching spaces became teaching laboratories.

Take the example below. One of the many observations of a teaching space is that the front rows are often empty. So they have provided different furniture at the front. Comfy seats and sofas, and suddenly the front third of the room is more heavily utilised. Of course, if this hadn’t had the desired outcome a different arrangement can be tried, and another, and another.

And so, as I reflect on my day, I am left asking myself “why haven’t I thought to do this before?”. It seems so simple, with hundreds of rooms, there is no reason why we also shouldn’t experiment, prototype and explore a wide variety of teaching spaces with a view to exploring what works and what doesn’t. Rather than wait and then refurbish large swathes of rooms with untested approaches we should play, learn, reflect and improve.

My sincere thanks go to Richard Jobson and John Ridgett of Design Engine (designengine.co.uk) for giving up two hours of their time to have such a wide-ranging conversation about the design of space and to Mat Jane who showed me around Winchester University with such enthusiasm and pride and also for all his insights on sustainability around the campus (including my free cup made from recycled chewing gum).

500 Words, News

Is There Any Link Between Design Thinking and Essays?

The following post was written by James Norman, a BILT Fellow and Senior Teaching Fellow in Civil Engineering. 

It’s strange how a number of unconnected events can form an idea in your mind.

This weekend I stayed with my sister and we watched the film ‘Blood Diamond’, a harrowing film made in the early noughties about the illegal diamond trade. We started discussing the role of the press and, at a more meta level, the film industry, who were indirectly profiting from the same trade through film profits. It reminded me of a magazine I read a few years back called ‘Colors Magazine’. Each issue focused on a specific concept and the one that came to mind was Issue #86, which is all about making the news. In it there is a page on war photography, which included a harrowing photo seen in many newspapers. The magazine presented it by covering most of the photo and leaving just the part of the image widely presented in the press visible. Turn the page and you see the whole photo; it is a different story. Gathered around the incident are dozens of journalists and photographers capturing the moment. It is a shocking moment.

Fast forward a couple of days and I had the pleasure of meeting Ann Padley. Ann is a teaching fellow who works on the new innovation programmes and specialises in design thinking. As a designer of buildings and a teacher of people who design buildings, I would like to think I know a thing or two about design thinking. But it turns out there is a lot more for me to learn. Over a rushed lunch we discussed problem solving, problem definition and redefinition. We discussed narrative as well as more empirical ways to come to design decisions. We talked about the importance of active listening in problem definition. And we talked about something I have been struggling with, how do we differentiate between outcome and process? Is it possible for a student to successfully go through a design process but come up with a less successful solution and probably more commonly a successful solution without going through a successful design process (or at least unable to articulate the design process- something I have struggled with as an engineer across my professional career because it is actually really hard to do and requires a lot of practice)? Ann described the methods they use to set and assess design problems which don’t just focus on outcome but focus on the successful implementation of design processes.

Later that same day I spent a very enjoyable hour talking to Zoe Backhouse, one of the BILT student fellows. Our conversation was wide-ranging but covered different forms of assessment. We discussed the essays that she had written, and it started to occur to me that what is presented in an essay, much like the photos in the newspaper I mentioned at the beginning, is the story that we choose to tell. But what happened to all the other stories? How do we know that we have presented the right one?

These thoughts linked me back to my conversation with Ann. I realised that design thinking is not just important for designers but for anyone who is given a problem (or title) and then has to deconstruct the problem and find what the real problem is before deciding on the solution (when there are many possible solutions). Maybe not just engineers, but all of us would benefit both from learning to articulate not just the solution, but how we got there. The narrative around the solution. The options we considered and discarded. And maybe not only would we benefit from articulating this but also from discussing it with our friends and tutors. To receive feedback (or more precisely feedforward) not just on the output but on the processes through which we have gone to arrive at the solution (or essay).

Ann Padley is a Teaching Fellow in Design Thinking 3 days a week and is an independent consultant on design thinking for her other work days.

Zoe Backhouse is a final year student on the four year masters degree in Liberal Arts, she is a BILT student fellow and a fledgling zine maker. Zoe would welcome any musings, poems, doodles or cartoons from students & staff about your experiences of assessment at Bristol Uni. If you’re a student, you’ll get a £25 Amazon voucher for whatever you contribute! Email zoe.backhouse@bristol.ac.uk for more details.

Colors Magazine Issue #86 ‘Making the News’ published April 2013.      

News

Bristol Conversations in Education: Does School Design Matter? 16/1/2018

Professor Daniels presented an account of AHRC sponsored research that sought to address the need for learning environments to better respond to changing needs of curriculum and pedagogy.

I am a BILT Student Fellow working on the theme ReThinking Spaces and on the 16th January 2019 I attended a seminar by Professor Harry Daniels from the University of Oxford entitled ‘Does School Design Matter?’. I was particularly interested in the relationship between design and pedagogy and lessons that can be learned for the redesigning of spaces at the University.

Professor Daniels began by describing the impact that a building or physical space has on the way that we behave within it. He gave the example of starting a new job and using the built environment to help to determine the expectations for behaviour, communication and interaction. This is partly affected by physical features such as the furniture layout, lighting, and decoration, as well as the way that people interact with the space and with each other.

Professor Daniels’ journey into this area began around 2003 when, noticing the impact of wall displays in schools, he asked himself; ‘if a wall display can be powerful, what about the building itself?’ This has led him to research the perception and actions of students and teachers at four secondary schools in Kent, which were part of the Building Schools for Future (BSF) programme introduced in 2004 and were newly built or refurbished between 2010 and 2012.

Professor Daniels emphasised that the way that a space is used is not necessarily how it was designed to be used. The four schools in question were designed to promote the personalisation of learning, with teachers to be viewed as coaches or mentors. The County Council deemed that:

  • learning spaces should be versatile and flexible to cover all curriculum areas
  • there should be breakout areas and informal learning zones
  • students should have greater independence and agency over their learning
  • teachers should share many spaces with students and other staff
  • staff should teach in teams
  • there should be a high degree of visibility with the use of glass and an open plan design
  • community engagement should be promoted

The School Connectedness Questionnaire was distributed to children at the end of primary school, at the beginning of Year 7 and Year 8, and every time there was a change of headteacher. The research found that when teaching practice aligned with the design, the connectedness score was significantly higher than when the practice did not align with the intended design.

Whilst some schools found positive outcomes, with improved behaviour of students and better formative assessment practices by teachers, others struggled to use the space effectively. Two of the schools closed off open areas with glass panels or furniture, effectively attempting to reverse the radical changes that had been made. Professor Daniels explained that these differences in success can partly be accounted for by different approaches to school leadership and management. In places where high visibility was seen by management as allowing passive control and surveillance, teachers and students felt watched over, whereas in more relaxed settings where visibility was viewed as a way to promote a sense of community and belonging, staff and students enjoyed being able to collaborate and socialise more easily with others.

Where the redesign was successful, there was a strong vision from the start and an excellent programme of staff training in how to successfully work in the new spaces. Staff continue to collaborate to solve problems related to design issues and students are included in this dialogue. Staff report that students feel wanted, have improved confidence and aspirations. The open-plan environment mirrors the professional working environment and develops the skills that the commercial world is demanding. Professor Daniels highlights the importance of learning from other similar learning environments when redesigning educational spaces.

So, what does this teach us about the relationship between design and pedagogy, and what can we learn from it? This seminar highlighted to me that redesigning space does not necessarily transform pedagogy. This requires an ethos of trust where staff feel confident enough to be observed and to collaborate with others, and where staff are trained in teaching practice which aligns with the space design. In Higher Education, we need to learn from similar institutions which have redesigned their spaces to align with the shift towards more active, collaborative and inquiry-based learning. Yesterday, BILT visited the new Waterside Campus at the University of Northampton who are doing just that, and BILT are gaining staff and student perspectives on teaching spaces, as well as providing resources to staff, in the hope that our space will be fit for an imagined future.

References

Harry Daniels, Hau Ming Tse, Andrew Stables & Sarah Cox (2018) Design as a social practice: the experience of new-build schools, Cambridge Journal of Education, DOI: 10.1080/0305764X.2018.1503643

News

No lecture theatres? No problem!

The following post was written by Amy Palmer, BILT Digital Resources Officer.

In 2012, the University of Northampton decided to embark on a challenge that would set them apart from all other UK universities.

Six years later their new, £330 million ‘Waterside’ campus was launched with one key difference – there are no lecture theatres*. All courses have been redesigned and adopted active-blended learning as their pedagogical approach, which has transformed the way students learn. Further to this, all staff offices (including the VC’s!) have been removed in place of communal workspaces and hotdesking. The eradication of passive learning experiences and focus on active, activity-based sessions is a daring and challenging move that has taken a huge amount of courage, time and commitment. The creation of a learning design team, as well as the support of both academic staff development and learning technologists has been central to the success of this project, as well as the unwavering support of senior management.

When asking the Dean of Learning and Teaching, Professor Alejandro Armellini, what really works about the Waterside project, his answer was clear – everything. Apart from the addition of a few more plug sockets in their ‘Learning Hub’ (a grand, multi-purpose building housing libraries, teaching and social spaces, though with no signs or labels defining these areas), there is nothing they would do differently. It’s too early to see how the new campus and educational approach will affect learning gain and student recruitment and retention, but the feeling so far is that it is working well.

This new and daring approach to higher education took a number of years to achieve and was only possible with the support of the Vice Chancellor who, when announcing the plans, told staff ‘you either get out of the way, or get on the bus’. Some staff did get out of the way, and many that stayed were hesitant to ‘get on the bus’, often feeling that the change in approach was a personal attack on their style of teaching. When the learning design team spoke with individuals and asked what they really valued, it was never ‘standing up in front of people speaking’ but rather ‘when I see my students have learnt something’ and ‘when students are engaged’. Extensive research was done into how to engage students with active-blended learning – you can read their findings here.

Teaching hours for staff have increased across the piece with students now split into groups of (max) 40 students, who they will stay with throughout their degree, with the intention this creates a sense of community and belonging among fellow students. This will no doubt help with issues around wellbeing and first-year student retention, though there may be some protests that it is very much like school and not the ‘traditional’ university experience where you anonymously sit in a huge lecture theatre and take down notes.

The Waterside project will be interesting to follow over the next couple of years, especially when it comes to crunching the data. They openly admit that there are some staff who are still lecturing at their students but believe that will change; the focus on teaching is gaining momentum yet there are still some who are yet to be caught up in it. We have invited colleagues from Northampton to visit us when the new Temple Quarter campus is built – we hope that some lessons can be learnt from our trip there!

*Okay – there is one lecture theatre, but it only seats 80 and is used mainly for external speakers.

News

Learning Games #2

The second ‘Learning Games’ event took place on 17th January. To give everyone a chance to eat their lunch, the session started with a discussion around the tables about where we would like to use games in our teaching, and barriers we have (except for time – time is a problem for everyone!). Each group fed back and the key barriers were:

  • Resistance to change – some colleagues may not believe that learning with games can be as effective as more ‘traditional’ forms of learning.
  • Not knowing where to start – lack of experience in making/ designing games, what to make the games for, what tools to use, etc.
  • Having the resource/ capacity – this is quite similar to lack of time but is a key point – many staff would like to take time to create a game for their learners but there is not capacity in the team.

Dr Kieren Pitts, a senior developer in Research IT, presented a game he has been working on as part of a research grant with colleagues from physiological science. The game, EyeTrain, was developed to improve oculomotor control in children and consists of three ‘scenes’ (one urban, one woodland and a high contrast scene) in which the player has to tap when they see an animal move. The game encourages the player to move their eyes in repeated, specific movements with both smooth and saccade motion. The game begins with an animal that has quite obvious movement (e.g. a hare that moves its ears) and as you improve more animals are unlocked, each with more subtle movements, and the backgrounds (scenes) becoming more complex and detailed as the player improves. Illustrations were done by Bristol-based illustrator, Alex Lucas, whose work can be seen in the School of Education and on walls across the city.

eyetrain.jpg
Example image from ‘EyeTrain’.

Settings in the game are highly configurable and it has been programmed to collect vast amounts of data to ensure its effectiveness. Early testing has shown it to be effective in improving oculomotor control in children. More information about the game can be found here.

We then heard from two members of staff who have recently been awarded Discretionary Seedcorn Funds from BILT. Dr Frankie MacMillan from the School of Pharmacology, Physiology and Neuroscience explained the card game they are making for students studying Histology. Students must place down a card, with the next player putting down a card linked the image on the card before and explaining why. If a student can not go, they can use red blood cell ‘counters’ to buy an answer off another player. They hope that this game will make quite a ‘dry’ topic more interesting and memorable as the students have to create links between the types of cells and tissue themselves.

Next, we heard from Dr Isabel Murillo Cabeza from the School of Cellular and Molecular Medicine and her game, Microbial Pursuit. The plan for this game is for it to be played across two sessions and is to be used as both a learning and revision tool. The first session students are split into small groups and each write multiple choice questions with three options. The students can use their lecture notes, eBioLab materials, tutorials, essays and other academic material to help them write the questions. In the second session, students are reshuffled into different groups and use the questions to play a board game, similar to the layout of Trivial Pursuit. Students can play as individuals, in pairs or in threes.

The session concluded with a short game that was based around weather predictions (but I’m not sure where the weather link came in!). We all started with a coloured counter balanced on the back of our hands and the aim of the game was to be the last person with their counter on their hand, while at the same time attempting to knock off other peoples.

If you’d like to come along to play a silly game, hear about what others are doing with games and their teaching and discuss your ideas for gamifying learning, get in touch with Chrysanthi Tseloudi or Suzi Wells to find out when the next session is on.