Tag: assessment

Assessment in higher education conference

Assessment in Higher Education Conference 2019

This event will be the seventh international Assessment in Higher Education conference. This research and academic development conference is a forum for critical debate of research and innovation focused on assessment and feedback practice and policy. The themes for our 2019 conference will invite a wide range of papers, practice exchanges and posters. Themed poster presentations, accompanied by a short pitch from the authors, have been a particular strength of the conference and have encouraged networking by delegates.

Keynote Speakers

Phil Dawson: Associate Professor at Deakin University

Bruce Macfarlane: Professor of Higher Education at University of Bristol

Thumbnail image of Humphrey Bourne

“Write my essay for me!”: Rising to the Challenge of Contract Cheating


Abstract
Reported incidents of ‘contract cheating’ or ghost writing are increasing along with the number of ‘essay mills’ – the providers of bought essays, projects and even dissertations – whose marketing is becoming ever more sophisticated.  In this seminar, we will explore the extent, nature and responses to this threat to academic integrity before suggesting ways in which we can alert and inform students and staff, counter the threat, and develop ways of minimising the occurrence of contract cheating in assessment.

Bio
Humphrey Bourne is Education Director (PGT) in the Faculty of Social Sciences and Law.  He teaches strategy in the School of Economics, Finance and Management and his research focuses on values in and around organizations, extending into research in organizational identity.  He chaired the University working group on academic integrity and misconduct, reporting in October 2017, and has since been closely involved in the implementation of its recommendations.

News

Should all assessments be inclusive?

BILT

The following post was written by Emilie Poletto-Lawson, an Educational Developer and BILT Fellow. 

I am a BILT fellow based in Academic Staff Development where I work as an Educational Developer. I have been working on the BILT theme of assessment – focusing on inclusive assessment since February 2018. I am undertaking a literature review with a view to making recommendations around inclusive assessment principles that we can embed into our units and programs at the University of Bristol to work alongside our institutional principles on Assessment and Feedback.

From my reading to date the  main take away is that inclusivity is predominantly discussed as a means for supporting students with disabilities. It is very much viewed as a deficit approach to considering assessment, however, I strongly believe it is far more than that, we want to be inclusive of all learners and for inclusive assessment to actually be more inclusive.

As part of my BILT fellow role I recently attended an event at the University of Leicester called “Making IT* Happen: from strategy to action (*Inclusive Teaching)’, led by Pete Quinn and Mike Wray (blog available here). The focus was very much on supporting disabled students in our institutions and ensuring universities are legally compliant with the Equality Act. In preparation for the event, the experts highlighted good practice in the work we do at Bristol, for example we received positive feedback on our institution website regarding inclusivity (http://www.bristol.ac.uk/disability-services/study-support/reasonable-adjustments/) and in particular videos created by Louise Howson from Academic Staff Development (http://www.bristol.ac.uk/staffdevelopment/academic/resources/learning-and-teaching-resources/learning-and-teaching-videos/ ).

Regarding the literature review I am working on, when researching the key words “inclusive” “assessment” in “higher education”, I obtained 9596 results in ERIC and yet, going through the abstracts not that many articles encompass all three parameters. It appears there might be a gap in the literature here despite inclusivity being key to university strategies in the UK and beyond for a number of years now. So far, the key emerging themes from my searching can be seen below.

Inclusive Assessment in Higher Education map created by Emilie Poletto-Lawson
Created with Mindmeister 21.09.2018

In the US literature the Inclusive aspects of articles relates to the idea of an inclusive campus and looks at inclusivity from the selection process (access) to the students completing their degree (success). In the UK, the literature shows there is an acknowledged need for policies, strategies and processes as well as professional development to bring about inclusive practices.

Initial readings suggest there is a rhetoric of inclusivity as a given good, but it is difficult to identify concrete examples, especially when it comes to assessment. The literature review is the first step to articulate a clear definition before focusing on what inclusive assessment means for us at the University of Bristol.

If you are interested in this topic why not read “Against being Inclusive” by Jeffrey Carlson, interim provost and Vice President for Academic Affairs at Dominican University? I appreciate it might be an odd recommendation since this post advocates that all assessments should be inclusive, but I think this article, published in 2016, does offer food for thought and reinforces the need to clearly define what we mean by “inclusivity” before we move to making recommendations at Bristol.

Meet the BILT Student Fellows

Meet the Student Fellows… Zoe Backhouse

BILT

We asked our Student Fellows to write us a short blog about their background and what they are doing as part of their BILT role. The following blog is from Zoe Backhouse, who has been a BILT Student Fellow since December 2018.

My name is Zoe Backhouse. I’m a fourth year Liberal Arts student newly appointed as a BILT Student Fellow. I’ll be working on a project called Improving Students’ Understanding of Assessment.

This marks my fifth year trying to shake up education practice here, which either makes me an education practice nerd or suggests there’s a lot to change at Bristol. Just kidding! I love Bristol Uni 😊.

When I started in 2014 I needed a part-time job and overheard that the Students’ Union was a great place to look (its Living Wage Employer status meant high £££ for first years). By my second week, I’d landed a role as an Administration Assistant with the Educational Representation team. Without realising, I’d found a passion. I worked with student reps and academics from across the University, supporting them to challenge the curriculum and change degrees from the grassroots. Inspired, I went on to become a course rep, a faculty rep and then took a year out after second year to work as the Undergraduate Education Officer.

I learned a huge amount over those three years. I encountered diverse opinions and practices and, although overwhelming, this taught me that there was no one way to experience this University. Despite the difference, there was something that became consistent to me – whether it was as a high or a low, most students’ time was defined in some way by their experience of assessment.

Just as I began to develop a detailed picture of student needs and was putting together a manifesto for the University to help address this time-old problem, my term ended at the Students’ Union and I had to go to McGill University in Montreal, Canada for my year abroad. What a shame!

I ended up having probably the best year of my life there. Although this was of course because I was in one of North America’s most exciting cities, I have to be honest in saying that the year was special in what it gave me educationally.

Suddenly I was at a University where I wasn’t just studying disciplines I’d encountered in school. I was being educated in Indigenous Studies, Jewish Studies, learning about contemporary Canadian history through Fine Art and historic Islamic law through English Literature. I was assessed through volunteering, creative writing, building real-life grant bids for a project I’d set up in the city, and, yes, through formal research papers. I felt I’d never learned so much in my life.

Coming back to Bristol, a university with many of the same qualities and challenges as McGill, I saw only opportunity in terms of where we can go. I love what BILT does and think it’s already had a huge effect in supporting academics to take risks with their teaching practice and experiment with whole new concepts in Higher Education.

Now I’d like to see how this applies to students. What is the potential of us co-creating assessment? How can we cross disciplinary boundaries and be assessed on, or even beyond, our programme? And, perhaps most importantly to me personally, how can we understand our mental health in the context of our learning and assessment (not just in the context of support services)?

Prepare for focus groups, debates and more blogs over the word limit. I can’t wait to start!

 

News

Tensions with Programme-Level Assessment

BILT

The following post was written by Helen Heath, a reader in Physics and BILT Fellow. 

I am a reader in the School of Physics at Bristol and currently 1st year coordinator and head of the newly formed Physics Education Group (PEG), which is a group for the pathway 3 staff within the School of Physics. I’ve been a BILT fellow since September 2017 with a focus on programme level assessment and this has been renewed for the next academic year.

While looking into programme level assessment I have felt that there are significant tensions. The move to more formative and less summative assessment should help student development but fewer summative assessments inevitably means that these assessments are higher stakes. Devising good innovative assessment methods that are targeted at testing the learning outcomes is desirable but too many different types of assessment can be confusing to students.

As part of the University move towards programme level assessment several Schools took part in assessment labs and have been working on moving their assessment to a more programme level approach. I am in the process of interviewing those involved with these pilot projects. I would like to understand the challenges they have faced in the programme design and the institutional changes, e.g. to regulations, that need to be made to enable the implementation of the revised assessment schemes they have proposed. I am also interested in how the perceptions of what programme level assessment is vary across the University.

Although the content of courses varies, the principles of good assessment design should be applicable to all programmes however it seems clear to me, from the first interviews, that there are structural differences between programmes that can hinder or help the adoption this approach. Joint honours programmes, for example, can pose a challenge. The programme level approach can offer the chance of an assessment which brings together the strands in these types of programmes, but this then requires clarity in the management of a joint assessment. Schools that have very many different joint honours programmes need to avoid a confusing range of different assessments.

Schools piloting programme level assessment are also in very different points in their curriculum review cycle. Adopting a programme level approach where a review is underway anyway is much more natural than making large changes to assessment when results of a previous review are still working their way through programmes.

As a second strand of my work I have been thinking about what programme level assessment might look like in my own School. We have joint programmes with Schools in our own Faculty and outside as well as several “Physics with” programmes but we have a well-defined Core in all programmes which could be assessed at year level. The “straw person” proposal for a way forward is submitted to our next Teaching Committee. Hopefully it promotes a lively discussion.

andy penalun portrait photo

Teaching and Assessing for Enterprise


A seminar held in collaboration with The Centre for Innovation & Entrepreneurship.

Speaker: Professor Andy Penaluna, Professor of Creative Entrepreneurship at University of Wales Trinity Saint David
How do we best teach and assess skills like creativity, opportunity recognition, self-awareness and self-efficacy, and taking action to implement ideas? In this open talk for educators in all disciplines, Professor Andy Penaluna will set out the principles that underpin his work with the QAA and the European Joint Research Centre, to help educators in every discipline teach and assess students’ enterprising competencies.

Bio:
A former Chair of Enterprise Educators UK, Andy was described by UK Government as the World’s first Professor of Creative Entrepreneurship. He conceptualised and chaired the Quality Assurance Agency’s Graduate Enterprise and Entrepreneurship Group that developed national UK Higher Education guidance, and led its five year review. Now referenced across Europe and beyond, it has even been translated into Mandarin.
Andy is also an expert at the United Nations in Geneva – where he supervised research that led to ‘for innovation’ curriculum development for 37 developing countries. He writes for the European Commission and helped to develop their ‘EntreComp’ framework. He also led the development of entrepreneurial teaching and learning modalities for 8 countries in South East Europe and writes for the OECD on developing entrepreneurial schools and colleges as well as HE level creativity. Funded by the World Bank, he led a team in what is believed to be the world’s first compulsory school curriculum for innovation and entrepreneurship (in Macedonia – FYROM).
In 2014 Andy’s contributions were recognised by the Enterprise Sector Skills body ‘SFEDI’ in the House of Lords, and in 2015 Andy received the Queens Award for Enterprise Promotion at Buckingham Palace. In 2016 he was named as one of the UK’s top Maserati 100 entrepreneurs.

Andy always acknowledges that his approach to teaching enterprise is heavily reliant on his extensive 30-year network of alumni, and that they motivated him to become a more entrepreneurial educator.

Two students recording using a video camera and boom microphone
Education Enhancement Funds

Developing a guide to support the use of video in undergraduate assessment

BILT

A Teaching Innovation Grant was awarded to Dr Emily Bell, Dr Rose Murray and Dr Andy Wakefield for the academic year 2017/18 – you can find a summary of the project they undertook with their grant below. If you would like to read the full report, please contact bilt-info@bristol.ac.uk. 

Project summary

Project members (PMs) (Bell, Murray and Wakefield) are developing an open-access framework to enable the easy implementation of innovative and creative assessment. This includes an interactive guide for teachers and learners which supports the creation of video as a means of assessment. This guide provides step-by-step instructions on how to use audio-visual production/editing software as well as helpful advice on suitable equipment. PMs are also creating an appropriate, transdisciplinary framework for evaluation of the students’ video projects. By focusing the assessment of the video at group level and embedding this within a portfolio of supporting tasks the PMs hope that their guidance will be applicable to a greater number of disciplines within higher education.

This novel assessment method was embedded into a field course module (June 2018), with 20 students working in groups of three or four. Students were instilled with a wide range of transferrable skills as well as having the opportunity to stretch their creative legs. In tandem, and via the same processes, students’ confidence was evaluated in four key skills areas (academic self-efficacy, groupwork, communication and digital capabilities) via data collection from pre-post questionnaires and student reflective critiques. Preliminary data analyses suggest that student confidence increased over
the course of the week in these areas.

Conclusions

Despite only preliminary analysis of the data gathered thus far, it is apparent that participants who took the filmmaking course perceived an increased level of confidence in several different skill areas; academic skills, social efficacy, communication sk ills and digital capabilities. The qualitative responses corroborate findings from the quantitative responses, and the authors look forward to evaluating the reflective critique data later in the year. On reflection, one of the main themes identified from the data was the increased confidence particularly in social efficacy. This was highlighted as a particular cause for anxiety or apprehension prior to the course by the participants, yet by the end, it was clearly an area in which participants had grown in confidence. This will be explored further as the project progresses. As this was a single field course from a collection of ~15 courses which run each year, it would be interesting to evaluate other courses to explore whether this increased confidence of participants is replicated, or whether it is a unique quality of the film making course. The authors hope to repeat this study in the next academic year with the following cohort of students to test the robustness of these findings.

Digital classroom
Education Enhancement Funds

Building Confident Engaged Researchers Through Active Partnership and Problem Based Learning

BILT

A Teaching Innovation Grant was awarded to Dr Chris Kent and Dr Jess Fielding for the academic year 2017/18 – you can find a summary of the project they undertook with their grant below. If you would like to read the full report, please contact bilt-info@bristol.ac.uk. 

Project Summary

This project was designed to assist the redevelopment of our research methods provision at Year 1.

The newly developed course will focus on active learning in small groups and continuous, low-risk, assessment. Specifically, it will address four main aims in our Year 1 teaching:

1. Enable students to programme and conduct their own experiments within TB1

2. Provide weekly continuous formative feedback on knowledge and understanding

3. Enable effective small group peer support via ‘homework clubs’

4. Embed a culture of student participation in lectures via interactive smart-phone response systems (SRS).

The BILT grant facilitated the redesign and comprised two main work packages

(WP). WP1: development and evaluation of a self-contained series of lab sessions designed to deliver design, programming and analysis skills to Y1 psychology students (meeting aims 1 and 3). WP2: developed a set of home and class activities for summative and formative assessment (meeting aims 2 and 4).

Conclusions

We feel the grant was extremely helpful to us. We have meet our aims and are in a much better place to implement our redesigned research method training courses.

Students, who undertook the pilot, showed a keen interest and disposition to research methods once they were exposed to a hands-on approach to building experiments and understanding why programming and statistics were important (via problem-based learning and active participation in work shops). The students left with a positive disposition towards further exploring research methods and we highly positivetowards the way in which the material was delivered. All students managed to develop their own independent research question, programme an experiment to test it, collect and analysis the data; this is quite impressive given the short six week time scale!

The research skills we developed by actively engaging students in research from the get-go, by showing them they could develop their own simple experiments and analysis with a few simple tools. The problem-based approach to introducing new software was successful and students felt confident in exploring the software.

Meet the BILT Fellows

Meet the BILT Fellows: Zoe Palmer

BILT

We asked our Fellows to write us a short blog about their background and what they are doing as part of their BILT Fellowship. The following blog is from Zoe Palmer, who has been a BILT Fellow since September 2018.

For the past six years (on and off!) I have been teaching in the School of Physiology, Pharmacology and Neuroscience in what is now the Faculty of Life Sciences.  Within our school we teach our own undergraduates and postgraduate students, but also students on professional programmes; vets, dentists and medics.  My involvement with the medical programme also extends to recently being appointed lead for teaching block one of year two of the new medical curriculum (MB21) and I have been developing material for an optional three week pharmacology skills development and training unit.  In addition, I am involved with outreach, widening participation and public engagement.  This summer I co-organised the first Biomedical Sciences International Summer School.  This new faculty-wide endeavour is aimed at external undergraduates who don’t have the opportunity to undertake many practical classes at their home universities and so visit us to take advantage of our laboratories and teaching skills.

I am particularly interested in assessment and during my BILT fellowship I intend to investigate methods of quality assurance in exam setting.  I recently submitted my CREATE Level 2 portfolio which included a project in which I retrospectively analysed and evaluated the reliability of standard setting exam papers.  Standard setting is a process whereby exam papers are scrutinised by a team of experts to (in theory) create a robust and fair pass mark, as opposed to employing an arbitrary pass mark of, for example, 50%.  The results of this investigation were thought-provoking.  I would like to use this preliminary work to explore whether there might be a more rigorous and accurate method of generating the pass mark for exams.  This, and finding out more about assessment processes across the university and beyond, will aid us in implementing best practice and making evidence-based decisions to ensure that our assessments are valid and fit for purpose.

Image of Chris Rust delivering workshop symposium
An interview with...

An interview with… Chris Rust

BILT

We spoke to Chris Rust, Professor Emeritus of Oxford Brookes University and author of ‘Assessment Literacy: The Foundation for Improving Student Learning’ and numerous other publications on assessment and pedagogy. Chris was BILT’s first visiting professor and has facilitated a number of workshops for BILT. He was the keynote speaker in BILT’s launch symposium in June 2017 on Assessment and Feedback.

What are the most common problems you tend to observe with current assessment practices?

I think the most common problem is a lack of alignment, or a fudging of alignment, between the learning outcomes and the task set. And then a further fudging when it comes to the assessment criteria (which may bear little or no connection to the outcomes), the fact that it as all then finally reduced to one virtually meaningless number (mark), and the subsequent opacity of the feedback given. There may be four or five excellent outcomes but then the task chosen to assess them may be an essay, or a report, or exam, or whatever (regardless of whether that will actually assess whether the outcome/s have been met or not) and the assessment criteria then tend to focus on the medium of the task rather than the individual outcomes – structure, fluency, grammar, spelling, referencing, etc. Now while those all may be important, they almost certainly do not explicitly feature in the learning outcomes. And then finally, the worse sin of all, the assessment decisions are aggregated.

What benefits do students experience through a programme level approach to assessment?

Well the programme specifications and subsequent programme level outcomes, should be the vital things the student needs to achieve to merit the qualification. So focussing on them should benefit both the teaching staff and the student. The problem with unitised or modular programmes is that outcomes can be atomised at the lower level to the point that they don’t add up to the espoused programme outcomes, or reach the greater depth and complexity of programme outcomes. A programme level approach should also benefit students by explicitly encouraging the integration of learning from the different units or modules.

How can Universities help students to understand these benefits?

By being explicit at all times – in programme and module documentation, when assessment tasks are set and discussed – and also be ensuring that assessment tasks are valid and, wherever possible, authentic.

What are the most valuable resources/articles you use?

I have summarised a lot of the useful research in a freely available paper: ‘What do we know about assessment?’ I would also recommend the Australian website Assessment Futures (found here).

What one piece of advice would you give to help improve students’ assessment literacy?

You must involve students in the activity of assessment – marking work and having to think like assessors – whether it is through marking exercises, giving self and/or peer feedback, or actually allocating actual marks.

You advocate ‘quick and dirty feedback’- what does this mean?

I only advocate this when detailed, individualised feedback may not be logistically possible, or perhaps necessary. In the case of, say, weekly lab reports it is much more useful to take them in and sample them and then send an all class e-mail with generic feedback than for students to receive detailed individualised feedback on a report they did three weeks ago, and since then they have done another two. I would also class on-line possibly multiple-choice quizzes in this category. They may not be able to assess at the higher end of Bloom’s taxonomy (discuss!) but they can give instant feedback to the student on how much they have understood this week’s topic, and depending on the software can also possibly give hints and tips when the answer is wrong.

What inspired you to first start looking at assessment practice and advocating change?

When I did my MEd at Bristol, I had a session from David Satterly and was introduced to his book Assessment in Schools which highlights many of the problems in assessment practice which sadly still exist today over 30 years later. And out of all of them, I am especially incensed by the misuse of numbers in assessment, and the fact that university assessment systems get away with doing things a first year statistics student would fail for.

Are there any models you would recommend following to redesign programme assessment? 

Yes. I particularly like the idea of requiring programmes to identify cornerstone and capstone modules, which are where the programme outcomes are explicitly assessed. I also think that Brunel’s system of allowing the separation of what they call study blocks from assessment blocks is especially ingenious and clearly allows for all sorts of creativity by the programme team.

Can you think of any case studies from other institutions that would inspire staff to change their programme assessment?

Further to what I said above, I think the Brunel model is certainly worth the effort needed to understand it because of the potential it opens up.

What is your view on 100 point marking scales and would you advocate use of any different forms of marking scales?

If I had my way I would ban the use of numbers in the assessment process completely – they are worse than unhelpful, and I have written on this at length! See for example: Rust, C. (2011) “The unscholarly use of numbers in our assessment practices; what will make us change?” International Journal for the Scholarship of Teaching and Learning, Vol. 5, No. 1, January 2011 (available here). I would advocate much simpler grading – pass/fail, or perhaps pass/merit/distinction, or at most a four-point scale (perhaps based on Biggs’ SOLO taxonomy) – specifically for each learning outcome.

What one film/book/resource would you like to share with the academic community?

In addition to those already mentioned, maybe the video A Private Universe. (available here). It is quite old now but still totally relevant regarding issues of teaching and the failure of many of our assessment practices.

If you could change one thing about HE in the UK what would it be?

Banning the use of numbers in assessment.

Who was your favourite teacher at school/university and why? 

That’s hard – I went to a boys’ grammar school – much easier to list the bad teachers, and why. Not sure about favourite but I can only remember two good teachers at school – Mr Allen for English and Mr Thomas for maths – and they were good in that they explained things in easily accessible ways, with humanity and humour, had passion for their subjects and appeared to care about us learning.