Date: August 2026
Authors: Laura Contu, Michael Smyth, Gaurav Saxena
Themes: Pedagogic research, GenAI use in HE
Faculty: Faculty of Health and Life Sciences
Subject area: Critical evaluation skill development. Ethical and informed. GenAI use skill development.
Enhancing research methods knowledge and skills
The Practice
AI tools are increasingly being used by students in higher education and our school, due to their many advantages such as generate ideas, summarising articles and handling routine tasks. Despite many benefits, there are concerns that over-reliance on GenAI may weaken students’ thinking and critical evaluation skills, compromise academic quality with misleading information, challenge academic integrity, and suppress academic ability and confidence development.
To explore how ethical and thoughtful GenAI use can be meaningfully included in teaching, we implemented a programme of AI activities and assessments into the sequential units Psychological Research Methods 1 (TB1) and Psychological Research Methods 2 (TB2) of the MSc Psychology Conversion programme 2025-26. The AI interventions were designed to help students develop their critical thinking skills through strategic GenAI use, particularly relevant to the topic of research methods which focuses on critical thinking and independent analysis.
To evaluate of the effectiveness of the interventions and student satisfaction, we conducted a study involving the thematic analysis of 3 focus group stages. Of the 42 students, 23 unique students participated in the three focus group stages (54.8% of the cohort), 7 students participated in two different FG stages, 3 students participated in all three.
The students were positioned as GenAI learners and evaluators of the AI interventions.
Stage 1
Focus Groups Stage 1: Pre-intervention
- Explore students’ current views on and experiences of GenAI and skill development to inform our understanding of AI’s current impact on students, and provide insights into how best develop ethical GenAI literacy and skills suitable to enhance learning at university and future employability skills.
- Two in-person FGs ~ 60 minutes each (4 + 8 participants = 28.6% of cohort).
Stage 2
Table 1
GenAI Workshop and Assessment Content (TB1)
| AI Activity | Week(s) |
| Live, interactive MCQs (slides + student notes). | WS 2, 3 |
| Roleplay. | WS 4 |
| Request explanations (slides + student notes + paper). | WS 5 |
| Paper summary (guided). | WS 7 |
| Generation of output + Critical evaluation with Checklist. | WS 8 |
| Discussion structure. | WS 9 |
| Introduction feedback. | WS 10 |
| Assessment (10%): GenAI output critical evaluation: Future research suggestion for their class project report. | Submission 10 |
Note. Workshop (WS) activities ~ 30 minutes each. Instructions included the task, prompts, boundaries, ethical considerations (when can use and not use).
Focus Groups Stage 2:
- Explore students’ reflections on their GenAI skill development, current views on GenAI and the GenAI activities and critical evaluation assessment during TB1. This helped us understand the effectiveness of the learning activities and content so we could refine them for TB2.
- Two in-person FGs ~ 60 minutes each (8 + 7 participants = 35.7% of cohort).
Stage 3
The AI activities were developed in response to student feedback in TB1 and the FG stage 2.
Table 2
GenAI Workshop and Assessment Content (TB2)
| AI Activity | Week |
| Language Analysis Activity – Evaluated the GenAI comparison of 2 news articles about the same event. | WS 14 |
| Quantitative vs Qualitative – Evaluated the GenAI comparison of 2 journal articles – 1 quantitative, 1 qualitative. | WS 15 |
| Assessment development Activity – Participated in the development of their critical evaluation assessment. | WS 17 |
| Thematic Analysis Activity – Evaluate the GenAI generated Thematic analysis of a 3-page interview extract about student-supervisory relationship. | WS 19 |
| Interpretative Phenomenological Analysis write-up Comparison – Evaluated the GenAI generated comparison of 2 IPA write-ups. | WS 20 |
| Critical evaluation assessment guidance. | WS 21 |
| Assessment (10%): GenAI output critical evaluation: A thematic analysis of a transcript (1 of 2 optional topics). | Submission 23 |
Note. Workshop (WS) activities ~ 30 minutes each. Instructions included the task, prompts,
boundaries, ethical considerations (when can use and not use).
Focus Group Stage 3:
- Explored students’ reflections on the qualitative focused GenAI activities, their GenAI skill development, current views on GenAI and the GenAI activities and critical evaluation assessment during TB2. Also views on the whole AI intervention process. This helped us understand the effectiveness of the learning activities and content so we can refine them for next year and propose suggestions to implement GenAI learning activities into other units and programmes.
- One online FG ~ 60 minutes (9 participants = 21.4% of cohort).
Findings
Across the three focus group stages, the findings can be group into themes related to the development of an AI programme of effective interventions:
Helpful
- Developed an appreciation that AI tools used selectively can support and extend learning rather than replace it.
- Built confidence in own judgement skills and ability to discern when and how to use AI, interpret output and evaluate its quality. AI output cannot be trusted.
- Learning that evaluating AI output can take longer than doing the task yourself.
- Helped to discover own weaknesses through awareness of tasks outsourced to AI.
Unhelpful
- The whole intervention process was repetitive, slow to progress to more complex AI activities, did not account for individual pace and levels of interest/expertise, and lead to ‘AI exhaustion’.
AI Perception and use shifts
- Sceptical -> Accepting AI as a useful learning tool
- Amazing -> Flawed
- Trusting -> Sceptical of AI output
- Not used -> Work more efficiently -> Depend on AI for specific tasks
- Unconfident -> Confident in AI use
- Less use since going through the AI intervention process
Main Concerns
- Dependence on AI leading to learning and skill atrophy.
- Uncertainty about institutional rules and how fairness is upheld.
Quantitative vs qualitative research
- AI is more suited to objective quantitative research than the subjective interpretation of qualitative data but provided a good start to find how qualitative analysis works.
- AI qualitative analysis is surface level, not as in-depth as a human researcher.
Future preferences
- Reduce the number of sessions, move to more sophisticated and varied AI activities sooner and have multiple activities to help people work at their own pace.
- Keep the critical evaluation assessments – low pressure motivation to learn (10% of unit mark) and very useful.
- Learn about AI background, how it works, other AI platforms and data privacy.
- Works well in research methods but provide more rationale for use in psychology research methods.
- More institutional clarity on contextualised AI use policy (when appropriate / not appropriate).
The Impact
According to students’ feedback and FGs analysis results, the interventions worked well to support the development of their AI skills, discerning optimal and ethical AI use, critical thinking skills, identifying their own academic weaknesses and research methods literacy.
The findings have shown that teaching students to think with AI and positioning GenAI as an object of critical inquiry is a constructive strategy that can encourage students to be more sceptical of it and more discerning consumers of AI. Open engagement did not increase uncritical use, whereas non-engagement might. This provides an alternative to only focusing on how to make assessments GenAI-proof and viewing AI as a threat to manage. Exposure can be an antidote to blind reliance.
Also, the findings have informed changes to the current programme of AI activities in the research methods units and provided considerations for initial development for AI interventions for other programmes and units:
The learning package matters
- GenAI literacy development is not a one-off session, but there can be too many sessions. Strategic progressive and tailored activities are required
- Provide some background on AI and its relevance to the specific topic area.
- Provide clear institutional / programme / unit policy on AI use (when appropriate/not appropriate).
- The workshops are given purpose through the assessment (10%) which provided a low-pressure motivation to learn.
Respect student capabilities
- Progress to more complex and sophisticated AI activities quickly.
- Provide multiple optional activities in a session to account for individual differences in pace and level of expertise/interest.
Next Steps
AI interventions and AI critical evaluation assessments will continue within the MSc Psychology conversion Research Methods units, but the AI activities will be tailored to the findings – reduced in number and progress quickly to more complex and meaningful tasks with less repetition. We will respond to student engagement and feedback and maintain relevance to students’ interests and effectiveness.
Post-intervention future planning will be discussed with the Education Director of the school to consider appropriate implementation in other programmes and units.
Publish journal article (e.g., Research and Practice in Technology Enhanced Learning, Journal of Further and Higher Education) in hope to inspire similar research within other HE programmes.
Contact
Laura Contu: laura.contu@bristol.ac.uk,
Michael Smyth: m.smyth@bristol.ac.uk
Leave a Reply