This is the first instalment of a four-part series looking at using Copilot Studio to create an AI agent that answers typical student questions regarding the submission of coursework.
Much of the discussion of AI in higher education has been on its use to support teaching and, perhaps more infamously, in the preparation of assessable materials. Aside from pedagogy, there is also an emerging interest in AI’s broader use in support and administrative services. Notable examples include Durham University’s Holly (an AI agent for providing information to prospective students) and an AI chatbot developed for the University of London’s Library to answer queries from distance learners. The use of AI in this area holds the familiar promises of increased efficiencies and relief of staff workload.
AI chatbots (agents) are becoming more common in the utilities and service sectors; many of us will have already encountered them on company websites as an initial point for customer support. Their advantage is that users can ask questions using natural language and be given the answer without having to navigate pages and pages of a knowledge base. The large language models (LLMs) used by these agents mean that it can interpret questions worded in a variety of ways. Agents are convenient because they are accessible 24/7. While they can resolve the majority of routine enquiries, a chatbot can (with enough cajoling) also escalate a query to a human operative, albeit within office hours.
Borrowing this idea from the real world of customer relations, a use of AI agents in higher education is to answer student queries that would normally be directed to academics or professional services. In the context of education, we can identify some obvious benefits to such a tool:
Benefits of AI chatbots
| Accessible at all times Conveniently for the user, chatbots can provide an ‘instant’ answer outside of office hours. |
| No social anxiety Students can be reticent to ask questions that they feel may make them appear naive. The impersonal nature of an automated response and text format could be more accessible especially for those where English is not their first language. |
| No need to read documentation Large, technical documentation can be less accessible for those with disabilities such as dyslexia and ADHD. Even for those where accessibility is not an issue, getting the answer directly is often more convenient. |
| May reduce staff workload Routine questions can be answered automatically with the option of escalating others. By ’triaging’ student queries, there is a potential workload saving. |
A bespoke AI chatbot requires instructions on how to react to user prompts and also its own specific knowledge used to answer these queries. A convenience of AI is that it can readily interpret documents that are written for human consumption i.e. Agents can be trained using existing websites, PDFs etc. and will be able to answer queries on that material with minimal development.
We demonstrate a simple example of an agent developed using the information for the coursework assessment of a pharmacology unit to answer routine student queries. We describe:
- the process of building this agent using a no-code approach in Microsoft Copilot Studio
- the agent’s performance with example prompts
- the potential advantages and limitations of using such agents, if deployed to students
In the Pharmacology of the Nervous System unit, students are asked to prepare an academic poster on a drug of their choice, which they subsequently present to their peers in a small group tutorial. There are a few stipulations about the content and presentation, so students are supported with an in-person assessment briefing, a preparatory tutorial with a member of academic staff as well as comprehensive written instructions.
The assessment has been previously delivered successfully to 165 students, who generally engaged well and satisfied most of the expected criteria. However, despite multiple contact points and the provision of written instructions, the majority of student queries that reach the administrative team are answerable from the provided material. While the reasons for this apparent lack of engagement with the given instruction may be complex, it could be pragmatic to provide access to this information via a chatbot – both for the convenience of students and the sanity of the unit’s administrators and director.
Further Reading
Industry report on the prevalence of AI in customer-facing service roles
Gartner
The US analytic firm, Gartner, predicts that conversational AI will reduce the labour costs associated with call centres by $80B worldwide with 1 in 10 agent interactions being fully automated by 2026. https://www.gartner.com/en/newsroom/press-releases/2022-08-31-gartner-predicts-conversational-ai-will-reduce-contac
Office for National Statistics June 2023 Report
In the UK, 17% of adults reported using AI chatbots for customer service. Interestingly, only 4% of businesses who responded in this poll reported using AI chatbots though whether these were for customer-facing applications was not recorded. https://www.ons.gov.uk/businessindustryandtrade/itandinternetindustry/articles/understandingaiuptakeandsentimentamongpeopleandbusinessesintheuk/june2023
Some examples of AI chatbots in the UK
UK Government https://insidegovuk.blog.gov.uk/2024/11/05/were-running-a-private-beta-of-gov-uk-chat/
View the second post in this series here: https://bilt.online/building-the-bot/
View the third post in this series here: https://bilt.online/getting-to-know-the-bot/
View the fourth post in this series here: https://bilt.online/evaluating-our-ai-agent-benefits-concerns-and-some-recommendations/
Leave a Reply