By now, most of us have engaged with an artificially intelligent (AI) chatbot or virtual assistant. The degree to which we engage with chatbots depends upon their sophistication, given the variety of technologies available today. For instance, some chatbots only provide guidance with a predetermined set of activities, others can interpret natural language questions, and a few even get more intelligent with each interaction. In any case, chatbots offer a self-service option accessible 24 hours a day, seven days a week.
Still, institutions value the needs and preferences of their students. Suffice to say, if students don’t like using a chatbot, it’s reasonable to expect that they won’t. At Indiana University, Antino Grim led a research team to help determine whether students prefer their AI chatbot, HoosierBot, or their traditional self-help resource.
Introducing HoosierBot
Before implementing HoosierBot, built by Ivy.ai, students had access to a search tool for self-help that functioned similarly to a Google search. To find answers, students entered one or more keywords to retrieve a list of ranked knowledge base articles. Indiana University created the resource more than three decades ago, and it has been a well-developed function on IU’s website ever since.
To determine which method students prefer more, the research team engaged in a controlled study of 261 undergraduate students who would interact with both the chatbot and the search tool. After participating, the students answered survey questions to measure satisfaction, likelihood of use, and perceived difficulty on a seven-point scale.
In the study, students were provided with questions to present to HoosierBot and the search tool. IU verified that both tools were equipped to provide answers for each question. In the end, researches found statistically significant outcomes in favour of HoosierBot.
Students rated HoosierBot with a higher level of overall satisfaction, and said they would be more likely to use HoosierBot to answer their questions than a traditional search method. According to the study, it’s also substantially easier to use the chatbot.
“It seems like an efficient way to get my questions answered without spending hours scrolling through IU’s website and randomly guessing what things are,” one student said.
“I like that it seems like you are texting someone when you are getting help,” said another.
Student satisfaction
Based on the results, researchers concluded that the chatbot had a significant positive effect on satisfaction, and that students are more likely to use a chatbot for self-help compared to traditional search methods. Importantly, students reported significantly lower perceived difficulty when using a chatbot to find answers to their questions.
These responses suggest that students appreciated the more conversational nature of a chatbot, while being shorter and more to-the-point. Specifically, traditional search methods require students to review multiple results and select the best answer.
Chatbots also have an advantage in that they can interpret meaning from natural language, whereas some search methods may rely on the presence of keywords. For example, at the university of Wisconsin Madison, the Office of Student Financial Aid learned that students would often use their AI chatbot for help with ‘down payments’ when referring to tuition deposits.
While the study at Indiana University provides independent and objective insight into student preference, the outcome is no surprise. In 2019, the University of Portsmouth installed their AI chatbot, Anna, and presented it as an option for students second to chatting with an agent. Overwhelmingly, they found that students elected to engage with Anna, so much so that it reduced their live chat inquiries by 50%.
Accordingly, AI chatbots are uniquely capable of delivering an improved service experience while also increasing operational efficiency. After Temple University installed Owl Bot on the Bursar’s website, call volume fell by 50% as students elected to engage with the bot for on-demand help rather than contact a representative.
AI chatbots have already had a significant impact on how colleges and universities connect with students. The Comprehensive Guide to AI Chatbots in Higher Education takes a deep dive into artificial intelligence, its role in chatbots, how chatbots differ from live chat, and the future of AI in higher ed. Get the guide and learn how your institution can benefit from AI.
To find out more, get the guide.