by Barbara Geyer, Rita Stampfl and Elisabeth Hauser (University of Applied Sciences Burgenland)
It is becoming increasingly important to integrate educational chatbots, particularly those that use Socratic methods. The development of Large Language Models such as GPT-4 have made it possible to create more nuanced and interactive conversations, enabling chatbots to support more complex learning processes and provide personalised learning experiences. Two examples of chatbots that use Socratic methods to help students understand and learn the Scrum framework are the Scrum Tutor and the Scrum Assistant. These tutors were developed using GPTs and were evaluated through controlled testing and user feedback to ensure technical functionality and didactic effectiveness. The Socratic teaching method encourages critical thinking and self-reflection among learners, making it an effective approach for developing educational chatbots that actively involve students in the learning process.
Chatbots, particularly those created for educational purposes, are becoming more and more significant in the rapidly changing world of education. This trend accompanied by the wide availability of Large Language Models (LLMs) like ChatGPT, particularly the ability to create GPTs without programming experience, creates new possibilities for integrating chatbots into learning environments. GPTs are customised versions of ChatGPT that can be created for specific purposes to assist users with tasks and share these customisations with other users [L1]. However, to ensure that these chatbots act not only as question-and-answer machines but also as intelligent tutors, it is important to instruct them accordingly. As demonstrated in this article, this can be accomplished with Socratic methods and Socratic educational chatbots for Scrum.
The Socratic teaching method was named after ancient Greek philosopher Socrates. Using a dialogue approach, the teacher asks questions to promote critical thinking and self-reflection among learners. According to Delic and Bećirović [1] this method does not simply provide answers, but encourages learners to find solutions themselves. The Socratic teaching method can also be used to develop chatbots that actively engage students in the learning process and stimulate self-reflection.
Since the 1960s, chatbots have been a part of the education sector, evolving from simple answering devices to sophisticated teaching assistants. Kuhail et al. [2] described the evolution of this technology, from teaching to interactive agents. According to them, the majority of chatbots have thus far used chatbot-driven conversation to guide users through pre-structured dialogues (flow-based), whereas a minority (11.11%) have relied on user-driven conversation in which learners control the interaction through AI-supported responses.
The development of LLMs, such as GPT, has fundamentally changed the capabilities of educational chatbots and encouraged a move towards user-driven conversation chatbots. Chatbots can now have more complex and engaging conversations than ever before thanks to these sophisticated models, which go far beyond preprogrammed responses. Chang [3] introduced the Socratic method based on GPT-3. These “Socratic chatbots” use targeted questions to promote critical thinking and self-learning instead of providing direct answers. This creates a dialogue-oriented learning approach that expands chatbots’ ability to support complex learning processes and enables personal learning experiences. Socratic educational chatbots can be implemented more readily due to the capabilities of GPT-4 and the ease with which GPTs can be created [L1].
Against this background, the advancements described by Chang [3] serve as the foundation for a new generation of educational chatbots. With the extended capacity of GPT-4 and the ease with which customised GPTs [L1] can now be created, Socratic educational chatbots can now be realised more efficiently. This makes it possible to create instructional resources that support learning and critical thinking in addition to offering solutions. When developing tutors, the initial focus was on identifying the learning objectives and analysing the needs of the target group. Based on this, a didactic concept was designed that integrates the Socratic method as the core of learning interaction, with a focus on the development of questioning strategies that promote critical thinking and autonomous learning. The prompts were newly developed but inspired by the prompt examples of Chang [3], Hardman [L2], and Smith [L3].
The purpose of the Scrum Tutor is to provide process- and understanding-oriented instruction to students who have never worked with Scrum before. Instead of providing direct answers, the tutor asks specific questions that encourage users to think about and critically question the Scrum framework’s principles. This not only promotes understanding of the topic, but also trains analytical thinking and problem-solving skills.
On the other hand, the Scrum Learning Assistant uses a more all-encompassing approach that is adaptable to various learning levels and not tailored to a specific target group. The assistant supports learners in expanding their knowledge through interactive and user-oriented conversations in which they actively take control of their learning process. With the help of dynamically generated questions that build on users’ previous answers, the Scrum Learning Assistant creates a personalised learning experience that encourages independence. Figure 1 shows an example of a dialogue with both the Scrum Tutor and the Scrum Learning Assistant.
Figure 1: Dialogue with the Scrum Tutor and the Scrum Learning Assistant.
The prototypes of these tutors were created using the GPTs in GPT-4 and evaluated in various test phases. Initial testing was conducted in a controlled environment to verify the technical and didactic effectiveness. User feedback was collected to measure the quality of interaction and learning success. This feedback was crucial for iterative adjustments to both the target group approach and functional design of chatbots. Ongoing tests and adjustments served to increase the effectiveness of tutors and ensure a personalised learning experience that actively supports users in their learning processes.
Both examples demonstrate how the application of the Socratic method in chatbots simulates a dialogue between teachers and learners that goes beyond the mere retrieval of information and supports the development of a sound understanding of the subject matter.
Sal Kahn impressively described how Khan Academy and OpenAI were working together to create a Socratic chatbot for mathematics in his TED Talk [L4]. This case study explains how educational institutions with limited resources can nevertheless apply this strategy by using GPTs based on GPT-4. In the master’s degree programme in e-learning and knowledge management, we will also teach students how to create Socratic educational chatbots in the coming summer semester [L5]. The integration of the Socratic method into teaching transforms these chatbots into core components of modern educational strategies.
Links:
[L1] https://openai.com/blog/introducing-gpts
[L2] https://drphilippahardman.substack.com/p/introducing-chatgpt-edu-mega-prompts
[L3] https://thatryanp.medium.com/my-go-to-prompt-for-chatgpt-socratic-coach-7bf0dd2c01ec
[L4] https://www.youtube.com/watch?v=hJP5GqnTrNo
[L5] https://www.fh-burgenland.at/en/master-e-learning-und-wissensmanagement/
References:
[1] H. Delic and S. Bećirović, “Socratic method as an approach to teaching,” European Researcher, vol. 111, no. 10, pp. 511–517, Oct. 2016, doi: 10.13187/er.2016.111.511
[2] M. A. Kuhail et al., “Interacting with educational chatbots: A systematic review,” Educ Inf Technol, vol. 28, no. 1, pp. 973–1018, 2022, doi: 10.1007/s10639-022-11177-3
[2] E. Y. Chang, “Prompting Large Language Models with the Socratic Method,” IEEE Computing and Communication Workshop and Conference (CCWC), March 2023, doi: 10.48550/ARXIV.2303.08769
Please contact:
Barbara Geyer, University of Applied Sciences Burgenland, Austria
