by Rita Stampfl (University of Applied Sciences Burgenland), Barbara Geyer (University of Applied Sciences Burgenland)
At the University of Applied Sciences Burgenland, a GPT-based chatbot has been developed to support students in creating research topic proposals for scientific papers. Large Language Models like GPT-4 enable interactive conversations, allowing chatbots to facilitate complex learning processes and provide personalised learning experiences. In the rapidly changing educational landscape, specially designed educational chatbots are gaining importance. This trend, combined with the accessibility of Large Language Models and the ability to create GPTs without programming knowledge, opens new possibilities for integration them into learning environments. To ensure these chatbots function as intelligent tutors rather than simple question-answer machines, appropriate instruction is essential.
To understand the context, it is worth looking briefly at historical developments. Chatbots have been present in the education sector since the 1960s and have evolved from simple response devices to highly sophisticated learning assistants. Kuhail et al. [1] describe this evolution from teaching agents to interactive ones. According to them, the majority of chatbots have used chatbot-driven conversation to guide users through pre-structured dialogues, whereas only a minority have relied on user-driven conversation in which learners control the interaction through AI-supported responses. LLMs like ChatGPT have fundamentally changed the capabilities of educational chatbots and made user-controlled conversations possible. Thanks to these models, chatbots can now conduct significantly more complex and demanding conversations than was possible with pre-programmed responses. These so-called Socratic chatbots use targeted questions to promote critical thinking and self-reflection instead of providing direct answers. This creates a dialogue-oriented learning approach that expands learners' abilities to support complex learning processes and enables personal learning experiences.
When developing a GPT as a research topic assistant [L1], the learning objectives were first identified and the needs of the target group analysed. Based on this, a didactic concept was developed that integrates the creation of a scientific paper as the core of the learning interaction. The focus is on questioning strategies that promote critical thinking and autonomous learning. The GPT was designed as a role-play game to leverage the advantages of game-based learning. These enable learning as a combination of theory and application in authentic situations [2]. Role-play games demonstrably increase student interest in learning. The GPT prompt was newly developed following the methodology described by Stampfl and Prodinger [3], who demonstrated how ChatGPT can be effectively employed as an assistant for topic selection in scientific papers.
Figure 1: Collaborative XR learning environment as a basis.
The “Themendispo Assistant” operates according to a structured dialogue principle. Students are systematically guided through various dimensions of their research project. The interaction begins with identifying the overarching topic area and systematically delves into more specific aspects such as problem formulation, resulting research gaps, goal definition, and scientific questioning. The theoretical background, choice of methods, and anticipated results are also critically examined. Through targeted questions, the tutor encourages users to consider and challenge the structure of their research project. This not only promotes understanding of the topic but also trains analytical thinking and problem-solving skills.
A particular strength of the “Themendispo Assistant” lies in its ability to evaluate logical consistency between elements of the research design. It accepts only precise, scientifically grounded answers. In cases of inconsistency or insufficient precision, it requests revision. The communicative style of the Themendispo Assistant is deliberately formal and demanding. At the same time, the constructive support function is not neglected. This requires a high degree of scientific rigour and conceptual clarity from students, corresponding to academic standards of excellence. The assistant supports learners in expanding their knowledge through dynamically generated questions that build on users' previous answers, creating a personalised learning experience that encourages independence.
The prototypes of these tutors were created using the GPTs in GPT-4 and evaluated in various test phases. Initial testing was conducted in a controlled environment to verify the technical and didactic effectiveness. User feedback was collected to measure the quality of interaction and learning success. This feedback was crucial for iterative adjustments to both the target group approach and functional design of the chatbot. Ongoing tests and adjustments served to increase the effectiveness of tutors and ensure a personalised learning experience that actively supports users in their learning processes.
The GPT was designed as a complement to conventional supervision. The Themendispo Assistant represents progress in applying artificial intelligence in education and illustrates the potential of adaptive dialogue systems for quality assurance in academic training. Through systematic guidance in developing consistent research concepts, the system promotes scientific competence among students while reducing demands on academic staff. The application of Socratic methods in chatbots stimulates a dialogue between teachers and learners that goes beyond the mere retrieval of information and supports the development of a sound understanding of the subject matter. Future research could examine the long-term effects of such AI tutors on the quality of scientific work.
Links:
[L1] https://chatgpt.com/g/g-679b33d93aa48191a4ce54252f77d89b-themendispo-assistent
[L2] https://openai.com/blog/introducing-gpts
[L3] https://barbarageyer.substack.com
[L4] https://www.linkedin.com/in/barbara-geyer/
References:
[1] M. A. Kuhail et al., "Interacting with educational chatbots: A systematic review," Education and Information Technologies, vol. 28, no. 1, pp. 973-1018, 2023. https://doi.org/10.1007/s10639-022-11177-3
[2] J. Matute Vallejo and I. Melero, "Learning through play: The use of business simulators in higher education teaching," Universia Business Review, vol. 51, pp. 72-111, 2016. https://doi.org/10.3232/UBR.2016.V13.N3.03
[3] R. Stampfl and M. Prodinger, "KI-Planspiel zur Themendisposition: ChatGPT als Assistent zur Themenfindung fur wissenschaftliche Arbeiten," R&E-SOURCE, vol. 11, no. 4, pp. 119-130, 2024. https://doi.org/10.53349/resource.2024.i4.a1345
Please contact:
Rita Stampfl
University of Applied Sciences Burgenland, Austria
