Evaluation of the Use of ChatGPT in Online Discussions: Its Impact on Student Understanding of Material and Interaction

Fadli Firdaus; Mia Syafrina

doi:10.51574/ijrer.v5i1.4531

Authors

Fadli Firdaus Department of Electrical Engineering, Politeknik Negeri Batam
Mia Syafrina Department of Business Management, Politeknik Negeri Batam

Keywords:

AI Dependence, ChatGPT, Material Understanding, Online Discussion, Student Interaction

Abstract

The rapid incorporation of artificial intelligence, notably ChatGPT, in online learning settings has raised questions about its impact on pedagogy. It is unclear whether AI use intensity inherently improves interaction quality and student pleasure or if other cognitive aspects like material knowledge are more important. This study analyzes two models—student interaction quality and student satisfaction—to assess ChatGPT's multifunctional role in online conversations. This quantitative study examines ChatGPT usage intensity, AI dependency, content understanding, interaction quality, and student happiness using structural models. The mechatronics program at Batam State Polytechnic collected data from students using AI techniques in online discussion forums. The findings indicated that the most consistent and dominant predictor of interaction quality and satisfaction is material understanding. ChatGPT intensity and content understanding promote interaction quality in Model 1, demonstrating its procedural support role. Model 2 demonstrates that student satisfaction is more influenced by material understanding and tool trust (dependence) than frequency of usage, suggesting that satisfaction is subjective and effective. The study reveals that AI integration alone does not guarantee success; pupils' cognitive competence and AI adaptation play a significant role. AI's procedural and affective effects on learning are distinguished in this educational technology study. It assists educators in positioning ChatGPT as a "thinking partner" instead of a quick fix. It suggests that pedagogical focus must remain on strengthening students' foundational grasp of concepts before introducing AI tools to ensure high-quality, sustainable online discourse.

References

Alabi, A. J., Oyekola, A. H., & Joyce, O. O. (2025). Lecturers’ Perception and Adoption of Artificial Intelligence (AI) Tools in Tertiary Institutions in Ondo State, Nigeria. Indonesian Journal of Research and Educational Review e–ISSN, 4(4), 1664-1676. https://doi.org/10.51574/ijrer.v4i4.3883

Al-Adwan, A. S., Li, N., Al-Adwan, A., Abbasi, G. A., Albelbisi, N. A., & Habibi, A. (2023). Extending the technology acceptance model (TAM) to Predict University Students’ intentions to use metaverse-based learning platforms. Education and Information Technologies, 28(11), 15381-15413. https://doi.org/10.1007/s10639-023-11816-3

Ba, S., Zhan, Y., Huang, L., & Lu, G. (2025). Investigating the impact of ChatGPT‐assisted feedback on the dynamics and outcomes of online inquiry‐based discussion. British Journal of Educational Technology. https://doi.org/10.1111/bjet.13605

Baskara, F. R., Puri, A. D., & Wardhani, A. R. (2023). ChatGPT and the Pedagogical Challenge: Unveiling the impact on early-Career academics in Higher Education. Indonesian Journal on Learning and Advanced Education (IJOLAE), 311-322. https://doi.org/10.23917/ijolae.v5i3.22966

Batsaikhan, B. Z., & Correia, A. P. (2024). The effects of Generative Artificial Intelligence on Intelligent Tutoring Systems in higher education: A systematic review. Studies in Technology Enhanced Learning, 4(1). https://stel.pubpub.org/pub/04-01-batsaikhan-correia/release/1

Bonde, L. (2024). A Generative Artificial Intelligence Based Tutor for Personalized Learning. In 2024 IEEE SmartBlock4Africa (pp. 1-10). IEEE. https://doi.org/10.1109/SmartBlock4Africa61928.2024.10779525

Cheah, C. W. (2021). Developing a gamified AI-enabled online learning application to improve students’ perception of university physics. Computers and Education: Artificial Intelligence, 2, 100032. https://doi.org/10.1016/j.caeai.2021.100032

Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications.

Fathony, M. H., Hizraini, A. A., Aulia, R., & Almaisarah, A. (2024). Persepsi Mahasiswa Terhadap Penggunaan Chatgpt Di Universitas Nahdlatul Ulama Kalimantan Selatan. Jurnal Review Pendidikan dan Pengajaran (JRPP), 7(4), 16601-16607. https://doi.org/10.31004/jrpp.v7i4.33299

Fatmadiwi, A., Hartoyo, A., & Erlina, E. (2025). Kebijakan Artificial Intelligence (AI) dalam Pembelajaran di Perguruan Tinggi. Jurnal Alwatzikhoebillah: Kajian Islam, Pendidikan, Ekonomi, Humaniora, 11(1), 284-291. https://doi.org/10.37567/alwatzikhoebillah.v11i1.3633

Garasut, N., Wenas, J. R., & Maukar, M. G. (2024). Persepsi Mahasiswa Terhadap Penggunaan ChatGPT pada Mata Kuliah Pemodelan Matematika. SOSCIED, 7(2), 570-576. https://doi.org/10.32531/jsoscied.v7i2.846

Gerlich, M. (2025). AI tools in society: Impacts on cognitive offloading and the future of critical thinking. Societies, 15(1), 6. https://doi.org/10.3390/soc15010006

Herlina, H. (2021). Persepsi Mahasiswa pada Pembelajaran Daring Mengunakan Forum Diskusi dan Gmeet pada Masa Pandemi COVID-19. Jurnal Pendidikan Indonesia, 2(11), 1897-1907. https://doi.org/10.59141/japendi.v2i11.338

Hooda, M., Rana, C., Dahiya, O., Rizwan, A., & Hossain, M. S. (2022). Artificial intelligence for assessment and feedback to enhance student success in higher education. Mathematical Problems in Engineering, 2022(1), 5215722. https://doi.org/10.1155/2022/5215722

Krystalli, P., & Arvanitis, P. (2025). Integrating AI Into Task-Based Language Teaching: Large Language Models in the Designing of Web Quests. In Harnessing Chatbots for Effective Language Teaching (pp. 117-150). IGI Global Scientific Publishing. https://doi.org/10.4018/979-8-3693-9974-3.ch005

Lee, H. Y., Chen, P. H., Wang, W. S., Huang, Y. M., & Wu, T. T. (2024). Empowering ChatGPT with guidance mechanism in blended learning: Effect of self-regulated learning, higher-order thinking skills, and knowledge construction. International Journal of Educational Technology in Higher Education, 21(1), 16. https://doi.org/10.1186/s41239-024-00447-4

Ouyang, F., Zheng, L., & Jiao, P. (2022). Artificial intelligence in online higher education: A systematic review of empirical research from 2011 to 2020. Education and Information Technologies, 27(6), 7893-7925. https://doi.org/10.1007/s10639-022-10925-9

Pahi, K., Hawlader, S., Hicks, E., Zaman, A., & Phan, V. (2024). Enhancing active learning through collaboration between human teachers and generative AI. Computers and Education Open, 6, 100183. https://doi.org/10.1016/j.caeo.2024.100183

Papaneophytou, C., & Nicolaou, S. A. (2025). Promoting Critical Thinking in Biological Sciences in the Era of Artificial Intelligence: The Role of Higher Education. Trends in Higher Education, 4(2), 24. https://doi.org/10.3390/higheredu4020024

Raheef, O. O., & Olayiwola, S. O. (2025). Intellectual Security and Artificial Intelligence: Managing the Balance Between Positivity and Negativity. ETDC: Indonesian Journal of Research and Educational Review, 4(4), 1143-1156. https://doi.org/10.51574/ijrer.v4i4.3194

Owoc, M. L., Sawicka, A., & Weichbroth, P. (2019). Artificial intelligence technologies in education: benefits, challenges and strategies of implementation. IFIP international workshop on artificial intelligence for knowledge management (pp. 37-58). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-85001-2_4

Saif, N., Khan, S. U., Shaheen, I., ALotaibi, F. A., Alnfiai, M. M., & Arif, M. (2024). Chat-GPT; validating Technology Acceptance Model (TAM) in education sector via ubiquitous learning mechanism. Computers in Human Behavior, 154, 108097. https://doi.org/10.1016/j.chb.2023.108097

Saykılı, A. (2019). Higher education in the digital age: The impact of digital connective technologies. Journal of Educational Technology and Online Learning, 2(1), 1-15. https://doi.org/10.31681/jetol.516971

Susmita, N., Zaim, M., Thahar, H. E., & Wahyuni, S. (2024). Pemanfaatan Media Kecerdasan Buatan dalam Pembelajaran Bahasa Indonesia Tingkat Sekolah Menengah Atas: Perspektif Siswa. Visipena, 15(1), 80-95. https://doi.org/10.46244/visipena.v15i2.2688

Thomas, D. R., Gatz, E., Gupta, S., Lin, J., Tipper, C., & Koedinger, K. R. (2024). Using generative AI to provide feedback to adult tutors in training and assess real-life performance. The Learning Ideas Conference (pp. 204-214). Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-73427-4_21

Zhai, C., Wibowo, S., & Li, L. D. (2024). The effects of over-reliance on AI dialogue systems on students' cognitive abilities: a systematic review. Smart Learning Environments, 11(1), 28. https://doi.org/10.1186/s40561-024-00316-7