Optimizing Database Query Learning: A Generative AI Approach for Semantic Error Feedback

Abdulrahman AlRabah; Sophia Yang; Abdussalam Alawini

Download Paper | Permalink

Conference: 2024 ASEE Annual Conference & Exposition
Location: Portland, Oregon
Publication Date: June 23, 2024
Start Date: June 23, 2024
End Date: June 26, 2024
Conference Session: Teaching with ML and Generative AI
Tagged Division: Computers in Education Division (COED)
Page Count: 19
DOI: 10.18260/1-2--47819
Permanent URL: https://peer.asee.org/47819
Download Count: 221

Paper Authors

biography

Abdulrahman AlRabah University of Illinois Urbana-Champaign

visit author page

Abdulrahman AlRabah is a Master of Science (M.S.) in Computer Science student at the University of Illinois at Urbana-Champaign. He holds a Graduate Certificate in Computer Science from the same institution and a Bachelor of Science in Mechanical Engineering from California State University, Northridge. He has experience in various industries and has served in multiple roles throughout his professional career, including in oil and gas and co-founding a food & beverage company. His research focuses on identifying semantic errors of students and optimizing AI feedback using customized large language models (LLMs) through fine-tuning. Abdulrahman has worked on enhancing AI-based feedback models using both open and closed source models. His work aims to improve SQL teaching methodologies and develop tools that integrate machine learning concepts to enhance both student learning and instructor teaching experiences in computer education.

visit author page

biography

Sophia Yang University of Illinois Urbana-Champaign orcid.org/0000-0003-1274-4851

visit author page

Sophia Yang is a second-year Ph.D. candidate with research work focused in the areas of Computing Education, Database Systems, Bioinformatics algorithms, Human-Computer Interaction, and interesting intersections of the above. Current research work focuses on quantitatively and qualitatively studying how students learn SQL by utilizing Bioinformatics alignment algorithms, aiming to create instructor and student-facing tools. Her work aims to better improve students’ learning and instructors’ teaching experiences in large-scale database courses.

visit author page

biography

Abdussalam Alawini University of Illinois Urbana-Champaign

visit author page

Abdussalam Alawini, a Teaching Associate Professor at the University of Illinois at Urbana-Champaign, combines an extensive industry background with academic achievements. Starting with a Computer Science degree from the University of Tripoli, he spent over six years in the tech industry before earning master's degrees in Computer Science and Engineering and Technology Management from Portland State University, where he also completed his Ph.D. His doctoral work focused on enhancing file-based dataset management for scientists. Dr. Alawini's research spans databases, applied machine learning, and educational technology, aiming to improve classroom experiences and develop advanced data management systems.

visit author page

Download Paper | Permalink

Abstract

Prior research has highlighted the significant challenges that students face when navigating database systems, particularly in mastering SQL and NoSQL query languages. These challenges typically fall into two categories: syntax errors and semantic errors. Syntax errors occur when a student's query violates the grammatical rules of the SQL or NoSQL language, resulting in queries that the database system cannot execute. On the other hand, semantic errors arise when a query is syntactically correct but does not produce the expected result because it does not accurately reflect the student's intention or understanding of the data. Numerous common error types and overarching learning hurdles have been identified among learners, with a predominant focus on syntax errors in previous research. However, there has been a noticeable gap in the study and categorization of semantic errors, which are equally critical for students’ learning and proficiency in database systems. Our study aims to fill this gap and contribute to the educational domain by significantly improving the precision and efficiency in identifying semantic errors in student submissions of SQL and NoSQL queries. We strive to achieve this by integrating the advanced capabilities of a Generative Pre-Trained Transformer (GPT) model with an existing feedback system, enhancing both the accuracy and effectiveness of error detection. We have utilized diverse datasets of student submissions, which were employed to fine-tune our GPT models. This tailored training process has enabled the models to better recognize and highlight semantic errors, while simultaneously providing constructive and meaningful feedback. The GPT models, through this customized training, have developed a deeper understanding of common student errors, leading to notable improvements in error detection accuracy and feedback quality. Preliminary results from our research are highly encouraging, demonstrating significant advancements and highlighting the potential of large language models in database learning. By integrating these state-of-the-art computational tools into the learning environment, our study lays the groundwork for the creation of intelligent systems that offer nuanced and context-aware feedback. Such systems have the potential to significantly alleviate the learning obstacles associated with database systems, thereby enhancing the educational experience and support available to students

Citation
Format

AlRabah, A., & Yang, S., & Alawini, A. (2024, June), Optimizing Database Query Learning: A Generative AI Approach for Semantic Error Feedback Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. 10.18260/1-2--47819

TY  - CPAPER
AB  - Prior research has highlighted the significant challenges that students face when navigating database systems, particularly in mastering SQL and NoSQL query languages. These challenges typically fall into two categories: syntax errors and semantic errors. Syntax errors occur when a student&#39;s query violates the grammatical rules of the SQL or NoSQL language, resulting in queries that the database system cannot execute. On the other hand, semantic errors arise when a query is syntactically correct but does not produce the expected result because it does not accurately reflect the student&#39;s intention or understanding of the data. Numerous common error types and overarching learning hurdles have been identified among learners, with a predominant focus on syntax errors in previous research.
However, there has been a noticeable gap in the study and categorization of semantic errors, which are equally critical for students’ learning and proficiency in database systems. Our study aims to fill this gap and contribute to the educational domain by significantly improving the precision and efficiency in identifying semantic errors in student submissions of SQL and NoSQL queries. We strive to achieve this by integrating the advanced capabilities of a Generative Pre-Trained Transformer (GPT) model with an existing feedback system, enhancing both the accuracy and effectiveness of error detection. We have utilized diverse datasets of student submissions, which were employed to fine-tune our GPT models. This tailored training process has enabled the models to better recognize and highlight semantic errors, while simultaneously providing constructive and meaningful feedback. The GPT models, through this customized training, have developed a deeper understanding of common student errors, leading to notable improvements in error detection accuracy and feedback quality.
Preliminary results from our research are highly encouraging, demonstrating significant advancements and highlighting the potential of large language models in database learning. By integrating these state-of-the-art computational tools into the learning environment, our study lays the groundwork for the creation of intelligent systems that offer nuanced and context-aware feedback. Such systems have the potential to significantly alleviate the learning obstacles associated with database systems, thereby enhancing the educational experience and support available to students
AU  - Abdulrahman AlRabah
AU  - Sophia Yang
AU  - Abdussalam Alawini
CY  - Portland, Oregon
DA  - 2024/06/23
PB  - ASEE Conferences
TI  - Optimizing Database Query Learning: A Generative AI Approach for Semantic Error Feedback
UR  - https://peer.asee.org/47819
DO  - 10.18260/1-2--47819
ER  -