Asee peer logo

Work in Progress: Intelligent Tutoring Systems in Computer Science and Software Engineering Education

Download Paper |


2015 ASEE Annual Conference & Exposition


Seattle, Washington

Publication Date

June 14, 2015

Start Date

June 14, 2015

End Date

June 17, 2015





Conference Session

Computers in Education Engineering Division Poster Session

Tagged Division

Computers in Education

Tagged Topic


Page Count


Page Numbers

26.1754.1 - 26.1754.12



Permanent URL

Download Count


Request a correction

Paper Authors


John C Nesbit Simon Fraser University

visit author page

John Nesbit is a professor in the Faculty of Education at Simon Fraser University, Canada, where he conducts research on argumentation and learning, multimedia learning, self-regulated learning and the effectiveness of intelligent tutoring systems. He has collaborated with colleagues to publish over 60 peer-reviewed journal articles, book chapters and indexed conference papers.

visit author page


Li Liu Simon Fraser University

visit author page

Li Liu is a second-year doctoral student in Educational Technology and Learning Design at Simon Fraser University. With an interdisciplinary background in interaction design, media arts, and education, her passion lies in exploring how innovative technologies can be harnessed to promote teaching and learning.

visit author page


Qing Liu Simon Fraser University

visit author page

Qing Liu is a doctoral student in Educational Technology and Learning Design at Simon Fraser University. Her research focuses on conceptual change, the potential of learning by arguing, the role of need for cognition in learning, the effectiveness of intelligent tutoring systems, and meta-analysis of empirical studies.

visit author page


Olusola O Adesope Washington State University-Pullman

visit author page

Dr. Olusola O. Adesope is an Assistant Professor of Educational Psychology at Washington State University, Pullman. His research is at the intersection of educational psychology, learning sciences, and instructional design and technology. His recent research focuses on the cognitive and pedagogical underpinnings of learning with computer-based multimedia resources; knowledge representation through interactive concept maps; meta-analysis of empirical research, and investigation of instructional principles and assessments in STEM.

visit author page

Download Paper |


Work in Progress: Intelligent Tutoring Systems in Computer Science and Software Engineering EducationQuantitative analysis of bibliographic databases indicates that research on intelligent tutoringsystems (ITS) has accelerated over the last decade and that scholarly interest in such systems hasnever been greater. Recent meta-analyses of research which compared the effectiveness of ITS toother forms of instruction have reported average effect sizes favoring ITS of approximately .40standard deviations. Notably, the meta-analyses found no significant differences whencomparing ITS to one-to-one human tutoring and small group instruction.A meta-analysis of the 22 existing studies which evaluated the effectiveness of ITS in computerscience or software engineering education found a mean effect size of .46 standard deviations,but because of the necessarily narrow meta-analytic inclusion criteria the 22 studies give anincomplete and non-representative picture of all the empirical and design-oriented researchreports on the use of ITS in this subject area. To address this gap, we are conducting a systematicreview of research on the use of ITS in computer science and software engineering education.The review will identify the types of ITS that have been studied (e.g., the types of studentmodels, whether they model misconceptions, whether they personalize task assignment, whetherthey incorporate a pedagogical agent), the subject domains (e.g., introductory programming,database design), how the ITS was used (e.g., individualized instruction, facilitation of groupprojects), and it will categorize the studies on approximately 20 other moderator variables toallow us to construct a detailed profile of the research base. For example, we will identify whichinstructional functions (e.g., feedback, task selection, motivational prompts) have beeninvestigated by ITS research and which are under-researched. The systematic review willcritically evaluate research on the use of ITS in software engineering education and makerecommendations for improving the quality of methods and reporting in primary studies.We completed a search of five major bibliographic databases (IEEE, SpringerLink, Web ofScience, PsycInfo, ERIC) using the following search expression: ("computer science education" OR "software engineering education" OR "computer literacy" OR "database design" OR "network security" OR "introductory computer" OR "introductory programming" OR "teach* programming" OR "learn* programming") AND ("intelligent tutor*" OR "adaptive tutor*" OR "cognitive tutor*")The 805 unique reports returned by this search were examined and filtered using the followinginclusion criteria: 1. The paper must deal in a substantial way with an ITS. 2. The ITS must be intended for use or substantially evaluated in the curricular domain of software engineering education, computer science education, computer literacy, or cognate areas. 3. The paper must be a review, a system design report, a system evaluation report, or an instructional efficacy evaluation.After applying these criteria, 181 research reports were retained for inclusion in the systematicreview.The retained research reports will be coded by two researchers, and inter-coder agreement willbe calculated. In addition to the outcomes already mentioned, we anticipate that the review willidentify promising areas of future research and unexplored opportunities for meta-analysis.

Nesbit, J. C., & Liu, L., & Liu, Q., & Adesope, O. O. (2015, June), Work in Progress: Intelligent Tutoring Systems in Computer Science and Software Engineering Education Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.25090

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015