June 14, 2015
June 14, 2015
June 17, 2015
Computers in Education
26.1754.1 - 26.1754.12
Work in Progress: Intelligent Tutoring Systems in Computer Science and Software Engineering EducationQuantitative analysis of bibliographic databases indicates that research on intelligent tutoringsystems (ITS) has accelerated over the last decade and that scholarly interest in such systems hasnever been greater. Recent meta-analyses of research which compared the effectiveness of ITS toother forms of instruction have reported average effect sizes favoring ITS of approximately .40standard deviations. Notably, the meta-analyses found no significant differences whencomparing ITS to one-to-one human tutoring and small group instruction.A meta-analysis of the 22 existing studies which evaluated the effectiveness of ITS in computerscience or software engineering education found a mean effect size of .46 standard deviations,but because of the necessarily narrow meta-analytic inclusion criteria the 22 studies give anincomplete and non-representative picture of all the empirical and design-oriented researchreports on the use of ITS in this subject area. To address this gap, we are conducting a systematicreview of research on the use of ITS in computer science and software engineering education.The review will identify the types of ITS that have been studied (e.g., the types of studentmodels, whether they model misconceptions, whether they personalize task assignment, whetherthey incorporate a pedagogical agent), the subject domains (e.g., introductory programming,database design), how the ITS was used (e.g., individualized instruction, facilitation of groupprojects), and it will categorize the studies on approximately 20 other moderator variables toallow us to construct a detailed profile of the research base. For example, we will identify whichinstructional functions (e.g., feedback, task selection, motivational prompts) have beeninvestigated by ITS research and which are under-researched. The systematic review willcritically evaluate research on the use of ITS in software engineering education and makerecommendations for improving the quality of methods and reporting in primary studies.We completed a search of five major bibliographic databases (IEEE, SpringerLink, Web ofScience, PsycInfo, ERIC) using the following search expression: ("computer science education" OR "software engineering education" OR "computer literacy" OR "database design" OR "network security" OR "introductory computer" OR "introductory programming" OR "teach* programming" OR "learn* programming") AND ("intelligent tutor*" OR "adaptive tutor*" OR "cognitive tutor*")The 805 unique reports returned by this search were examined and filtered using the followinginclusion criteria: 1. The paper must deal in a substantial way with an ITS. 2. The ITS must be intended for use or substantially evaluated in the curricular domain of software engineering education, computer science education, computer literacy, or cognate areas. 3. The paper must be a review, a system design report, a system evaluation report, or an instructional efficacy evaluation.After applying these criteria, 181 research reports were retained for inclusion in the systematicreview.The retained research reports will be coded by two researchers, and inter-coder agreement willbe calculated. In addition to the outcomes already mentioned, we anticipate that the review willidentify promising areas of future research and unexplored opportunities for meta-analysis.
Nesbit, J. C., & Liu, L., & Liu, Q., & Adesope, O. O. (2015, June), Work in Progress: Intelligent Tutoring Systems in Computer Science and Software Engineering Education Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.25090
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015