Assessing Learning Gains Attributable to Curricular Innovations

Mukasa E. Ssemakula; Gene Yeau-Jian Liao; Shlomo S. Sawilowsky

Download Paper | Permalink

Conference: 2016 ASEE Annual Conference & Exposition
Location: New Orleans, Louisiana
Publication Date: June 26, 2016
Start Date: June 26, 2016
End Date: June 29, 2016
ISBN: 978-0-692-68565-5
ISSN: 2153-5965
Conference Session: Assessment II: Learning Gains and Conceptual Understanding
Tagged Division: Educational Research and Methods
Page Count: 14
DOI: 10.18260/p.26301
Permanent URL: https://216.185.13.131/26301
Download Count: 457

Request a correction

Paper Authors

biography

Mukasa E. Ssemakula Wayne State University

visit author page

Mukasa E. Ssemakula is a Professor in the Division of Engineering Technology, at Wayne State University in Detroit, Michigan. He received his Ph.D. from the University of Manchester Institute of Science and Technology, in England. After working in industry, he served on the faculty of the University of Maryland before joining Wayne State. He is a leader in developing and implementing new pedagogical approaches to engineering education. He also has research interests in the area of manufacturing systems. Contact: m.e.ssemakula@wayne.edu

visit author page

biography

Gene Yeau-Jian Liao Wayne State University

visit author page

GENE LIAO is currently Director of the Electric-drive Vehicle Engineering and Alternative Energy Technology programs and Professor at Wayne State University. He received a M.S. in mechanical engineering from Columbia University, and a doctor of engineering from University of Michigan, Ann Arbor. He has over 17 years of industrial practices in the automotive sector prior to becoming a faculty member. Dr. Liao has research and teaching interests in the areas of hybrid vehicles, energy storage, and advanced manufacturing.

visit author page

biography

Shlomo S. Sawilowsky Wayne State University

visit author page

https://en.wikipedia.org/wiki/Shlomo_Sawilowsky
http://coe.wayne.edu/profile/aa3290
http://www.genealogy.ams.org/id.php?id=85370
https://scholar.google.com/citations?user=6FvTCOwAAAAJ

visit author page

Download Paper | Permalink

Abstract

This Evidence-Based Practice paper is motivated by industry’s identification of lack of hands-on experience as one of the major competency gaps in engineering education. This need has led to the development of new engineering and technology curricula balancing theoretical knowledge with integrated hands-on experiences. While such curricular are a welcome development, less has been done to formally assess the learning gains actually attributable to these new approaches. This paper describes a long-term project which has developed an innovative curricular model that provides students with hands-on skills highly sought by industry; as well as an accompanying standardized test to measure student achievement on the competencies spanned by the curricular innovation. It gives a formal summative evaluation of the curricular model; and describes a comparative study to compare the learning gains achieved under the curricular model with those attained by comparison groups studying the same content but without participating in the specific curricular innovation.

This proof-of-concept, utilizing a standardized test incorporating a physical manipulative to evaluate attainment of hands-on engineering competencies, is unique in the field. There are four implementing schools and three comparison schools involved in the study. The standardized test is administered at all participating schools, with the student performance data being transmitted to investigators at the project coordinating site for analysis.

During the 2014/15 academic year, the standardized test was administered to N = 125 students. A variety of psychometric analyses were conducted on the test results. Chronbach’s α was found to be 0.783 and specific test items for revision or deletion to improve test reliability were identified. Exploratory factor analysis was then used iteratively, which produced the final factor analysis solution. The final 5 factor model accounted for 60.8% of total explained variance. Finally, individual test item statistics were compiled detailing each item’s difficulty index and discrimination ability. The results show a robust and reliable test instrument.

Citation
Format

Ssemakula, M. E., & Liao, G. Y., & Sawilowsky, S. S. (2016, June), Assessing Learning Gains Attributable to Curricular Innovations Paper presented at 2016 ASEE Annual Conference & Exposition, New Orleans, Louisiana. 10.18260/p.26301

TY - CPAPER
AB -
This Evidence-Based Practice paper is motivated by industry’s identification of lack of hands-on experience as one of the major competency gaps in engineering education. This need has led to the development of new engineering and technology curricula balancing theoretical knowledge with integrated hands-on experiences. While such curricular are a welcome development, less has been done to formally assess the learning gains actually attributable to these new approaches. This paper describes a long-term project which has developed an innovative curricular model that provides students with hands-on skills highly sought by industry; as well as an accompanying standardized test to measure student achievement on the competencies spanned by the curricular innovation. It gives a formal summative evaluation of the curricular model; and describes a comparative study to compare the learning gains achieved under the curricular model with those attained by comparison groups studying the same content but without participating in the specific curricular innovation.

This proof-of-concept, utilizing a standardized test incorporating a physical manipulative to evaluate attainment of hands-on engineering competencies, is unique in the field. There are four implementing schools and three comparison schools involved in the study. The standardized test is administered at all participating schools, with the student performance data being transmitted to investigators at the project coordinating site for analysis.

During the 2014/15 academic year, the standardized test was administered to N = 125 students. A variety of psychometric analyses were conducted on the test results. Chronbach’s α was found to be 0.783 and specific test items for revision or deletion to improve test reliability were identified. Exploratory factor analysis was then used iteratively, which produced the final factor analysis solution. The final 5 factor model accounted for 60.8% of total explained variance. Finally, individual test item statistics were compiled detailing each item’s difficulty index and discrimination ability. The results show a robust and reliable test instrument.

AU - Mukasa E. Ssemakula
AU - Gene Yeau-Jian Liao
AU - Shlomo S. Sawilowsky
CY - New Orleans, Louisiana
DA - 2016/06/26
PB - ASEE Conferences
TI - Assessing Learning Gains Attributable to Curricular Innovations
UR - https://216.185.13.131/26301
DO - 10.18260/p.26301
ER -