Asee peer logo

Assessing Learning Gains Attributable to Curricular Innovations

Download Paper |


2016 ASEE Annual Conference & Exposition


New Orleans, Louisiana

Publication Date

June 26, 2016

Start Date

June 26, 2016

End Date

August 28, 2016





Conference Session

Assessment II: Learning Gains and Conceptual Understanding

Tagged Division

Educational Research and Methods

Page Count




Permanent URL

Download Count


Request a correction

Paper Authors


Mukasa E. Ssemakula Wayne State University

visit author page

Mukasa E. Ssemakula is a Professor in the Division of Engineering Technology, at Wayne State University in Detroit, Michigan. He received his Ph.D. from the University of Manchester Institute of Science and Technology, in England. After working in industry, he served on the faculty of the University of Maryland before joining Wayne State. He is a leader in developing and implementing new pedagogical approaches to engineering education. He also has research interests in the area of manufacturing systems. Contact:

visit author page


Gene Yeau-Jian Liao Wayne State University

visit author page

GENE LIAO is currently Director of the Electric-drive Vehicle Engineering and Alternative Energy Technology programs and Professor at Wayne State University. He received a M.S. in mechanical engineering from Columbia University, and a doctor of engineering from University of Michigan, Ann Arbor. He has over 17 years of industrial practices in the automotive sector prior to becoming a faculty member. Dr. Liao has research and teaching interests in the areas of hybrid vehicles, energy storage, and advanced manufacturing.

visit author page

Download Paper |


This Evidence-Based Practice paper is motivated by industry’s identification of lack of hands-on experience as one of the major competency gaps in engineering education. This need has led to the development of new engineering and technology curricula balancing theoretical knowledge with integrated hands-on experiences. While such curricular are a welcome development, less has been done to formally assess the learning gains actually attributable to these new approaches. This paper describes a long-term project which has developed an innovative curricular model that provides students with hands-on skills highly sought by industry; as well as an accompanying standardized test to measure student achievement on the competencies spanned by the curricular innovation. It gives a formal summative evaluation of the curricular model; and describes a comparative study to compare the learning gains achieved under the curricular model with those attained by comparison groups studying the same content but without participating in the specific curricular innovation.

This proof-of-concept, utilizing a standardized test incorporating a physical manipulative to evaluate attainment of hands-on engineering competencies, is unique in the field. There are four implementing schools and three comparison schools involved in the study. The standardized test is administered at all participating schools, with the student performance data being transmitted to investigators at the project coordinating site for analysis.

During the 2014/15 academic year, the standardized test was administered to N = 125 students. A variety of psychometric analyses were conducted on the test results. Chronbach’s α was found to be 0.783 and specific test items for revision or deletion to improve test reliability were identified. Exploratory factor analysis was then used iteratively, which produced the final factor analysis solution. The final 5 factor model accounted for 60.8% of total explained variance. Finally, individual test item statistics were compiled detailing each item’s difficulty index and discrimination ability. The results show a robust and reliable test instrument.

Ssemakula, M. E., & Liao, G. Y., & Sawilowsky, S. S. (2016, June), Assessing Learning Gains Attributable to Curricular Innovations Paper presented at 2016 ASEE Annual Conference & Exposition, New Orleans, Louisiana. 10.18260/p.26301

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2016 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015