New Orleans, Louisiana
June 26, 2016
June 26, 2016
August 28, 2016
Educational Research and Methods
This Evidence-Based Practice paper is motivated by industry’s identification of lack of hands-on experience as one of the major competency gaps in engineering education. This need has led to the development of new engineering and technology curricula balancing theoretical knowledge with integrated hands-on experiences. While such curricular are a welcome development, less has been done to formally assess the learning gains actually attributable to these new approaches. This paper describes a long-term project which has developed an innovative curricular model that provides students with hands-on skills highly sought by industry; as well as an accompanying standardized test to measure student achievement on the competencies spanned by the curricular innovation. It gives a formal summative evaluation of the curricular model; and describes a comparative study to compare the learning gains achieved under the curricular model with those attained by comparison groups studying the same content but without participating in the specific curricular innovation.
This proof-of-concept, utilizing a standardized test incorporating a physical manipulative to evaluate attainment of hands-on engineering competencies, is unique in the field. There are four implementing schools and three comparison schools involved in the study. The standardized test is administered at all participating schools, with the student performance data being transmitted to investigators at the project coordinating site for analysis.
During the 2014/15 academic year, the standardized test was administered to N = 125 students. A variety of psychometric analyses were conducted on the test results. Chronbach’s α was found to be 0.783 and specific test items for revision or deletion to improve test reliability were identified. Exploratory factor analysis was then used iteratively, which produced the final factor analysis solution. The final 5 factor model accounted for 60.8% of total explained variance. Finally, individual test item statistics were compiled detailing each item’s difficulty index and discrimination ability. The results show a robust and reliable test instrument.
Ssemakula, M. E., & Liao, G. Y., & Sawilowsky, S. S. (2016, June), Assessing Learning Gains Attributable to Curricular Innovations Paper presented at 2016 ASEE Annual Conference & Exposition, New Orleans, Louisiana. 10.18260/p.26301
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2016 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015