Asee peer logo

How We Know They're Learning: Comparing Approaches to Longitudinal Assessment of Transferable Learning Outcomes

Download Paper |


2016 ASEE Annual Conference & Exposition


New Orleans, Louisiana

Publication Date

June 26, 2016

Start Date

June 26, 2016

End Date

June 29, 2016





Conference Session

Assessment II: Learning Gains and Conceptual Understanding

Tagged Division

Educational Research and Methods

Tagged Topic


Page Count




Permanent URL

Download Count


Request a correction

Paper Authors


Brian M. Frank Queen's University

visit author page

Brian Frank is the DuPont Canada Chair in Engineering Education Research and Development, and the Director of Program Development in the Faculty of Engineering and Applied Science at Queen's University where he works on engineering curriculum development, program assessment, and developing educational technology. He is also an associate professor in Electrical and Computer Engineering.

visit author page


Natalie Simper Queen's University

visit author page

Natalie Simper coordinates a Queen's research project investigating the development and measurement of general learning outcomes. Natalie comes from an Australian Senior-Secondary/ Post-Secondary teaching background, with experience at the State-wide level in curriculum development, large-scale assessment, and evaluation and assessment of outcomes based education.

visit author page


James A. Kaupp Queen's University

visit author page

Assessment and Quality Assurance Coordinator (Msc ’06, PhD ’12) at Queen’s University, Kingston, Ontario, Canada in the Faculty of Engineering and Applied Science. Educational research interests include engineering education development, cultural change in higher education, higher-order thinking development and assessment, outcomes-based data-informed continuous improvement, information visualization & analysis and authentic performance-based assessment.

visit author page

Download Paper |


This research paper describes interim results from a 4-year longitudinal study of how engineering students develop critical thinking, problem solving, and communication skills. The sample includes approximately 400 students in a mid-sized research intensive Canadian institution. The students were assessed using multiple approaches, including standardized test, in-course activities, surveys, and course artefacts scored by a trained team using program-wide rubrics. Outcomes demonstrated in student course artefacts externally scored by VALUE rubric assessment increased over the two years. Scores on standardized tests generally trend upward with the Critical thinking Assessment Test (CAT) but are mixed on the Collegiate Learning Assessment (CLA+), most likely due to motivational and alignment issues. Student motivation is a significant issue in the project. The paper compares the assessment methods, and finds that using externally scored course artefacts is both less expensive and preferred by course instructors over standardized tests.

Frank, B. M., & Simper, N., & Kaupp, J. A. (2016, June), How We Know They're Learning: Comparing Approaches to Longitudinal Assessment of Transferable Learning Outcomes Paper presented at 2016 ASEE Annual Conference & Exposition, New Orleans, Louisiana. 10.18260/p.25493

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2016 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015