June 12, 2005
June 12, 2005
June 15, 2005
10.335.1 - 10.335.18
Concept Inventories Meet Cognitive Psychology: Using Beta Testing as a Mechanism for Identifying Engineering Student Misconceptions
Ronald L. Miller, Ruth A. Streveler Colorado School of Mines
Mary A. Nelson, Monica R. Geist University of Colorado/University of Northern Colorado
Barbara M. Olds National Science Foundation and Colorado School of Mines
This paper summarizes our analysis of beta test results collected at four U.S. engineering schools using the Thermal and Transport Concept Inventory (TTCI) currently under development. The instrument consists of questions in heat transfer, fluid mechanics, and thermodynamics and is designed to elicit information about the nature of undergraduate engineering students’ misconceptions in these disciplines. In addition to using conventional methods for determining TTCI reliability with correlation coefficients, we have applied cross-tabulation data analysis for six heat transfer questions to identify robust student misconceptions. The results of these analyses identified two categories of misconceptions which persist across heat transfer problems and contexts: 1) confusion about differences between energy and temperature, and 2) confusion about differences in steady-state and equilibrium processes.
With funding from the National Science Foundation (DUE-0127806), a team of researchers at the Colorado School of Mines (CSM) is creating a concept inventory to measure engineering students’ understanding of difficult concepts in thermal and transport science, the Thermal and Transport Concept Inventory (TTCI).  Our project began with a Delphi survey of experts to determine the concepts that were the most difficult and important for engineering students studying thermodynamics, heat transfer, and fluid mechanics.  Once the key concepts were identified, we created open-ended questions related to each concept and asked students to think aloud as they answered the questions. Multiple-choice questions were then developed using students’ misunderstandings as documented in the think-alouds as the basis for writing distractors (incorrect but plausible answers). The resulting inventory was alpha tested at CSM , expanded, and beta tested at four other engineering institutions.
Standard psychometric procedures include testing to establish various forms of reliability (how repeatable are the results?) and validity (are we really testing the concepts that we intend to test?) for an instrument under development. We are currently analyzing beta test data from the TTCI to establish these parameters, and more detailed findings will be reported elsewhere. However, we are also using the beta test data to develop methods for identifying the kinds of misunderstandings or misconceptions students hold about concepts included in the TTCI. In
Proceedings of the 2005 American Society for Engineering Education Annual Conference & Exposition Copyright © 2005, American Society for Engineering Education
Nelson, M., & Olds, B., & Geist, M., & Miller, R., & Streveler, R. (2005, June), Concept Inventories Meet Cognitive Psychology: Using Beta Testing As A Mechanism For Identifying Engineering Student Misconceptions Paper presented at 2005 Annual Conference, Portland, Oregon. https://peer.asee.org/14620
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2005 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015