Asee peer logo

Development and Validation of the Engineering Design Metacognitive Questionnaire

Download Paper |


2014 ASEE Annual Conference & Exposition


Indianapolis, Indiana

Publication Date

June 15, 2014

Start Date

June 15, 2014

End Date

June 18, 2014



Conference Session

Best of DEED

Tagged Division

Design in Engineering Education

Page Count


Page Numbers

24.412.1 - 24.412.11



Permanent URL

Download Count


Request a correction

Paper Authors


Oenardi Lawanto Utah State University

visit author page

Oenardi Lawanto is an assistant professor in the Department of Engineering Education at Utah State University, USA. He received his B.S.E.E. from Iowa State University, his M.S.E.E. from the University of Dayton, and his Ph.D. from the University of Illinois at Urbana-Champaign. Before coming to Utah State, Dr. Lawanto taught and held several administrative positions at one large private university in Indonesia. He has developed and delivered numerous international workshops on student-centered learning and online learning-related topics during his service. Dr. Lawanto’s research interests include cognition, learning, and instruction, and online learning.

visit author page


Harry B. Santoso Utah State University

visit author page

Harry B. Santoso is a faculty member at Faculty of Computer Science, University of Indonesia. He received a BS and MS from Universitas Indonesia (UI) in Computer Science. Before pursuing his Ph.D. program majoring Engineering Education at Department of Engineering Education, Utah State University, he taught some courses at UI (e.g., computer-assisted instruction and multimedia technique). He has been an administrator of e-Learning system for several years in his department and university. He is also a member of E-School for Indonesia (Esfindo) research group that has main objective to promote a wide-access Internet-based e-Infrastructure for K-12 education. His research interest includes learning personalization, cognition and metacognition, multimedia content, e-Learning standardization, and distance learning.

visit author page

Download Paper |


Development and Validation of the Engineering Design Metacognitive Questionnaire AbstractMetacognition is the process of thinking about thinking, which refers to students’ ability tocontrol cognition to ensure that learning goals are achieved or problem is solved. It is a complexprocess that depends on and influences students’ understanding about themselves as thinkers andlearners, and usually precedes and follows cognitive activity. Metacognitive skill plays aparticularly critical role in real-life or open-ended tasks, such as solving ill-structured designproblems. While there is growing interest in metacognitive research, few assessment tools havebeen developed in the context of engineering design, particularly within authentic classroomenvironments. The objective of the present paper is to discuss the process of Engineering Designand Metacognitive Questionnaire (EDMQ) development and validation, specifically the processof face and content validity.The instrument development is grounded in Butler and Cartier’s self-regulated learning (SRL)model which describes the interplay between motivation, cognition, and metacognition withinacademic activities such as design. The questionnaire is adapted from their works include theInquiry Learning Questionnaire and the Learning through Reading Questionnaire. A rubricmatrix combined Butler and Cartier’s SRL features and the Dym and Little’s design process wasused in the instrument development. Dym and Little contended that the design process consistsof five phases: problem definition, conceptual design, preliminary design, detailed design, anddesign communication.The EDMQ include items that address cognitive strategies both in design process and teammanagement activities. Three subsections of the EDMQ were designed to capture students’perception of metacognition at the early, middle, and final stages of the design task across designprocesses, respectively; the first subsection of EDMQ captures task interpretation and planningstrategies; the second subsection captures cognitive actions and monitoring and fix-up strategies;the third subsection captures students’ judgment of their design outcomes.Six undergraduate engineering students were invited in the face validity process. Moreover, thecontent validity involved two engineering professors and two experts in self-regulated learning.The resulting survey instrument contains 127 questionnaire items assessing five SRL features:task interpretation (21 items), planning strategies (22 items), cognitive actions (30 items),monitoring and fix-up strategies (33 items), and criteria of success (21 items). This developedand validated survey instrument may be useful for cognitive and metacognitive research andassessing design processes in the context of engineering design project.Keywords: engineering design, instrument development, metacognitive, questionnaire

Lawanto, O., & Santoso, H. B. (2014, June), Development and Validation of the Engineering Design Metacognitive Questionnaire Paper presented at 2014 ASEE Annual Conference & Exposition, Indianapolis, Indiana. 10.18260/1-2--20303

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2014 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015