Development and Validation of the Engineering Design Metacognitive Questionnaire

Oenardi Lawanto; Harry B. Santoso

Download Paper | Permalink

Conference: 2014 ASEE Annual Conference & Exposition
Location: Indianapolis, Indiana
Publication Date: June 15, 2014
Start Date: June 15, 2014
End Date: June 18, 2014
ISSN: 2153-5965
Conference Session: Best of DEED
Tagged Division: Design in Engineering Education
Page Count: 11
Page Numbers: 24.412.1 - 24.412.11
DOI: 10.18260/1-2--20303
Permanent URL: https://peer.asee.org/20303
Download Count: 532

Paper Authors

biography

Oenardi Lawanto Utah State University

visit author page

Oenardi Lawanto is an assistant professor in the Department of Engineering Education at Utah State University, USA. He received his B.S.E.E. from Iowa State University, his M.S.E.E. from the University of Dayton, and his Ph.D. from the University of Illinois at Urbana-Champaign. Before coming to Utah State, Dr. Lawanto taught and held several administrative positions at one large private university in Indonesia. He has developed and delivered numerous international workshops on student-centered learning and online learning-related topics during his service. Dr. Lawanto’s research interests include cognition, learning, and instruction, and online learning.

visit author page

biography

Harry B. Santoso Utah State University

visit author page

Harry B. Santoso is a faculty member at Faculty of Computer Science, University of Indonesia. He received a BS and MS from Universitas Indonesia (UI) in Computer Science. Before pursuing his Ph.D. program majoring Engineering Education at Department of Engineering Education, Utah State University, he taught some courses at UI (e.g., computer-assisted instruction and multimedia technique). He has been an administrator of e-Learning system for several years in his department and university. He is also a member of E-School for Indonesia (Esfindo) research group that has main objective to promote a wide-access Internet-based e-Infrastructure for K-12 education. His research interest includes learning personalization, cognition and metacognition, multimedia content, e-Learning standardization, and distance learning.

visit author page

Download Paper | Permalink

Abstract

Development and Validation of the Engineering Design Metacognitive Questionnaire AbstractMetacognition is the process of thinking about thinking, which refers to students’ ability tocontrol cognition to ensure that learning goals are achieved or problem is solved. It is a complexprocess that depends on and influences students’ understanding about themselves as thinkers andlearners, and usually precedes and follows cognitive activity. Metacognitive skill plays aparticularly critical role in real-life or open-ended tasks, such as solving ill-structured designproblems. While there is growing interest in metacognitive research, few assessment tools havebeen developed in the context of engineering design, particularly within authentic classroomenvironments. The objective of the present paper is to discuss the process of Engineering Designand Metacognitive Questionnaire (EDMQ) development and validation, specifically the processof face and content validity.The instrument development is grounded in Butler and Cartier’s self-regulated learning (SRL)model which describes the interplay between motivation, cognition, and metacognition withinacademic activities such as design. The questionnaire is adapted from their works include theInquiry Learning Questionnaire and the Learning through Reading Questionnaire. A rubricmatrix combined Butler and Cartier’s SRL features and the Dym and Little’s design process wasused in the instrument development. Dym and Little contended that the design process consistsof five phases: problem definition, conceptual design, preliminary design, detailed design, anddesign communication.The EDMQ include items that address cognitive strategies both in design process and teammanagement activities. Three subsections of the EDMQ were designed to capture students’perception of metacognition at the early, middle, and final stages of the design task across designprocesses, respectively; the first subsection of EDMQ captures task interpretation and planningstrategies; the second subsection captures cognitive actions and monitoring and fix-up strategies;the third subsection captures students’ judgment of their design outcomes.Six undergraduate engineering students were invited in the face validity process. Moreover, thecontent validity involved two engineering professors and two experts in self-regulated learning.The resulting survey instrument contains 127 questionnaire items assessing five SRL features:task interpretation (21 items), planning strategies (22 items), cognitive actions (30 items),monitoring and fix-up strategies (33 items), and criteria of success (21 items). This developedand validated survey instrument may be useful for cognitive and metacognitive research andassessing design processes in the context of engineering design project.Keywords: engineering design, instrument development, metacognitive, questionnaire

Citation
Format

Lawanto, O., & Santoso, H. B. (2014, June), Development and Validation of the Engineering Design Metacognitive Questionnaire Paper presented at 2014 ASEE Annual Conference & Exposition, Indianapolis, Indiana. 10.18260/1-2--20303

TY  - CPAPER
AB  -      Development and Validation of the Engineering Design Metacognitive                              Questionnaire                                            AbstractMetacognition is the process of thinking about thinking, which refers to students’ ability tocontrol cognition to ensure that learning goals are achieved or problem is solved. It is a complexprocess that depends on and influences students’ understanding about themselves as thinkers andlearners, and usually precedes and follows cognitive activity. Metacognitive skill plays aparticularly critical role in real-life or open-ended tasks, such as solving ill-structured designproblems. While there is growing interest in metacognitive research, few assessment tools havebeen developed in the context of engineering design, particularly within authentic classroomenvironments. The objective of the present paper is to discuss the process of Engineering Designand Metacognitive Questionnaire (EDMQ) development and validation, specifically the processof face and content validity.The instrument development is grounded in Butler and Cartier’s self-regulated learning (SRL)model which describes the interplay between motivation, cognition, and metacognition withinacademic activities such as design. The questionnaire is adapted from their works include theInquiry Learning Questionnaire and the Learning through Reading Questionnaire. A rubricmatrix combined Butler and Cartier’s SRL features and the Dym and Little’s design process wasused in the instrument development. Dym and Little contended that the design process consistsof five phases: problem definition, conceptual design, preliminary design, detailed design, anddesign communication.The EDMQ include items that address cognitive strategies both in design process and teammanagement activities. Three subsections of the EDMQ were designed to capture students’perception of metacognition at the early, middle, and final stages of the design task across designprocesses, respectively; the first subsection of EDMQ captures task interpretation and planningstrategies; the second subsection captures cognitive actions and monitoring and fix-up strategies;the third subsection captures students’ judgment of their design outcomes.Six undergraduate engineering students were invited in the face validity process. Moreover, thecontent validity involved two engineering professors and two experts in self-regulated learning.The resulting survey instrument contains 127 questionnaire items assessing five SRL features:task interpretation (21 items), planning strategies (22 items), cognitive actions (30 items),monitoring and fix-up strategies (33 items), and criteria of success (21 items). This developedand validated survey instrument may be useful for cognitive and metacognitive research andassessing design processes in the context of engineering design project.Keywords: engineering design, instrument development, metacognitive, questionnaire
AU  - Oenardi Lawanto
AU  - Harry B. Santoso
CY  - Indianapolis, Indiana
DA  - 2014/06/15
PB  - ASEE Conferences
TI  - Development and Validation of the Engineering Design Metacognitive Questionnaire
UR  - https://peer.asee.org/20303
DO  - 10.18260/1-2--20303
ER  -