Defining Measurement Constructs for Assessing Learning in Makerspaces

Leonardo Pollettini Marcos; Julie S. Linsey; Melissa Wood Aleman; Robert L. Nagel; Kerrie A. Douglas; Eric Holloway

Download Paper | Permalink

Conference: 2024 ASEE Annual Conference & Exposition
Location: Portland, Oregon
Publication Date: June 23, 2024
Start Date: June 23, 2024
End Date: July 12, 2024
Conference Session: Educational Research and Methods Division (ERM) Technical Session 8
Tagged Division: Educational Research and Methods Division (ERM)
Permanent URL: https://peer.asee.org/47111

Request a correction

Paper Authors

biography

Leonardo Pollettini Marcos Purdue University

visit author page

Leonardo Pollettini Marcos is a 3rd-year PhD student at Purdue University's engineering education program. He completed a bachelor's and a master's degree in Materials Engineering at the Federal University of Sao Carlos, Brazil. His research interests are in assessment instruments and engineering accreditation processes.

visit author page

biography

Julie S. Linsey Georgia Institute of Technology

visit author page

Dr. Julie S. Linsey is a Professor in the George W. Woodruff School of Mechanical Engineering at the Georgia Institute of Technological. Her research focus is on design methods, theory, and engineering education with a particular focus on innovation and conceptual design.

visit author page

biography

Melissa Wood Aleman James Madison University

visit author page

Dr. Melissa Aleman (Ph.D. University of Iowa) is Professor of Communication Studies at James Madison University and has published research using qualitative interviewing, ethnographic and rhetorical methods to examine communication in diverse contexts. Sh

visit author page

biography

Robert L. Nagel Carthage College

visit author page

Dr. Robert Nagel is a Professor and Director of the Department of Engineering at Carthage College. Dr. Nagel, a mechanical engineer by training, performs research on engineering student learning and engagement with a focus on interventions, pedagogies, and design methodologies. He seeks to gain applicable knowledge for increasing student engagement and reducing barriers in engineering, design, and making.

visit author page

biography

Kerrie A. Douglas Purdue University orcid.org/0000-0002-2693-5272

visit author page

Dr. Douglas is an Associate Professor in the Purdue School of Engineering Education. Her research is focused on improving methods of assessment in engineering learning environments and supporting engineering students.

visit author page

biography

Eric Holloway Purdue University orcid.org/0000-0002-0343-1709

visit author page

Prof. Eric Holloway currently serves as a Professor of Engineering Practice in the School of Mechanical Engineering at Purdue University at West Lafayette. He also holds a courtesy faculty appointment in the School of Engineering Education. His research focuses on assessment development and the professional formation of students.

visit author page

Download Paper | Permalink

Abstract

This research paper presents the initial construct definitions for an assessment instrument to measure student learning in makerspaces. Makerspaces enable learning through social interaction and hands-on activities when creating physical solutions to a problem. Due to the positive perception of the impact of makerspaces on student learning, these spaces have drawn the attention of different types of institutions, including libraries, communities, and higher education. As such, new makerspaces are constantly being created, and research about those spaces is also proliferating. However, there are currently no instruments with evidence of validity and reliability that can be used for assessing learning in makerspaces. Therefore, as the first step in the process of generating an instrument, this study seeks to answer the research question: “What are the definitions of constructs for learning in a makerspace?” To create our construct definitions, we first assembled a team of instrument development experts and makerspace experts. The makerspace experts had previously conducted several qualitative studies on makerspaces, culminating in a Learning Typology for Makerspaces. Following the principles of instrument development outlined by Netemeyer et al. (2003), our team created concise essence statements for each of the constructs in the typology, which summarize the main idea behind each of the constructs we want to measure. Next, we created conceptualization statements derived from the essence statements, expanding on each construct’s meaning by incorporating key empirical knowledge of the makerspace experts. Finally, we conducted a literature review to ground the final definitions for each construct. This literature review was guided by the ideas present in the essence and conceptualization statements, and thus, the final definitions expand on the empirical knowledge of our experts with other perspectives reported on the literature. We created a set of essence and conceptualization statements along with a formal definition supported by the literature for a total of six constructs related to learning in makerspaces. The six constructs are (1) Learning by Doing, related to the process of learning through active engagement in maker activities; (2) Learning by Others, related to the process of learning through engagement with other people or artifacts created by others; (3) Content Knowledge and Skills, related to the technical disciplinary knowledge learned in makerspaces; (4) Cultural Knowledge and Skills, related to learning and navigating the culture of a makerspace; (5) Ingenuity, related to the inventiveness of learners when creating solutions constrained by their making environment; and (6) Self-awareness, related to learners’ development of transferable attitudes, motivation, and character. The definitions were created as a starting point for developing a quantitative instrument for measuring learning in makerspaces. Having experts in makerspaces along with experts in instrument development proved to be beneficial to the process, as it allowed the concepts to be explored in greater depth. These resultant definitions enable the continuation of the development of a makerspace instrument, while also serving as an operationalization of learning in makerspaces for the wider research community.

Citation
Format

Pollettini Marcos, L., & Linsey, J. S., & Aleman, M. W., & Nagel, R. L., & Douglas, K. A., & Holloway, E. (2024, June), Defining Measurement Constructs for Assessing Learning in Makerspaces Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/47111

TY - CPAPER
AB - This research paper presents the initial construct definitions for an assessment instrument to measure student learning in makerspaces. Makerspaces enable learning through social interaction and hands-on activities when creating physical solutions to a problem. Due to the positive perception of the impact of makerspaces on student learning, these spaces have drawn the attention of different types of institutions, including libraries, communities, and higher education. As such, new makerspaces are constantly being created, and research about those spaces is also proliferating. However, there are currently no instruments with evidence of validity and reliability that can be used for assessing learning in makerspaces. Therefore, as the first step in the process of generating an instrument, this study seeks to answer the research question: “What are the definitions of constructs for learning in a makerspace?” To create our construct definitions, we first assembled a team of instrument development experts and makerspace experts. The makerspace experts had previously conducted several qualitative studies on makerspaces, culminating in a Learning Typology for Makerspaces. Following the principles of instrument development outlined by Netemeyer et al. (2003), our team created concise essence statements for each of the constructs in the typology, which summarize the main idea behind each of the constructs we want to measure. Next, we created conceptualization statements derived from the essence statements, expanding on each construct’s meaning by incorporating key empirical knowledge of the makerspace experts. Finally, we conducted a literature review to ground the final definitions for each construct. This literature review was guided by the ideas present in the essence and conceptualization statements, and thus, the final definitions expand on the empirical knowledge of our experts with other perspectives reported on the literature. We created a set of essence and conceptualization statements along with a formal definition supported by the literature for a total of six constructs related to learning in makerspaces. The six constructs are (1) Learning by Doing, related to the process of learning through active engagement in maker activities; (2) Learning by Others, related to the process of learning through engagement with other people or artifacts created by others; (3) Content Knowledge and Skills, related to the technical disciplinary knowledge learned in makerspaces; (4) Cultural Knowledge and Skills, related to learning and navigating the culture of a makerspace; (5) Ingenuity, related to the inventiveness of learners when creating solutions constrained by their making environment; and (6) Self-awareness, related to learners’ development of transferable attitudes, motivation, and character. The definitions were created as a starting point for developing a quantitative instrument for measuring learning in makerspaces. Having experts in makerspaces along with experts in instrument development proved to be beneficial to the process, as it allowed the concepts to be explored in greater depth. These resultant definitions enable the continuation of the development of a makerspace instrument, while also serving as an operationalization of learning in makerspaces for the wider research community.
AU - Leonardo Pollettini Marcos
AU - Julie S. Linsey
AU - Melissa Wood Aleman
AU - Robert L. Nagel
AU - Kerrie A. Douglas
AU - Eric Holloway
CY - Portland, Oregon
DA - 2024/06/23
PB - ASEE Conferences
TI - Defining Measurement Constructs for Assessing Learning in Makerspaces
UR - https://peer.asee.org/47111
ER -