Asee peer logo

Development of a Direct Assessment for Measuring Students’ Ability to Make Connections

Download Paper |

Conference

2021 ASEE Virtual Annual Conference Content Access

Location

Virtual Conference

Publication Date

July 26, 2021

Start Date

July 26, 2021

End Date

July 19, 2022

Conference Session

Entrepreneurship and Engineering Innovation Division Technical Session 2

Tagged Division

Entrepreneurship & Engineering Innovation

Page Count

11

DOI

10.18260/1-2--36956

Permanent URL

https://peer.asee.org/36956

Download Count

31

Request a correction

Paper Authors

biography

Meg West Ohio State University Orcid 16x16 orcid.org/0000-0002-1031-7381

visit author page

Meg West is an engineering education graduate student at The Ohio State University. She is a graduate research associate for the Department of Engineering Education.

visit author page

biography

Meagan Eleanor Ita Ohio State University

visit author page

Meagan is a Postdoctoral Scholar in the Engineering Education Department at The Ohio State University. She graduated with her Bachelor's (2013) and Master's (2014) in Biomedical Engineering from The Ohio State University and completed her Ph.D. in Bioengineering at the University of Pennsylvania in 2020. Meagan conducted research on the biomechanics and physiology of chronic pain for her doctorate degree and has experience teaching undergraduate first-year engineering and mid-level biomechanics courses. Meagan is currently working with the KEEN partnership at OSU, integrating her interests in STEM education, entrepreneurial partnerships, and community engagement. Meagan values authenticity, connection with others, & integrity and prioritizes these values as an educator, bioengineer, and scientist.

visit author page

biography

Laine Rumreich Ohio State University

visit author page

Laine Rumreich is a graduate student studying Computer Science and Engineering at The Ohio State University. She completed her undergraduate research thesis in the Department of Engineering Education and is now a graduate research associate in the department. Her primary research interests are in the areas of coding education and engineering entrepreneurship.

visit author page

biography

Rachel Louis Kajfez Ohio State University Orcid 16x16 orcid.org/0000-0001-9745-1921

visit author page

Dr. Rachel Louis Kajfez is an Assistant Professor in the Department of Engineering Education at The Ohio State University. She earned her B.S. and M.S. degrees in Civil Engineering from Ohio State and earned her Ph.D. in Engineering Education from Virginia Tech. Her research interests focus on the intersection between motivation and identity of undergraduate and graduate students, first-year engineering programs, mixed methods research, and innovative approaches to teaching. She is the faculty lead for the Research on Identity and Motivation in Engineering (RIME) Collaborative.

visit author page

biography

Krista M. Kecskemety Ohio State University

visit author page

Krista Kecskemety is an Assistant Professor of Practice in the Department of Engineering Education at The Ohio State University. Krista received her B.S. in Aerospace Engineering at The Ohio State University in 2006 and received her M.S. from Ohio State in 2007. In 2012, Krista completed her Ph.D. in Aerospace Engineering at Ohio State. Her engineering education research interests include investigating first-year engineering student experiences, faculty experiences, and the connection between the two.

visit author page

Download Paper |

Abstract

The integration of entrepreneurial minded learning (EML) into engineering courses to develop students’ entrepreneurial mindset (EM) is growing in popularity through efforts such as the Kern Entrepreneurial Engineering Network (KEEN). As the integration of EML occurs, it is important to assess students’ development of an EM to inform the efficacy of course changes. Following KEEN’s definition, we operationalize EM as the ability to create value, make connections, and be curious. While we acknowledge the importance of all aspects of EML, in this full paper, we focus on assessing students’ ability to make connections. Concept maps have been previously used to assess student development of an EM as a whole, and we believe they can also be used to specifically assess the ability of students to make connections. Specifically, we collected responses from a pre-existing individual concept map activity used in two sections of a first-year engineering course and two sections of an aerospace engineering course at The Ohio State University. A total of 238 responses were collected, 106 responses from the first-year engineering course and 132 responses from the aerospace engineering course. The concept maps were evaluated using the traditional concept map scoring method. Through our analysis, we found no strong correlation between course grades and scores on concept maps developed by students for the course. This result supports moving forward with the concept map scoring methodology without the need for a correction factor related to grades. That said, other results indicated the need for modifications to the concept map instructions and scoring method that accounted for intra-hierarchy connections. In future studies, we will explore these findings further including the possibility of creating a new concept map scoring method with a stronger focus on measuring connections.

For this work, we collected responses from a pre-existing concept map activity used in 2 sections of a first-year engineering course and 2 sections of an aerospace engineering course at a large, public Midwest university. The activity in both courses involved students individually creating a concept map of their course at the end of the semester. A total of 200 responses were collected, 135 responses from the first-year engineering course and 170 responses from the aerospace engineering course. The concept maps were evaluated using the Integrated Rubric for Scoring Concept Maps. This rubric assesses concept maps for their comprehensiveness, organization, and correctness. One concern of using the Integrated Rubric was the possibility of bias that could occur due to the potential connection between students’ success in the course and their ability to make meaningful connections in the concept map about the course. This potential bias would provide an inaccurate measure of students’ ability to make connections. Using each student’s concept map score and final grade in the course, statistical analysis was performed to determine if the correlation between the scores and grades was high enough to warrant a change in the assessment methodology. This change would account for potential bias if this activity were to be used to measure students’ ability to make connections.

Through our analysis, which is currently ongoing but will be completed by the conference paper deadline, we posit there will be a strong positive correlation between students’ final grade and their concept map score. This analysis will inform modifications made to the direct assessment methodology including possibly adding a correction factor based on student success in the course so the concept mapping activity can be used to measure students’ ability to make connections. The direct assessment developed as a result of this study will be used in future studies related to the integration of EML into various engineering course content.

West, M., & Ita, M. E., & Rumreich, L., & Kajfez, R. L., & Kecskemety, K. M. (2021, July), Development of a Direct Assessment for Measuring Students’ Ability to Make Connections Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--36956

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2021 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015