Columbus, Ohio
June 24, 2017
June 24, 2017
June 28, 2017
Educational Research and Methods
8
10.18260/1-2--28315
https://peer.asee.org/28315
491
Lori C. Bland, Ph.D., is an associate professor at George Mason University. She teaches courses in educational assessment, program evaluation, and data-driven decision-making. Bland received her Ph.D. in Educational Psychology from the University of Virginia. Her current research focuses on identifying, examining, and assessing learning and professional outcomes in formal and informal learning environments in K-12, higher education, and the workforce; how data is used from assessments to inform decision-making; and the application of assessment or evaluation methods to solve educational problems.
Stephanie Kusano is an assessment and evaluation postdoctoral research associate at the Center for Research on Learning and Teaching at University of Michigan. She has a Ph.D. in Engineering Education, M.S. in Biomedical Engineering, and B.S. in Mechanical Engineering, all from Virginia Tech. Her research interests include engaged learning and high impact practices, assessment, and design education. Her teaching experience has primarily been with first-year engineering.
Xingya Xu is a Ph.D candidate of the Department of Educational Psychology in the College of Education and Human Development at George Mason University. She has a M.S. in Psychological Science at Western Kentucky University. Her research interests include metacognition, epistemic cognition and self-regulated learning.
Aditya Johri is Associate Professor in the department of Information Sciences & Technology. Dr. Johri studies the use of information and communication technologies (ICT) for learning and knowledge sharing, with a focus on cognition in informal environments. He also examine the role of ICT in supporting distributed work among globally dispersed workers and in furthering social development in emerging economies. He received the U.S. National Science Foundation’s Early Career Award in 2009. He is co-editor of the Cambridge Handbook of Engineering Education Research (CHEER) published by Cambridge University Press, New York, NY. Dr. Johri earned his Ph.D. in Learning Sciences and Technology Design at Stanford University and a B.Eng. in Mechanical Engineering at Delhi College of Engineering.
Motivation and Background Co-curricular engineering experiences range from very informal, such as face-to-face social networking within an engineering club to service learning experiences that approximate the workplace. Such experiences situate learning within an environment that may foster integration of knowledge and skills to solve problems (Pierrakos, Borrego, & Lo, 2007). These informal learning environments represent degrees of complexity. Therefore, applying design or problem solving within such environments may also lead to other types of latent outcomes, such as innovative ideas or adaptive expertise, or navigational flexibility (Author, 2014a; Pellegrino, DiBello, & Brophy, 2014; Sawyer & Greeno, 2008; Stevens et al., 2008). However, research about how to assess outcomes attained via participation in informal learning endeavors is nascent. We applied situated learning theory as a theoretical framework (Author, 2014b). We empirically examined student outcomes across six different informal or co-curricular experiences, as aligned with the 11 ABET (2013) student outcomes, outcomes that are critical to students learning to become an engineer (Balascio, 2014). While we report out the results based on the ABET outcomes, the main purposes for this paper were to: (a) examine the nature of the reflective discourse that revealed student self-directed learning of non-cognitive outcomes; (b) investigate the interactive nature of the technical and professional skills outcomes; and (c) explain the methods used to map the reflective discourse to develop assessments of student outcomes.
Methods We used qualitative methods to collect and analyze the data. Then, we applied content and response process validation methods (AERA, APA, NCME, 2014; Messick, 1995). This study took place in two large public universities. Both with multiple engineering majors. Data were collected from undergraduate engineering students participating in several different informal environments, including a engineering competitions, service learning, and a robotics club. Approximately 60 students participated across the events. We conducted focus groups and individual interviews with students participating in the challenge. Focus groups provided insight into the collective learning experiences (Ryan, Gandha, Culbertson, & Carlson, 2014). Sample protocol included open-ended questions, such as: “How would you describe your process?”
Results Preliminary Results indicated the incidence of learning outcomes associated with each of the ABET (2013) student technical skills. In addition, outcomes were inter-related across the ABET criteria. For example, students’ reflections often integrated their learning processes about design, experimentation, and problem-solving. Importantly, students also demonstrated adaptability throughout the competition. Illustrative examples of the mapping of response processes and test content are provided.
Implications This study provided evidence of student learning across the ABET technical and professional skills across a range of informal and co-curricular engineering experiences adding to the growing body of research examining learning in informal environments. More importantly, students engaged adaptive expertise (Walker, Cordray, King, & Brophy, 2006) understanding the need for flexibility and self-evaluation to solve a problem (Zimmerman, 2002). In addition, the study provided examined response processes as a validation method to establish test content.
Bland, L. C., & Kusano, S. M., & Xu, X., & Johri, A. (2017, June), Examining Learner-driven Constructs in Co-curricular Engineering Environments: The Role of Student Reflection in Assessment Development Paper presented at 2017 ASEE Annual Conference & Exposition, Columbus, Ohio. 10.18260/1-2--28315
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2017 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015