Education and Evaluation for the NRT: Accounting for Numerous Requirements, Multiple Disciplines, and Small Cohorts

Nathalie Duval-Couetil; Soohyun Yi

Download Paper | Permalink

Conference: 2021 ASEE Virtual Annual Conference Content Access
Location: Virtual Conference
Publication Date: July 26, 2021
Start Date: July 26, 2021
End Date: July 19, 2022
Conference Session: Graduate Studies Division Technical Session 2
Tagged Division: Graduate Studies
Tagged Topic: Diversity
Page Count: 20
DOI: 10.18260/1-2--37001
Permanent URL: https://peer.asee.org/37001
Download Count: 464

Request a correction

Paper Authors

biography

Nathalie Duval-Couetil Purdue University at West Lafayette orcid.org/0000-0002-0260-0208

visit author page

Nathalie Duval-Couetil is the Director of the Certificate in Entrepreneurship and Innovation Program, Associate Director of the Burton D. Morgan Center, and a Professor in the Department of Technology Leadership and Innovation at Purdue University. She is responsible for the launch and development of the university’s multidisciplinary undergraduate entrepreneurship program, which involves 2000 students from all majors per year. She has established entrepreneurship capstone, global entrepreneurship, and women and leadership courses and initiatives at both the undergraduate and graduate levels. Prior to her work in academia, Nathalie spent several years in the field of market research and business strategy consulting in Europe and the United States with Booz Allen and Hamilton and Data and Strategies Group. She received a BA from the University of Massachusetts at Amherst, an MBA from Babson College, and MS and PhD degrees from Purdue University.

visit author page

biography

Soohyun Yi Texas Tech University

visit author page

Soohyun Yi is an Assistant Professor of Educational Psychology at Texas Tech University. She received her Ph.D. in Educational Studies from Purdue University and M.A. in Psychological Measurement from Ewha Womans University. Her scholarly trajectory aims to improve education for underserved and underchallenged students with impactful research and evidence-based interventions. Longitudinal research methodology is the main area of her expertise, which has enabled her a) to investigate growth trajectories of motivation and career choices; b) to identify opportunity gaps within underserved groups; and c) to evaluate and improve educational interventions in STEM. With the expertise in quantitative research methodology, she is engaged in collaborative research with entrepreneurship education and other interdisciplinary programs.

visit author page

Download Paper | Permalink

Abstract

The purpose of this paper is to describe our approach to program evaluation for the National Science Foundation National (NSF) Research Traineeship (NRT) Program in Sustainable Food, Energy, and Water Systems at X University. The NRT program is designed to educate and train next generation of engineers, agronomists and scientists to meet local food, energy and water management needs with solar energy. This requires educating interdisciplinary cohorts of graduate students in basic scientific and technical principles to enable them to develop economically viable, innovative, interdisciplinary solutions. This is achieved through many interdisciplinary courses, research experiences and professional development requirements.

Assessment and evaluation activities, designed to measure impacts on training and career development, are important aspects of this work, but require significant attention to capture the range of activities undertaken by very small cohorts of interdisciplinary students and faculty. Our goal was to develop “sustainable” evaluation activities given our observation that programs often begin with very ambitious assessment and data collection goals in the first year, that diminish over time. To do so, we addressed challenges associated with startup and operational sustainability, such as:

• Tracking individual graduate student participation in various NRT program requirements can be labor intensive and is often done manually (e.g., combinations of disciplinary courses, interdisciplinary courses, professional development modules, and scholarly activities). • There are few existing validated instruments for assessing interdisciplinary training programs, and limited financial support for PI/co-PI involvement in instrument development. • While external evaluators serve an important role in NSF training grants, when programs are in startup mode, they are often far removed from subject matter experts and programmatic offerings. • Response rates can be low when busy students are expected to complete course and program surveys that go beyond evaluations mandated through the university.

To overcome these challenges, we optimized evaluation activities to increase compliance and effectiveness through several activities that will be discussed in this paper. These include the creation of a professional development tracking “hub” within our university’s learning management system (LMS); the development and dissemination of an annual survey to track various learning and career outcomes over time; interdisciplinary course evaluations; and annual focus group interviews. This paper will report on the rationale for and results of these education and evaluation activities.

Citation
Format

Duval-Couetil, N., & Yi, S. (2021, July), Education and Evaluation for the NRT: Accounting for Numerous Requirements, Multiple Disciplines, and Small Cohorts Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--37001

TY  - CPAPER
AB  - The purpose of this paper is to describe our approach to program evaluation for the National Science Foundation National (NSF) Research Traineeship (NRT) Program in Sustainable Food, Energy, and Water Systems at X University. The NRT program is designed to educate and train next generation of engineers, agronomists and scientists to meet local food, energy and water management needs with solar energy. This requires educating interdisciplinary cohorts of graduate students in basic scientific and technical principles to enable them to develop economically viable, innovative, interdisciplinary solutions. This is achieved through many interdisciplinary courses, research experiences and professional development requirements.

Assessment and evaluation activities, designed to measure impacts on training and career development, are important aspects of this work, but require significant attention to capture the range of activities undertaken by very small cohorts of interdisciplinary students and faculty. Our goal was to develop “sustainable” evaluation activities given our observation that programs often begin with very ambitious assessment and data collection goals in the first year, that diminish over time. To do so, we addressed challenges associated with startup and operational sustainability, such as:

•	Tracking individual graduate student participation in various NRT program requirements can be labor intensive and is often done manually (e.g., combinations of disciplinary courses, interdisciplinary courses, professional development modules, and scholarly activities). 
•	There are few existing validated instruments for assessing interdisciplinary training programs, and limited financial support for PI/co-PI involvement in instrument development. 
•	While external evaluators serve an important role in NSF training grants, when programs are in startup mode, they are often far removed from subject matter experts and programmatic offerings.  
•	Response rates can be low when busy students are expected to complete course and program surveys that go beyond evaluations mandated through the university. 

To overcome these challenges, we optimized evaluation activities to increase compliance and effectiveness through several activities that will be discussed in this paper. These include the creation of a professional development tracking “hub” within our university’s learning management system (LMS); the development and dissemination of an annual survey to track various learning and career outcomes over time; interdisciplinary course evaluations; and annual focus group interviews. This paper will report on the rationale for and results of these education and evaluation activities. 

AU  - Nathalie Duval-Couetil
AU  - Soohyun Yi
CY  - Virtual Conference
DA  - 2021/07/26
PB  - ASEE Conferences
TI  - Education and Evaluation for the NRT: Accounting for Numerous Requirements, Multiple Disciplines, and Small Cohorts
UR  - https://peer.asee.org/37001
DO  - 10.18260/1-2--37001
ER  -