Early Validation of the Motivation in Team Projects (MTP) Assessment
- Conference
- Location
-
Columbus, Ohio
- Publication Date
-
June 24, 2017
- Start Date
-
June 24, 2017
- End Date
-
June 28, 2017
- Conference Session
- Tagged Division
-
Educational Research and Methods
- Page Count
-
22
- DOI
-
10.18260/1-2--28193
- Permanent URL
-
https://peer.asee.org/28193
- Download Count
-
680
Paper Authors
Peter Rogers The Ohio State University
Dr. Peter Rogers is a Professor of Practice in the Department of Engineering Education at The Ohio State University. He joined the university in October 2008 bringing with him 35 years of industry experience. His career includes senior leadership roles in engineering, sales, and manufacturing developing products using multidisciplinary teams to convert customer needs to commercially viable products and services.
Rogers co-led the development of an ABET-approved year-long Capstone design experience. With a focus on providing students with a broader experience base, the multidisciplinary program applies teams of engineers, business, design, and other students to work with companies to help them be more competitive. Rogers expanded this one-year program to a four-year Integrated Business and Engineering (IBE) honors program.
Rogers earned his Ph.D. at the University of Massachusetts, Amherst, focused on mechanical engineering and manufacturing
Denny C. Davis The Ohio State University
Dr. Davis is Visiting Professor in the Engineering Education Department at The Ohio State University and Emeritus Professor of Engineering Education at Washington State University. For three decades, he taught engineering design and led multi-institution teams in the development and testing of curriculum materials and assessments for engineering design learning. He is also the owner of Verity Design Learning LLC, a publisher of instructional materials for design reviews and teamwork development. He is a Fellow of the American Society for Engineering Education. Dr. Davis received his PhD in Agricultural Engineering at Cornell University.
Sarah Winfree The Ohio State University
Sarah Winfree is an undergraduate research assistant in the Department of Engineering Education at The Ohio State University. She joined the University in August 2013 working towards a Bachelor of Science degree in Food Engineering. Her career includes motivating students on how to succeed in academia and working on product development at Vesco Medical. She joined the research team in December 2015 and is currently working on assessing how motivation plays a role in student success in academia and industry.
Kaycee Ash The Ohio State University
Kaycee Ash is a Chemical Engineering undergraduate student at The Ohio State University. She started working towards her Bachelor’s of Science degree in August 2014. In the past, she has worked with other students by helping them better themselves academically, and she has worked on chemical manufacturing and transport at Ohio Chemical Services. She joined the research team in December of 2015 and is currently working on assessing motivation in academia.
Bashirah Ibrahim Ohio State University
Bashirah Ibrahim is a postdoctoral researcher in physics education in the Department of Teaching and Learning at the Ohio State University. She has interdisciplinary research experience ranging from physics education, science education to teacher education. Her research interests include problem solving, the role of visualization in the teaching and learning of physics and scientific reasoning. She has co-authored publications in peer reviewed conference proceedings and journals such as the International Journal of Science Education, Journal of Research in Science Teaching, Physical Review Physics Education Research and the African Journal of Research in Mathematics, Science and Technology Education. She is also a referee for various international journals and conferences.
Prior to joining the Ohio State University, Bashirah Ibrahim worked at Kansas State University and the Marang Centre for Mathematics and Science Education, University of Witwatersrand in South Africa.
Lin Ding The Ohio State University
Lin Ding, Ph. D.
Associate Professor
Department of Teaching and Learning
The Ohio State University
Abstract
This theory paper describes a process for developing and validating instruments for assessing student motivation--a learning outcome of engineering graduates highly desired by industry. Industry expectations influence ABET criteria and curricula change necessary to ensure that graduate engineers are motivated to meet ever-changing global challenges. Efficient, effective assessment instruments are needed to guide and evaluate efforts in curricular change.
Although a few well-known instruments exist for assessing motivation in the context of student learning, none is designed specifically for use in engineering design projects where students engage in authentic professional learning experiences. Proposed instruments must be designed for efficient use while effectively evaluating and enhancing student learning to ensure wide-spread adoption.
The assessment instrument development process follows the steps described by Liu (2010): 1. Identify assessment purpose and target audience 2. Define construct (topical content and levels) to be measured 3. Prepare a test specification that defines the combination of number and types of items 4. Construct an initial pool of items to be considered 5. Conduct expert review of items proposed 6. Pilot test the instrument with representative students 7. Field test the assessment instrument prototype in actual capstone contexts
The target audience for assessing motivation includes capstone students and instructors and employers of engineering graduates. Assessments provide feedback for improvement, communicate expectations to students and instructors, and provide measures of motivation that are authentic to the engineering profession.
Aspects of motivation were developed based upon goal orientation, task value, self-efficacy, and self-determination theories and models, as well as employer feedback regarding employee motivation. A construct includes three major elements of motivation: attitudes, behaviors, and development. Attitude examples are drawn from capstone instructor experiences. Behavior elements include those observable throughout a capstone design experience. The development element is defined from an affective domain taxonomy for learning.
An important step in the instrument development process is defining user (target audience) needs and design requirements. The authors interviewed ten capstone instructors to derive a needs/requirements matrix to guide the design of assessment instruments and to form the foundation for verification. The team identified test length, test specification, and item format as key requirements for design.
Three assessment instruments (attitude, behavior, and development) are being implemented over the duration of a capstone project. Initial implementation establishes a baseline of attitude followed by intermediate instruments designed with single-point rubrics to prompt self-reflection and planning for change along with peer evaluation using multi-point rubrics. Final implementation provides evidence of students’ motivational changes and documents their understanding and action to manage their motivations.
The motivation assessment instruments have been pilot tested with over 200 students in a diverse set of capstone courses at one institution. The validity of the instruments is discussed in terms of satisfying design requirements. Details are presented for follow-on, multi-institution validation of the instruments.
Rogers, P., & Davis, D. C., & Winfree, S., & Ash, K., & Ibrahim, B., & Ding, L. (2017, June), Early Validation of the Motivation in Team Projects (MTP) Assessment Paper presented at 2017 ASEE Annual Conference & Exposition, Columbus, Ohio. 10.18260/1-2--28193
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2017 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015