June 24, 2017
June 24, 2017
June 28, 2017
Educational Research and Methods
This theory paper describes a process for developing and validating instruments for assessing student motivation--a learning outcome of engineering graduates highly desired by industry. Industry expectations influence ABET criteria and curricula change necessary to ensure that graduate engineers are motivated to meet ever-changing global challenges. Efficient, effective assessment instruments are needed to guide and evaluate efforts in curricular change.
Although a few well-known instruments exist for assessing motivation in the context of student learning, none is designed specifically for use in engineering design projects where students engage in authentic professional learning experiences. Proposed instruments must be designed for efficient use while effectively evaluating and enhancing student learning to ensure wide-spread adoption.
The assessment instrument development process follows the steps described by Liu (2010): 1. Identify assessment purpose and target audience 2. Define construct (topical content and levels) to be measured 3. Prepare a test specification that defines the combination of number and types of items 4. Construct an initial pool of items to be considered 5. Conduct expert review of items proposed 6. Pilot test the instrument with representative students 7. Field test the assessment instrument prototype in actual capstone contexts
The target audience for assessing motivation includes capstone students and instructors and employers of engineering graduates. Assessments provide feedback for improvement, communicate expectations to students and instructors, and provide measures of motivation that are authentic to the engineering profession.
Aspects of motivation were developed based upon goal orientation, task value, self-efficacy, and self-determination theories and models, as well as employer feedback regarding employee motivation. A construct includes three major elements of motivation: attitudes, behaviors, and development. Attitude examples are drawn from capstone instructor experiences. Behavior elements include those observable throughout a capstone design experience. The development element is defined from an affective domain taxonomy for learning.
An important step in the instrument development process is defining user (target audience) needs and design requirements. The authors interviewed ten capstone instructors to derive a needs/requirements matrix to guide the design of assessment instruments and to form the foundation for verification. The team identified test length, test specification, and item format as key requirements for design.
Three assessment instruments (attitude, behavior, and development) are being implemented over the duration of a capstone project. Initial implementation establishes a baseline of attitude followed by intermediate instruments designed with single-point rubrics to prompt self-reflection and planning for change along with peer evaluation using multi-point rubrics. Final implementation provides evidence of students’ motivational changes and documents their understanding and action to manage their motivations.
The motivation assessment instruments have been pilot tested with over 200 students in a diverse set of capstone courses at one institution. The validity of the instruments is discussed in terms of satisfying design requirements. Details are presented for follow-on, multi-institution validation of the instruments.
Rogers, P., & Davis, D. C., & Winfree, S., & Ash, K., & Ibrahim, B., & Ding, L. (2017, June), Early Validation of the Motivation in Team Projects (MTP) Assessment Paper presented at 2017 ASEE Annual Conference & Exposition, Columbus, Ohio. 10.18260/1-2--28193
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2017 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015