Asee peer logo

Instrumentation for Evaluating Design-learning and Instruction Within Courses and Across Programs

Download Paper |


2021 ASEE Virtual Annual Conference Content Access


Virtual Conference

Publication Date

July 26, 2021

Start Date

July 26, 2021

End Date

July 19, 2022

Conference Session

Faculty Perspectives of Active Learning, Inequity, and Curricular Change

Tagged Division

Educational Research and Methods

Tagged Topic


Page Count


Permanent URL

Download Count


Request a correction

Paper Authors

author page

Steven Santana Harvey Mudd College

Download Paper |


This work in progress describes an instrumentation plan, applied through action research, to evaluate student learning of engineering design. The instruments described herein will be applied to an entry-level design course. This course situates teams as engineers contracted to work on a project provided by an external client. The instructors solicit, select, and refine projects that present engineering problems requiring conceptual design and tangible, mechanical solutions. In this course, the overarching learning outcome is to implement and practice a design process to identify, frame, and solve open-ended and ill-structured engineering problems. This outcome emphasizes the inherent complexity, ambiguity, and nonlinearity of the design process and is, as a result, a challenge to teach and difficult to learn. Students often find the reality of engineering design overwhelming the first time they experience it. Instructor observations indicate that students proceed through the design process in a linear fashion, spend insufficient time scoping problems, and minimally implement design activities to advance their work and learning. This trend is well documented. A primary learning outcome of this course is for students to internalize the design process as a set of activities that engineers deploy with adaptive expertise to create high-quality deliverables that satisfy needs while developing their identity as engineers. Thus, this work seeks to develop a mixed-methods, instrument suite to (1) assess students implementation of design activities within authentic design projects, (2) evaluate final deliverable quality, (3) survey students’ engineering design abilities, and (4) record students’ evolving engineering identity and sense of belonging in the engineering community. Future work includes developing prototypes of the aforementioned instruments and assessing their validity and reliability. To quantify design activities, this instrument includes design diaries to capture engagement with design phases and activities. To assess the quality of final designs, this suite includes a rubric with the goal of connecting processes to outcomes. To measure students’ perceptions of their abilities, engineering identity, and sense of belonging within engineering this work includes surveys to capture development, on an individual level, in each of these areas. This work will extend prior scholarship by lengthening the timeline of instrument use to a nearly semester-long project; by applying the instrument to a student team’s deliverable, as opposed to an individual’s work; and, by using a standard rubric to assess unique designs created by students within similar settings. Once developed and validated, these combined measures will be deployed to understand students’ learning as a function of course pedagogy, activities, structures, and climate and to iteratively improve student outcomes and instruction. A future goal is to apply this tool and analysis framework to improve instruction across courses and programs, wherein design is, or could be, central to student learning. This work is done in service to equipping engineering graduates with expertise, fluency, and with the capacity to lead in engineering and design.

Santana, S. (2021, July), Instrumentation for Evaluating Design-learning and Instruction Within Courses and Across Programs Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference.

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2021 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015