Asee peer logo

Testing a Reflective Judgement Scale for Suitability with First-Year Student Reflective Responses

Download Paper |


2019 ASEE Annual Conference & Exposition


Tampa, Florida

Publication Date

June 15, 2019

Start Date

June 15, 2019

End Date

June 19, 2019

Conference Session

First-Year Programs: Focusing on Student Success

Tagged Division

First-Year Programs

Page Count




Permanent URL

Download Count


Request a correction

Paper Authors


Natalie C.T. Van Tyne Virginia Tech Orcid 16x16

visit author page

Natalie Van Tyne is an Associate Professor of Practice at Virginia Polytechnic Institute and State University, where she teaches first year engineering design as a foundation courses for Virginia Tech's undergraduate engineering degree programs. She holds bachelors and masters degrees from Rutgers University, Lehigh University and Colorado School of Mines, and studies best practices in pedagogy, reflective learning and critical thinking as aids to enhanced student learning.

visit author page


Lisa D. McNair Virginia Tech

visit author page

Lisa D. McNair is a Professor of Engineering Education at Virginia Tech, where she also serves as Director of the Center for Research in SEAD Education at the Institute for Creativity, Arts, and Technology (ICAT). Her research interests include interdisciplinary collaboration, design education, communication studies, identity theory and reflective practice. Projects supported by the National Science Foundation include exploring disciplines as cultures, liberatory maker spaces, and a RED grant to increase pathways in ECE for the professional formation of engineers.

visit author page

Download Paper |


Testing a Reflective Judgement Scale for Suitability with First Year Student Reflective Responses

This Complete Evidence-Based Practice paper describes the use of reflection in a first-year engineering design course. Reflection is an essential part of learning, but it is not widely used in engineering curricula. Using reflective learning techniques in the classroom can help students develop critical thinking skills, which are highly valued in the modern workplace. Critical thinking consists of an objective analysis and reconstruction of available information, often from multiple sources, before deciding what to accept as valid. While we expect that the ability to think critically develops with practice and time, it would be useful to discern how well our students could learn to reflect and think critically during their first year of college.

Two theoretical models are useful when evaluating student performance on reflective learning tasks. The Perry Model of intellectual development (1996) suggests that students with extensive practice in open-ended problem solving involving reflection will be more successful than their peers. Similarly, the King and Kitchener Reflective Judgement Stages model (1994) contains a scale which is useful for measuring increased complexity of reflective thinking over time, another indicator of future success. Both models emphasize that knowledge is largely contextual, meaning that it is sometimes true, and sometimes not true, depending on the applicable context. Thus, context is another important factor to consider when evaluating students reflective learning performance or ability.

In the present study, we assess the extent to which students use reflective judgement and critical thinking skills when writing about their experiences in a semester long introductory design course. Based on our theoretical models, we hypothesized that students whose reflections acknowledge the role of context in learning will additionally show more critical thinking development over the course of the semester, as well as earn a higher grade on successive assignments. Therefore, we wanted to determine whether students’ ability to reflect on their experiences in the course improved over time.

Participants were approximately 100 first year engineering students who completed a series of five graded short essays. Four essays incorporated reflection about a specific three-week period in the course, while the fifth essay prompted reflection about the entire course content. In each case, students answered three questions whereby they identified the most important item of knowledge that they learned, why it was important, and where else they could use this knowledge, outside of the course. Assignment prompts also stipulated that reflection was to extend beyond the specific needs of the course, such as contributions to homework assignments.

We operationalized the ability to exercise critical thinking by reflecting on multiple sources of knowledge and recognizing alternative contexts by using the Perry Model and King and Kitchener’s reflective judgement stages in parallel with an internally-developed grading rubric. Rubric criteria included the level of relevant detail in describing items of knowledge and their context, and the application of specific knowledge to aspects of lifelong learning. By comparing the grades for each assignment over time, in conjunction with corresponding Perry Model and King and Kitchener stages, we expect to find a mixture of continual, sporadic or lack of progress over time. These results will help further develop this course in the future and can provide new insights as to how students view the importance of reflection and critical thinking for their learning.

Van Tyne, N. C., & McNair, L. D. (2019, June), Testing a Reflective Judgement Scale for Suitability with First-Year Student Reflective Responses Paper presented at 2019 ASEE Annual Conference & Exposition , Tampa, Florida. 10.18260/1-2--33364

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2019 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015