Asee peer logo

Epistemic Network Analysis as a Tool for Engineering Design Assessment

Download Paper |


2015 ASEE Annual Conference & Exposition


Seattle, Washington

Publication Date

June 14, 2015

Start Date

June 14, 2015

End Date

June 17, 2015





Conference Session

Assessment and Outcomes: ERM Roundtable

Tagged Division

Educational Research and Methods

Page Count


Page Numbers

26.679.1 - 26.679.19



Permanent URL

Download Count


Request a correction

Paper Authors


Golnaz Arastoopour University of Wisconsin, Madison

visit author page

Before becoming interested in education, Golnaz Arastoopour studied Mechanical Engineering at the University of Illinois at Urbana-Champaign with a minor in Spanish. While earning her Bachelor’s degree in engineering, she worked as a computer science instructor at Campus Middle School for Girls in Urbana, Ill. Along with a team of undergraduates, she headlined a project to develop a unique computer science curriculum for middle school students. She then earned her M.A. in mathematics education at Columbia University. Afterwards, she taught in the Chicago Public School system at Orr Academy High School (an AUSL school) for two years. Currently, Golnaz is working with the Epistemic Games Research Group at the University of Wisconsin-Madison where she has led the efforts on engineering virtual internship simulations for high school and first year undergraduate students. Golnaz's current research is focused on how games and simulations increase student engagement in STEM fields, how players learn engineering design in real-world and virtual professional environments, and how to assess engineering design thinking.

visit author page


Naomi C. Chesler University of Wisconsin, Madison

visit author page

Naomi C. Chesler is Professor and Vice Chair of Biomedical Engineering with an affiliate appointment in Educational Psychology. Her research interests include vascular biomechanics, hemodynamics and cardiac function as well as the factors that motivate students to pursue and persist in engineering careers, with a focus on women and under-represented minorities.

visit author page

author page

David Williamson Shaffer


Zachari Swiecki University of Madison-Wisconsin

visit author page

Graduate student in educational psychology, learning sciences area.

visit author page

Download Paper |


Using Epistemic Network Analysis as a Tool for Measuring Engineering Design ThinkingDesign is a central activity in engineering [1], [2]. Engineers identify problems and use theirtechnical knowledge to design a solution for a client. Doing so is an iterative process ofquantified choice under constraint. To prepare future generations of engineers to be innovativedesigners, educators and researchers must fundamentally understand the complex nature ofdesign thinking in the context of real-world engineering. In recent decades, many models fordescribing the engineering design process have been developed ranging from linear [3] toiterative [4] and from prescriptive [5] to heuristic [6]. Critical stages in the design processinclude brainstorming, generating alternative designs, conducting research [4], finding anindependent design [7], and creating ranking matrices or objective trees [8].While these models give educators rough blueprints for design activities, they do not provideinformation about the complex nature of design thinking and the way that this thinking develops.The development of design thinking requires an understanding of technical knowledge and skills,as well as the intellectual warrants that validate activity [9]. More important, it does not meanjust acquiring these problem solving elements, but about understanding the relationships amongthem. In this view, complex design thinking can be thought of in terms of connections thatengineers make as they are designing: which values to consider before taking a certain action orwhat knowledge to gather before making a particular decision. This type of connected thinkinghas been generalized to other professions as well, which Shaffer [10] has operationalized interms of an epistemic frame—a way that professionals connect skills, values, identities,knowledge, and epistemologies to solve complex problems in the field. In turn, we callconnected engineering design thinking, epistemic design thinking.To assess engineering students’ epistemic design thinking, we can use epistemic networkanalysis (ENA). Originally designed to measure the connections between elements in epistemicframes [11], [12], ENA has the capability of measuring connections in discourse and actions asthey manifest over time. Since we describe epistemic design thinking as connected learning anditeration over time, ENA may be an effective way to quantify the engineering design process asnetwork of connections and in turn, be an assessment tool for engineering design learning.In this paper, we will describe the method of ENA and provide a case study of its usage as anengineering design assessment tool. First, we will collect discourse and engineering notebookdata from an engineering virtual internship program in which students participate in solving anauthentic engineering design problem [13]. Then, we will assess design thinking by (1) applyinga traditional design rubric used in an introductory engineering course and (2) modeling networksof engineering design thinking over time using ENA. We will then correlate both conditions withthe quality of students’ design products and determine which method of assessment is a betterpredictor of high quality engineering design solutions.References[1] C. L. Dym, A. Agogino, O. Eris, D. Frey, and L. Leifer, “Engineering design thinking, teaching and learning,” J. Eng. Educ., vol. 94, no. 1, pp. 103–120, 2005.[2] H. A. Simon, The sciences of the artificial, 3rd ed. Cambridge, MA: MIT Press, 1996.[3] M. French, Form, Structure, and Mechanism. London, UK: Macmillan, 1992.[4] N. Cross, Engineering Design Methods: Strategies for Product Design, 2nd ed. Chichester, UK: John Wiley & Sons, 1994.[5] G. Pahl and W. Beitz, Engineering Design, Section Ed. London, UK: Springer-Verlag, 1996.[6] C. L. Dym and P. Little, Engineering Design: A Project-Based Introduction, 2nd ed. Hoboken, NJ: Wiley, 2003.[7] N. P. Suh, The Principles of Design. New York, NY: Oxford University Press, 1990.[8] C. L. Dym, Engineering Design A Synthesis of Views. Cambridge, MA: Cambridge University Press, 1994.[9] D. W. Shaffer, “Learning in Design,” in Foundations for the Future in Mathematics Education, R. A. Lesh, J. J. Kaput, and E. Hamilton, Eds. Mahweh, NJ: Lawrence Erlbaum Associates, 2007, pp. 99–126.[10] D. W. Shaffer, How Computer Games Help Children Learn. New York: Palgrave, 2007.[11] G. Arastoopour, N. C. Chesler, and D. W. Shaffer, “Epistemic persistence: A simulation- based approach to increasing participation of women in engineering,” J. Women Minor. Sci. Eng., vol. 20, no. 3, pp. 211–234, 2014.[12] D. W. Shaffer, D. Hatfield, G. Svarovsky, P. Nash, A. Nulty, E. A. Bagley, K. Franke, A. Rupp, and R. Mislevy, “Epistemic Network Analysis: A prototype for 21st century assessement of learning,” Int. J. Learn. Media, vol. 1, no. 1, pp. 1–21, 2009.[13] N. C. Chesler, G. Arastoopour, C. M. D’Angelo, E. A. Bagley, and D. W. Shaffer, “Design of professional practice simulator for educating and motivating first-year engineering students.,” Adv. Eng. Educ., 2012.

Arastoopour, G., & Chesler, N. C., & Shaffer, D. W., & Swiecki, Z. (2015, June), Epistemic Network Analysis as a Tool for Engineering Design Assessment Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.24016

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015