Asee peer logo

Developing a Learning Analytics Dashboard for Undergraduate Engineering Using Participatory Design

Download Paper |

Conference

2015 ASEE Annual Conference & Exposition

Location

Seattle, Washington

Publication Date

June 14, 2015

Start Date

June 14, 2015

End Date

June 17, 2015

ISBN

978-0-692-50180-1

ISSN

2153-5965

Conference Session

Data Analysis and Assessment

Tagged Division

Computers in Education

Page Count

11

Page Numbers

26.485.1 - 26.485.11

DOI

10.18260/p.23824

Permanent URL

https://peer.asee.org/23824

Download Count

839

Paper Authors

biography

David B. Knight Virginia Tech, Department of Engineering Education Orcid 16x16 orcid.org/0000-0003-4576-2490

visit author page

David Knight is an Assistant Professor in the Department of Engineering Education and affiliate faculty with the Higher Education Program, Center for Human-Computer Interaction, and Human-Centered Design Program. His research focuses on student learning outcomes in undergraduate engineering, learning analytics approaches to improve educational practices and policies, interdisciplinary teaching and learning, organizational change in colleges and universities, and international issues in higher education.

visit author page

biography

Cory Brozina Virginia Tech

visit author page

Cory Brozina is a PhD Candidate in Engineering Education at Virginia Tech. He has his B.S. and M.S. in Industrial & Systems Engineering also from Virginia Tech. His research interests are in Learning Analytics, Engineering Education Assessment, and Educational Technology.

visit author page

author page

Eric M. Stauffer Virginia Tech

biography

Chris Frisina Virginia Tech

visit author page

Ph.D. student at Virginia Tech

visit author page

biography

Troy D. Abel Virginia Tech

visit author page

Dr. Troy Abel received his Ph.D. in Human Computer Interaction and MFA in Graphic Design from Iowa State University. He is currently an Assistant Professor of Visual Communication Design at Virginia Tech and is also the director of the new Eye Tracking and Usability Testing lab (ETUT) at VT. He is affiliated with the Center for Human Computer Interaction and the Institute for Creativity in Art and Technology, as a participating faculty member and researcher. His current research areas investigate the intersection of perception and usability evaluation methodologies utilizing multiple data-streams including eye tracking, qualitative data and facial reading. Additionally his lab is exploring new participatory design methods within Human Centered Design which create novel inter-disciplinary knowledge. His lab is currently developing an instrument system designers can utilize during the iterative phases of design to assure positive affective interactions as well as exploring data visualizations of usability data.

visit author page

Download Paper |

Abstract

Developing a Learning Analytics Dashboard for Undergraduate Engineering Using Participatory DesignA convergence of pressures has led researchers to seek innovative ways to measure and trackstudent learning outcomes and empirically identify the conditions that lead to their development.Learning analytics is an emerging field of inquiry that uses existing student traces to aggregateand illuminate student data through visualizations and dashboards in an attempt to improvelearning outcomes. Though there are currently efforts both in vendor and academic arenas to tryto understand the long-term learning and decision-making effects of such dashboards, thereappears to be a missed opportunity in the development of these dashboards in vivo using human-centered usability practices to develop these new tools for learning. Practices that select relevantdata traces and develop dashboards with learners instead of for learners may lead to strongerstudent self-efficacy, build on existing social learning theory, and benefit from perspectivesfound within human-centered design practices.Our interdisciplinary team of faculty and graduate students from engineering education,computer science, human computer interaction, human centered design, the learning sciences,and visual communications are following a mixed-methods, human-centered approach todashboard development that breaks new ground in learning analytics by involving the end usersthroughout the design and development process. In this paper we report findings from aparticipatory design session held with a group of eight engineering students enrolled in a firstyear general engineering course. The session’s protocol was organized to gather the followinginformation from the participants: 1) defining success as a university student, 2) identifyingpotential data streams and information, 3) usefulness of peer benchmarking data, 4) credibilityand ethical issues with learning analytics, and 5) students’ use of technology for learning.During the last third of the session, students split into teams and produced designs of a learningdashboard. The entire session was transcribed and coded by each member of our research teamusing NVivo, a qualitative data analysis software package.Results identified the features of learning dashboards that students deem necessary to spur theirinterest and engagement with the systems. Students repeatedly pointed to time elements as beingan important characteristic of a dashboard. They wanted to know how their personal time ontask related to their classmates’ time on task and saw value in incorporating scheduling or timemanagement into the dashboard. As these first year students were still early in their programsand adjusting to life on their own, they also pointed to how such a dashboard especially couldhelp with that transition, perhaps more so than with their in-class learning. Students alsodescribed various opt-in scenarios in which they would feel comfortable with the institutionutilizing their individual data in a dashboard learning environment. Findings from this work willinform future participatory design sessions with students and faculty, help our team developinitial wireframes of a student-driven dashboard, and help determine the data traces that willreceive our focus as we carry out learning analytics quantitative modeling.

Knight, D. B., & Brozina, C., & Stauffer, E. M., & Frisina, C., & Abel, T. D. (2015, June), Developing a Learning Analytics Dashboard for Undergraduate Engineering Using Participatory Design Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.23824

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015