June 14, 2015
June 14, 2015
June 17, 2015
Computers in Education
26.485.1 - 26.485.11
Developing a Learning Analytics Dashboard for Undergraduate Engineering Using Participatory DesignA convergence of pressures has led researchers to seek innovative ways to measure and trackstudent learning outcomes and empirically identify the conditions that lead to their development.Learning analytics is an emerging field of inquiry that uses existing student traces to aggregateand illuminate student data through visualizations and dashboards in an attempt to improvelearning outcomes. Though there are currently efforts both in vendor and academic arenas to tryto understand the long-term learning and decision-making effects of such dashboards, thereappears to be a missed opportunity in the development of these dashboards in vivo using human-centered usability practices to develop these new tools for learning. Practices that select relevantdata traces and develop dashboards with learners instead of for learners may lead to strongerstudent self-efficacy, build on existing social learning theory, and benefit from perspectivesfound within human-centered design practices.Our interdisciplinary team of faculty and graduate students from engineering education,computer science, human computer interaction, human centered design, the learning sciences,and visual communications are following a mixed-methods, human-centered approach todashboard development that breaks new ground in learning analytics by involving the end usersthroughout the design and development process. In this paper we report findings from aparticipatory design session held with a group of eight engineering students enrolled in a firstyear general engineering course. The session’s protocol was organized to gather the followinginformation from the participants: 1) defining success as a university student, 2) identifyingpotential data streams and information, 3) usefulness of peer benchmarking data, 4) credibilityand ethical issues with learning analytics, and 5) students’ use of technology for learning.During the last third of the session, students split into teams and produced designs of a learningdashboard. The entire session was transcribed and coded by each member of our research teamusing NVivo, a qualitative data analysis software package.Results identified the features of learning dashboards that students deem necessary to spur theirinterest and engagement with the systems. Students repeatedly pointed to time elements as beingan important characteristic of a dashboard. They wanted to know how their personal time ontask related to their classmates’ time on task and saw value in incorporating scheduling or timemanagement into the dashboard. As these first year students were still early in their programsand adjusting to life on their own, they also pointed to how such a dashboard especially couldhelp with that transition, perhaps more so than with their in-class learning. Students alsodescribed various opt-in scenarios in which they would feel comfortable with the institutionutilizing their individual data in a dashboard learning environment. Findings from this work willinform future participatory design sessions with students and faculty, help our team developinitial wireframes of a student-driven dashboard, and help determine the data traces that willreceive our focus as we carry out learning analytics quantitative modeling.
Knight, D. B., & Brozina, C., & Stauffer, E. M., & Frisina, C., & Abel, T. D. (2015, June), Developing a Learning Analytics Dashboard for Undergraduate Engineering Using Participatory Design Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.23824
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2015 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015