Asee peer logo

Analysis of Online Robotics Challenge Submissions - Fundamental

Download Paper |


2021 ASEE Virtual Annual Conference Content Access


Virtual Conference

Publication Date

July 26, 2021

Start Date

July 26, 2021

End Date

July 19, 2022

Conference Session

Pre-College Engineering Education Division Technical Session 11

Tagged Division

Pre-College Engineering Education

Page Count




Permanent URL

Download Count


Request a correction

Paper Authors


Sara Willner-Giwerc Tufts University

visit author page

Sara Willner-Giwerc is a Ph.D. candidate in mechanical engineering at Tufts University. She graduated from Tufts University with a B.S. in mechanical engineering and a double minor in engineering education and engineering management in 2018. She is a National Science Foundation Graduate Research Fellow, which supports her research at the Tufts Center for Engineering Education and Outreach (CEEO) on the design of technological tools, learning experiences, and environments for supporting solution diversity in engineering classrooms pre-k through college.

visit author page


Miela Efraim Tufts University

visit author page

Miela Efraim is a student at Tufts University in the Class of 2024 studying Engineering Psychology. She works as an undergraduate research assistant at the Tufts Center for Engineering Education and Outreach.

visit author page


Chris B. Rogers Tufts University

visit author page

Chris is a professor of Mechanical Engineering at Tufts University with research in engineering education, robotics, musical instrument design, IoT, and anything else that sounds cool.

visit author page

Download Paper |


Our goal in this research project was to see what we could conclude about student creativity and learning from looking at student final artifacts. Typically, researchers examine the entire learning journey of students as they complete an engineering design challenge, but we wanted to see how much we could measure and understand about student learning in a situation when we were not able to capture these journeys. This becomes particularly relevant when thinking about asynchronous, online robotics challenges, where often the students’ problem solving processes are not captured in the submissions. We chose the library of student inventions submitted to Dr. E’s Challenges ( to evaluate because it was an existing, well-defined, public gallery of student work. Professor Ethan (“Dr. E”) Danahy from the Tufts Center for Engineering Education and Outreach (CEEO) released a robotics challenge for students to solve using LEGO Robotics at regular intervals from 2014–2019. The challenge series collected almost 1000 submissions (in the form of images, videos, and short written descriptions) from 57 different countries around the world. These challenges were designed to provide community members (students, teachers, parents, etc.) with an opportunity to create and share solutions to asynchronous online robotics challenges. We conducted a quantitative and qualitative analysis of these submissions in an attempt to discern meaningful indicators of different types of student learning. We were interested in solution diversity (how many different types of submissions were received for each challenge), functionality of solution, complexity of build and code, quality of documentation, use of non-LEGO materials, and variance from given examples. We found that in challenges where examples were given, over 40% of submissions on average for that challenge were a replica of one of the examples provided. Students that instead invented their own solutions tended to have more complex codes and builds. Furthermore, when students invented their own solutions, the length and quality of their documentation increased. Likewise, the 42.6% of submissions that included non-LEGO materials into their build tended to write longer and more descriptive reports to accompany their images and videos. This analysis technique and results have the potential to help online robotics challenge coordinators and creators think critically about the types of submissions they are receiving and the ways that they can improve challenge prompts, supports, and scaffolds for future challenges to elicit engineering thinking.

Willner-Giwerc, S., & Efraim, M., & Rogers, C. B. (2021, July), Analysis of Online Robotics Challenge Submissions - Fundamental Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--36684

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2021 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015