Asee peer logo

Assessment of Project-Based Learning Courses Using Crowd Signals

Download Paper |


2019 ASEE Annual Conference & Exposition


Tampa, Florida

Publication Date

June 15, 2019

Start Date

June 15, 2019

End Date

October 19, 2019

Conference Session

SED Technical Session: Applications

Tagged Division

Systems Engineering

Tagged Topic


Page Count




Permanent URL

Download Count


Request a correction

Paper Authors


Georgios Georgalis Purdue University-Main Campus, West Lafayette (College of Engineering)

visit author page

Georgios is currently a Ph.D. candidate in the School of Aeronautics and Astronautics at Purdue University and has completed his undergraduate degree at the National Technical University of Athens (NTUA). His research concentrates on a new approach to project risk assessment that is human-centric and allows for prediction of upcoming failures, which gives practitioners the opportunity to prevent them.

visit author page


Karen Marais Purdue University-Main Campus, West Lafayette (College of Engineering)

visit author page

Dr. Karen Marais' educational research focuses on improving systems engineering education. She is the author of several technical publications, including 17 journal papers and two book chapters. She received an NSF CAREER award in 2014. Dr. Marais has worked in engineering for two decades, first in industry and then in academia. She holds a B. Eng. in Electrical and Electronic Engineering from the University of Stellenbosch, a B.Sc. in Mathematics from the University of South Africa, and an S.M and Ph.D. from the Department of Aeronautics and Astronautics at MIT.

visit author page

Download Paper |


Project-based learning (PBL) is a growing component of engineering education in the United States. Its perceived educational value is exemplified by its explicit mention in ABET’s Criterion 5, which requires engineering programs to provide a culminating design experience that incorporates engineering standards and multiple constraints. Capstone courses and design-build-test projects allow students to synthesize and apply engineering knowledge, skills, and tools to open-ended design problems. Students work and communicate in teams to complete tasks like generating requirements, and testing and integrating equipment. There appears to be widespread consensus that project-based learning is valuable, but, how well do these projects prepare students for engineering challenges in professional practice? We consider one aspect of professional practice—failure. Despite many improvement efforts by organizations, systems engineering failures continue to occur. Previous research identified a set of common causes for these project failures. Does PBL provide students with opportunities to fail safely, and thereby learn to avoid failure in professional practice? We present here an approach to compare the rates of occurrence of failure causes in student team projects with industry projects. By comparing the occurrence rates, we achieve our first research goal to evaluate whether PBL offers sufficient opportunities of failure to students. Out of the ten failure causes we examined, we found that four are underrepresented in PBL, two of which are fundamentally related to systems engineering practice. Failure causes may be hard to identify without the benefit of hindsight, so, we developed a set of crowd signals that may point to their presence. We observed 18 different student design projects across two semesters. Each week, the students answered a set of questions we developed to measure these crowd signals, while the instructors directly pointed out any instances of the failure causes they observed during the students’ efforts. With the availability of such data we built logistic regression models to find correlations between specific crowd signals and the occurrence of a particular failure cause. By interpreting the regression coefficients, we achieve our second research goal to suggest specific improvements that instructors can use to give their students more failure opportunities during PBL.

Georgalis, G., & Marais, K. (2019, June), Assessment of Project-Based Learning Courses Using Crowd Signals Paper presented at 2019 ASEE Annual Conference & Exposition , Tampa, Florida. 10.18260/1-2--32129

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2019 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015