Asee peer logo

Assessing Evolving Conceptual Knowledge in Software Engineering Students

Download Paper |


2012 ASEE Annual Conference & Exposition


San Antonio, Texas

Publication Date

June 10, 2012

Start Date

June 10, 2012

End Date

June 13, 2012



Conference Session

Pedagogical Approaches for Software Engineering

Tagged Division

Software Engineering Constituent Committee

Page Count


Page Numbers

25.213.1 - 25.213.11



Permanent URL

Download Count


Request a correction

Paper Authors


Kevin A. Gary Arizona State University, Polytechnic

visit author page

Kevin Gary, Ph.D., is an Associate Professor in the Department of Engineering within the College of Technology and Innovation at Arizona State University's Polytechnic campus. At ASU, Kevin led the development of the new bachelor's of software engineering program, and the revised design of the master's of computing studies. Gary designed and implemented the Software Enterprise, an NSF-funded hybrid pedagogical method for conducting project-based courses. More than 40 industry projects have been conducted
by the Software Enterprise over the past eight years, and the Enterprise is the foundation of the project spine in the software engineering degree program. In addition to teaching software engineering, Gary also teaches in the areas of web/enterprise systems and database
management systems. In 2009, Gary conducted one of the first Startup Weekend events in a university setting, promoting entrepreneurship and innovation among ASU students. Gary's research has focused primarily on software architectures, agile methods, and open source software for the operating room of the future. In cooperation with the Georgetown University Medical Center, Gary has received funding from the National Institutes of Health to develop the Image-Guided Surgical Toolkit (IGSTK). For his 2011-12 sabbatical appointment, Gary has been working as Chief Software Architect in the Bioengineering Initiative of the Sheik Zayed Center for Pediatric Surgical Innovation at the Children's
National Medical Center in Washington, D.C., in the area of microrobotics for surgical applications. Gary has authored over forty peer-reviewed publications and received funding from the National Science Foundation, the National Institutes of Health, IBM, the Kaufmann Foundation, and the state of Arizona. He is an active member of the ACM, IEEE, and ASEE, and serves on a number of program committees for the community.

visit author page


Yegeneswari Nagappan Unicon, Inc.

visit author page

Yegeneswari Nagappan works as Software Developer at Unicon, Inc. She holds a master's degree in computing studies from Arizona State University.

visit author page


Supreet Verma Delasoft, Inc.

visit author page

Supreet Verma was born and raised in India, mostly lived in Lucknow, capital of Uttar Pradesh (one of the states in India). His father did his bachelor's of science and master's of science in mathematics that influenced me to choose my career in the field of engineering. He has completed senior secondary schooling from City Montessori School in Lucknow and choose science, mathematics, and computers as my main subjects. He cracked IIT-JEE entrance and joined Indian Institute of Technology, Roorkee, India, to do his undergrad in electrical engineering (B.Tech.). In his second year of college, he got more interested in computer science (CS). He took some courses of CS, which his department offered and did his final year project in the same field. To choose this as his career path, he did his internship in a software firm SRM TechSol Pvt, Ltd., Lucknow, India, and joined Sasken Communication Technologies, Ltd., Bangalore, India, after his graduation. To acquire more domain knowledge, he came to Arizona State University as a master’s student in computing studies. He was a research assistant under Kevin A. Gary. Under his guidance, he wrote algorithms and enhanced the existing one for concept map comparison which is used for assessment of software enterprise course. Currently, I’m working as a programmer analyst with Delasoft, Inc. in Newark, Del., developing distributed applications on different platforms (.Net, Java, and Android).

visit author page


Russell J. Branaghan Arizona State University

visit author page

Russell Branaghan is an Assistant Professor of cognitive science and engineering at Arizona State University in Mesa, Ariz. His research interests include measuring the development and refinement of structural knowledge.

visit author page

Download Paper |


Assessing Evolving Conceptual Knowledge in Software Engineering StudentsOver the past decade, the software engineering education community has focused significantlyon defining the body of concepts that both undergraduate and graduate students should (in somesense) “know” and be able to apply upon graduation. Efforts such as the SWEBOK, the SEEK,and the GSwE2009 together with associated certification processes from IEEE (CSDP andCSDA) and ISO (ISO/IEC 24773:2008) go to great lengths to define outcomes primarily ascoverage of the maturing body of knowledge (BOK) in software engineering. The evolution ofthe software engineering BOK and certifications over the past decade+ is a good thing as itspeaks to the maturing of the profession. However, defining program outcomes in terms ofcontent taxonomies flies in the face of emerging trends toward active and discovery-basedteaching and learning techniques (be it called “hands-on”, “project-centric”, or “self-paced”).One criticism of active or discovery-based learning approaches is that they are not the most cost-effective modality for transmitting information. To say it directly, an instructor can cover farmore content in a traditional lecture format that in a project (or other applied) format. In light ofrecent pervasive budget pressures in higher education, the ability to cover more with less formore customers (students) is a practical reality. A second criticism of the active approach is thatindividual assessment is difficult as project work is typically done in teams. Further, it isnecessary for an instructor to continuously monitor class progress as a whole toward learningoutcomes, which is complex in an environment where team progress evolves distinctly andnonlinearly. Teams do not learn at the same rate, go about applied work in the same way, andcome upon the hoped-for “aha” moment at the same time. The authors believe hybridpedagogical approaches such as scaffolding or the Software Enterprise are the best ways toaddress this issue.Is it more important to “collect more knowledge” or to gain an understanding of how thoseconcepts relate so that a sustainable learning structure is formed? Active learning proponentswould suggest it is more important to create a durable conceptual foundation by which newexperiences are assimilated to grow the personal knowledge base in an orderly, well-formedway. But how can one assess that this is happening? In our project-centric courses we haveadopted concept maps as a technique for evaluating the evolution of student understanding ofconceptual knowledge in software engineering. This paper will describe the concept mappingexercise students are asked to conduct at the beginning and end of every semester, the evaluationof concept maps through a unique expert ranking process, and the automation of concept mapevaluation to expert concept maps through graph comparison algorithms. Data collected fromalmost two years’ of concept map practice will be presented and analyzed from the perspectivewhether this is a viable technique in the present context of software engineering education.

Gary, K. A., & Nagappan, Y., & Verma, S., & Branaghan, R. J. (2012, June), Assessing Evolving Conceptual Knowledge in Software Engineering Students Paper presented at 2012 ASEE Annual Conference & Exposition, San Antonio, Texas. 10.18260/1-2--20973

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2012 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015