Asee peer logo

Weighted Social Tagging as a Research Methodology for Determining Systemic Trends in Engineering Education Research

Download Paper |


2011 ASEE Annual Conference & Exposition


Vancouver, BC

Publication Date

June 26, 2011

Start Date

June 26, 2011

End Date

June 29, 2011



Conference Session

Knowing Ourselves: Research on Engineering Education Researchers

Tagged Division

Educational Research and Methods

Page Count


Page Numbers

22.1675.1 - 22.1675.20

Permanent URL

Download Count


Request a correction

Paper Authors


Xin (Cindy) Chen Purdue University

visit author page

Xin (Cindy) Chen is currently a Ph.D. student in School of Engineering Education at Purdue University. Her research focuses on the influences of modern technologies on science and engineering education, including science and engineering virtual organizations, mobile devices and social media.

visit author page


Nikitha Sambamurthy Purdue University

visit author page

Nikitha Sambamurthy is a Ph.D. student in the School of Engineering Education at Purdue University.

visit author page


Corey M. Schimpf Purdue University, West Lafayette

visit author page

Corey Schimpf is a Ph.D. student in Engineering Education at Purdue University. He is interested understanding how students use and interact with technology in order to improve its deployment in and for the classroom.

visit author page

author page

Hanjun Xian Purdue University, West Lafayette


Krishna Madhavan Purdue University, West Lafayette

visit author page

Dr. Krishna P.C. Madhavan is an Assistant Professor in the School of Engineering Education at Purdue University. He is also a member of the Education Research Team of the NSF-funded Network for Computational Nanotechnology ( Prior to his arrival at Purdue, he was an Assistant Professor with a joint appointment in the School of Computing and the Department of Engineering and Science Education at Clemson University. Dr. Madhavan also served as a Research Scientist at the Rosen Center for Advanced Computing, Information Technology at Purdue University where he led the education and the educational technology effort for the NSF-funded Network for Computational Nanotechnology (NCN). His work focuses on how semantic grid-based technologies and tools can co-exist with students’ lifestyles, learning patterns, and technology choices. Dr. Madhavan was the Chair of the IEEE/ACM Supercomputing Education Program 2006 and was the curriculum director for the Supercomputing Education Program 2005. In 2008, he was awarded the NSF CAREER award for work on learner-centric, adaptive cyber-tools and cyber-environments. He was one of 49 faculty members selected as the nation’s top engineering educators and researchers by the US National Academy of Engineering to the Frontiers in Engineering Education symposium.

visit author page

Download Paper |


Weighted Social Tagging as a Research Methodology for Determining Systemic Trends in Engineering Education ResearchAs a new and emerging problem space, engineering education research continues to define itscore content, methods, and theory. The research literature in engineering education clearlydemonstrates that as a community, we continue to apply and extend methods that have beentraditionally available in the fields of learning sciences, education, psychology, and numerousother methodological traditions. However, the field of engineering education research has notfully utilized or innovated new methods that leverage more modern web 2.0 techniques tounderstand systemic trends within the problem space. Recently scientific, peer-reviewed papershave begun to emerge that utilize simplistic tag clouds (e.g. Wordles™) as a way of showcasingthe core concepts conveyed within a problem space. In this paper, we introduce a new andinnovative technique called weighted social tagging as a research methodology.Social tagging is a categorizing system that relies on users, as opposed to machines, to generatekeyword descriptions—known as “tags”—about a resource, such as a picture, video, ordocument. This categorizing is useful even for dealing with volatile and poorly defined resourcesand allows communities to provide definitions according to their own standards andunderstandings. Weighted social tagging extends this notion significantly and combines thisapproach with the ability for users to assign relative weights to their tags and also a confidencerating.We demonstrate the application of weighted social tagging on a small-scale dataset of papersfrom the Journal of Engineering Education (JEE) that extend over a period of 5 years from 2005to 2009 – a total of 155 papers. We attempt to address the following questions: (1) What are thetrends and core topics in JEE from 2005 to 2009? (2) How accurate is weighted social tagging inidentifying trends and core concepts? Each paper in the dataset was read by 3 researchers withina maximum time of 4 minutes per paper (arbitrarily short period of time). Each article wasassigned approximately 7 tags per member, with a breakdown of 3 words describing thebackground of the article, 2 for the methods used, and 2 for the results and impact of theresearch. Each tag was then weighted on a scale of 1-100 based on its perceived importance inthe context of the paper, such that the sum of weights for all article tags was equal to 100. Eachtag was also designated a confidence rank between 0 and 1 to demonstrate how certainindividuals felt about their tag weight. All the tags and corresponding weights were combinedand averaged to aggregate each tag’s importance. Figure 1 shows the evolution of the keywordassessment over the period 2005 to 2009 as seen in JEE research articles. We provide inter-tagger semantic correlations in the full paper as a verification process.In the full paper, using techniques found in the field of data mining and visual analytics, weshow how the weighted social tagging method can be combined with graph-based visualizationtechniques to gain a deeper understanding of the dataset. The power of this technique lies in itsability to quickly leverage the collective intelligence of a community of researchers. Clearly, justone reader’s tags will be insufficient to derive the full context and meaning of a paper. However,when we engage a large group of researchers, the tags as a collection quickly render a significantportion of the meaning of a dataset. When this dataset is placed on a timeline – trend patterns ofconcepts, methodologies, and findings points begin to emerge.Figure 1. The evolution of the keyword assessment is shown here. Weighted social tagging is afast and effective method for understanding systemic trends within the engineering educationresearch problem space.

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2011 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015