June 26, 2011
June 26, 2011
June 29, 2011
Educational Research and Methods
22.1675.1 - 22.1675.20
Weighted Social Tagging as a Research Methodology for Determining Systemic Trends in Engineering Education ResearchAs a new and emerging problem space, engineering education research continues to define itscore content, methods, and theory. The research literature in engineering education clearlydemonstrates that as a community, we continue to apply and extend methods that have beentraditionally available in the fields of learning sciences, education, psychology, and numerousother methodological traditions. However, the field of engineering education research has notfully utilized or innovated new methods that leverage more modern web 2.0 techniques tounderstand systemic trends within the problem space. Recently scientific, peer-reviewed papershave begun to emerge that utilize simplistic tag clouds (e.g. Wordles™) as a way of showcasingthe core concepts conveyed within a problem space. In this paper, we introduce a new andinnovative technique called weighted social tagging as a research methodology.Social tagging is a categorizing system that relies on users, as opposed to machines, to generatekeyword descriptions—known as “tags”—about a resource, such as a picture, video, ordocument. This categorizing is useful even for dealing with volatile and poorly defined resourcesand allows communities to provide definitions according to their own standards andunderstandings. Weighted social tagging extends this notion significantly and combines thisapproach with the ability for users to assign relative weights to their tags and also a confidencerating.We demonstrate the application of weighted social tagging on a small-scale dataset of papersfrom the Journal of Engineering Education (JEE) that extend over a period of 5 years from 2005to 2009 – a total of 155 papers. We attempt to address the following questions: (1) What are thetrends and core topics in JEE from 2005 to 2009? (2) How accurate is weighted social tagging inidentifying trends and core concepts? Each paper in the dataset was read by 3 researchers withina maximum time of 4 minutes per paper (arbitrarily short period of time). Each article wasassigned approximately 7 tags per member, with a breakdown of 3 words describing thebackground of the article, 2 for the methods used, and 2 for the results and impact of theresearch. Each tag was then weighted on a scale of 1-100 based on its perceived importance inthe context of the paper, such that the sum of weights for all article tags was equal to 100. Eachtag was also designated a confidence rank between 0 and 1 to demonstrate how certainindividuals felt about their tag weight. All the tags and corresponding weights were combinedand averaged to aggregate each tag’s importance. Figure 1 shows the evolution of the keywordassessment over the period 2005 to 2009 as seen in JEE research articles. We provide inter-tagger semantic correlations in the full paper as a verification process.In the full paper, using techniques found in the field of data mining and visual analytics, weshow how the weighted social tagging method can be combined with graph-based visualizationtechniques to gain a deeper understanding of the dataset. The power of this technique lies in itsability to quickly leverage the collective intelligence of a community of researchers. Clearly, justone reader’s tags will be insufficient to derive the full context and meaning of a paper. However,when we engage a large group of researchers, the tags as a collection quickly render a significantportion of the meaning of a dataset. When this dataset is placed on a timeline – trend patterns ofconcepts, methodologies, and findings points begin to emerge.Figure 1. The evolution of the keyword assessment is shown here. Weighted social tagging is afast and effective method for understanding systemic trends within the engineering educationresearch problem space.
ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2011 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015