Asee peer logo

Evaluating Publications' Keywords in Computer Science Education Research: A Bibliometric NLP Approach

Download Paper |

Conference

2021 ASEE Virtual Annual Conference Content Access

Location

Virtual Conference

Publication Date

July 26, 2021

Start Date

July 26, 2021

End Date

July 19, 2022

Conference Session

Computing and Information Technology Division Technical Session 3

Tagged Division

Computing and Information Technology

Page Count

15

Permanent URL

https://peer.asee.org/37104

Download Count

221

Request a correction

Paper Authors

biography

Jia Zhu Florida International University Orcid 16x16 orcid.org/0000-0001-9234-5919

visit author page

Jia Zhu is a Ph.D. student in the Knight Foundation School of Computing and Information Science at Florida International University (FIU). Her research interests include computer science education, educational data mining, and data science, with a focus on broadening participation in computing.

visit author page

biography

Leila Zahedi Florida International University Orcid 16x16 orcid.org/0000-0002-7325-1025

visit author page

Leila Zahedi is a Ph.D. candidate in computer science at the School of Computing and Information Sciences (SCIS) at Florida International University (FIU). Her research interests span the fields of educational data mining, machine learning optimization, and data science. Leila also received an M.S. degree in Management of Advanced Information Systems, in addition to her B.S. and M.S. degrees in Computer Science.

visit author page

biography

Monique S. Ross Florida International University Orcid 16x16 orcid.org/0000-0002-6320-636X

visit author page

Monique Ross, Assistant Professor in the School of Computing and Information Sciences and STEM Transformation Institute at Florida International University, designs research focused on broadening participation in computer science through the exploration of: 1) race, gender, and disciplinary identity; 2) discipline-based education research (with a focus on computer science and computer engineering courses) in order to inform pedagogical practices that garner interest and retain women (specifically Black and Hispanic women) in computer-related engineering fields. 

visit author page

Download Paper |

Abstract

In the rapidly growing field of computer science education research (CSER), the collection of publication keywords represent the knowledge landscape of the research domain. This helps stakeholders to contextualize existing research topics, identify the strategic direction for future works, and shape the culture of this growing community. For the research studies published in CSER, the use of proper keywords will help to improve publication visibility in research networks. Publication visibility is defined as the share of traffic a study receives based on its ranking from Search Engine Result Pages (SERPs), which can be gained through indexing relevant keywords to help search engines to find the related studies. It is important to acknowledge that analyzing keyword selections in the context provided by the publication abstract, enables us to determine the keywords’ relevance to the research topics. However, evaluating the selection of publication keywords is an important but lesser-addressed issue.

In this work, we utilized a bibliometric approach to scrutinize publication keywords in CSER publications from 2015 to 2020. We gathered the titles, abstracts, and keywords collectively from studies published in two major CS education conferences and journals. We also extracted the strategic directions from their mission statements. By applying the Natural Language Processing (NLP) technique, known as Term Frequency-Inverse Document Frequency (tf-idf), we aimed to answer: 1) What are the most prevalent research foci represented in CSER through the publication keywords? 2) Are publication keywords in alignment with the CSER publications’ stated strategic directions? 3) Do selected keywords in CSER publications adequately represent the research topics presented via publication abstracts?

Our preliminary results suggest that, as expected, the most prevalent research foci represented by the most commonly used keywords of CSER were on teaching and learning and broadening participation, which is in alignment with the corresponding strategic directions. Our analysis suggests that the CSER community is putting efforts to formulate a diversified culture through teaching and learning for inclusivity. However, our results also suggest that there is a misalignment between keyword identification by authors and the topics presented in the abstracts. With our work, we hope to motivate scholars to carefully evaluate and select the keywords that will be used in the future for indexing publications to improve the research topic relevancy and publication visibility for broader impact.

Zhu, J., & Zahedi, L., & Ross, M. S. (2021, July), Evaluating Publications' Keywords in Computer Science Education Research: A Bibliometric NLP Approach Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. https://peer.asee.org/37104

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2021 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015