Asee peer logo

Comparative Analysis of OpenAI GPT-4o and DeepSeek R1 for Scientific Text Categorization Using Prompt Engineering

Download Paper |

Conference

2025 ASEE North Central Section (NCS) Annual Conference

Location

Marshall University, Huntington, West Virginia

Publication Date

March 28, 2025

Start Date

March 28, 2025

End Date

March 29, 2025

Tagged Topic

Diversity

Page Count

17

Permanent URL

https://peer.asee.org/54654

Paper Authors

author page

Aniruddha Maiti

author page

Samuel Adewumi

author page

TEMESGEN ALEMAYEHU TIKURE

author page

Zichun Wang

author page

Niladri Sengupta

author page

Anastasiia Sukhanova Marshall Community & Technical College

author page

Ananya Jana Marshall University

Download Paper |

Abstract

This study examines how large language models categorize sentences from scientific papers using prompt engineering. We use two advanced web-based models, OpenAI’s GPT-4o and DeepSeek R1, to classify sentences into predefined relationship categories. DeepSeek R1 has been tested on benchmark datasets in its technical report. However, its performance in scientific text categorization remains unexplored. To address this gap, we introduce a new evaluation method designed specifically for this task. We also compile a dataset of cleaned scientific papers from diverse domains. This dataset provides a platform for comparing the two models. Using this dataset, we analyze their effectiveness and consistency in categorization

Maiti, A., & Adewumi, S., & TIKURE, T. A., & Wang, Z., & Sengupta, N., & Sukhanova, A., & Jana, A. (2025, March), Comparative Analysis of OpenAI GPT-4o and DeepSeek R1 for Scientific Text Categorization Using Prompt Engineering Paper presented at 2025 ASEE North Central Section (NCS) Annual Conference, Marshall University, Huntington, West Virginia. https://peer.asee.org/54654

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2025 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015