Tang, Y., & Ding, L., & Bai, H., & Catrambone, R. (2023, June), Work in Progress: Evaluating the Effect of Symbolic Problem Solving on Testing Validity and Reliability  Paper presented at 2023 ASEE Annual Conference & Exposition, Baltimore , Maryland. 10.18260/1-2--44248

\bibitem{asee_peer_44248}
  Tang, Y., \& Ding, L., \& Bai, H., \& Catrambone, R. (2023, June), \emph{Work in Progress: Evaluating the Effect of Symbolic Problem Solving on Testing Validity and Reliability}
  Paper presented at 2023 ASEE Annual Conference \& Exposition, Baltimore , Maryland.
  10.18260/1-2--44248

Yan Tang, Lin Ding, Haiyan Bai, and Richard Catrambone.  "Work in Progress: Evaluating the Effect of Symbolic Problem Solving on Testing Validity and Reliability".  2023 ASEE Annual Conference & Exposition, Baltimore , Maryland, 2023, June.  ASEE Conferences, 2023.  https://peer.asee.org/44248 Internet.  13 Jul, 2025

\bibitem{asee_peer_44248}
  Yan Tang, Lin Ding, Haiyan Bai, and Richard Catrambone.
  "Work in Progress: Evaluating the Effect of Symbolic Problem Solving on Testing Validity and Reliability".
  \emph{2023 ASEE Annual Conference \& Exposition, Baltimore , Maryland, 2023, June}.
  ASEE Conferences, 2023.
  https://peer.asee.org/44248 Internet.  13 Jul, 2025

@INPROCEEDINGS{asee_peer_44248
author = "Yan Tang, Lin Ding, Haiyan Bai, and Richard Catrambone"
title = "Work in Progress: Evaluating the Effect of Symbolic Problem Solving on Testing Validity and Reliability"
booktitle = "2023 ASEE Annual Conference \& Exposition"
year = "2023"
month = "June"
address = "Baltimore , Maryland"
publisher = "ASEE Conferences"
note = {https://peer.asee.org/44248}
number = {10.18260/1-2--44248}
}

TY  - CPAPER
AB  - Problem-solving is a typical assessment topic in engineering dynamics tests. To solve a problem, students need to set up equations and find a numeric answer. It usually takes ten to thirty minutes to solve a quantitative problem, depending on its difficulty and complexity level. Due to the time constraint of in-class testing, a typical test can only include a limited number of problems with insufficient problem types. It may lower testing validity and reliability, two essential factors which contribute to assessment results.

A test with a high validity should cover proper content. It should be able to distinguish high-performing students from low-performing students and every student in between. A reliable test should have a sufficient number of items to provide consistent information about the student’s mastery of the materials. Both validity and reliability are two important criteria for quality assessment.  

In this work-in-progress study, we will investigate whether we can develop a valid and reliable test on students’ solving symbolic engineering problems in engineering dynamics tests. Symbolic problem solving in this study refers to solving problems by setting up a system of equations without finding numeric solutions. It usually takes much less time. As a result, more problems of a variety of types can be included on a test. We will use the Classical Test Theory to evaluate the validity and reliability metrics of a collection of symbolic problems. Examples on rectilinear kinematics and angular motion will be provided to illustrate how symbolic problem solving is used in both homework and testing. 

As numerous studies in the literature have shown that symbolic questions impose greater challenges to students because of their difficulties with math, we will also share strategies on how to prepare students to ease the learning curve.
AU  - Yan Tang
AU  - Lin Ding
AU  - Haiyan Bai
AU  - Richard Catrambone
CY  - Baltimore , Maryland
DA  - 2023/06/25
PB  - ASEE Conferences
TI  - Work in Progress: Evaluating the Effect of Symbolic Problem Solving on Testing Validity and Reliability
UR  - https://peer.asee.org/44248
DO  - 10.18260/1-2--44248
ER  -