Refining a Taxonomy for Categorizing the Quality of Engineering Student Questions

Saryn R. Goldberg; Chris Venters; Amy Masnick

Download Paper | Permalink

Conference: 2021 ASEE Virtual Annual Conference Content Access
Location: Virtual Conference
Publication Date: July 26, 2021
Start Date: July 26, 2021
End Date: July 19, 2022
Conference Session: NSF Grantees Poster Session
Tagged Topic: NSF Grantees Poster Session
Page Count: 9
DOI: 10.18260/1-2--37649
Permanent URL: https://strategy.asee.org/37649
Download Count: 167

Request a correction

Paper Authors

biography

Saryn R. Goldberg Hofstra University

visit author page

Dr. Saryn R. Goldberg is an Associate Professor of Mechanical Engineering in Hofstra University’s DeMatteis School of Engineering and Applied Sciences. Dr. Goldberg received her Sc.B. in Engineering with a focus on materials science from Brown University, her M.S. degree in Biomedical Engineering with a focus on biomaterials from Northwestern University, and her Ph.D. in Mechanical Engineering with a focus on biomechanics from Stanford University. At Hofstra she teaches courses in mechanical engineering and materials science. Her research in engineering education focuses on the use of student question-asking to promote metacognition.

visit author page

biography

Chris Venters East Carolina University

visit author page

Chris Venters is an Assistant Professor in the Department of Engineering at East Carolina University in Greenville, North Carolina, USA. He teaches introductory courses in engineering design and mechanics and upper-level courses in fluid mechanics. He earned his Ph.D. in Engineering Education from Virginia Tech in 2014, and his research primarily focuses on conceptual understanding in engineering mechanics courses. He received his M.S. in Aerospace Engineering from Virginia Tech and his B.S. in Aerospace Engineering from North Carolina State University.

visit author page

biography

Amy Masnick Hofstra University

visit author page

Dr. Amy Masnick is a Professor of Psychology at Hofstra University. Dr. Masnick received both her B.S. and Ph.D. in Human Development at Cornell University. At Hofstra she teaches courses in introductory psychology, research methods, cognitive psychology, and child development. Dr. Masnick is interested in conceptual development, reasoning about science and number in children and adults, and in science and engineering education.

visit author page

Download Paper | Permalink

Abstract

The ability to identify one’s own confusion and to ask a question that resolves it is an essential metacognitive skill that supports self-regulation (Winne, 2005). Yet, while students receive substantial training in how to answer questions, little classroom time is spent training students how to ask good questions. Past research has shown that students are able to pose more high-quality questions after being instructed in a taxonomy for classifying the quality of their questions (Marbach‐Ad & Sokolove, 2000). As pilot data collection in preparation for a larger study funded through NSF-DUE, we provided engineering statics students training in writing high-quality questions to address their own confusions. The training emphasized the value of question-asking in learning and how to categorize questions using a simple taxonomy based on prior work (Harper et al., 2003). The taxonomy specifies five question levels: 1) an unspecific question, 2) a definition question, 3) a question about how to do something, 4) a why question, and 5) a question that extends knowledge to a new circumstance. At the end of each class period during a semester-long statics course, students were prompted to write and categorize a question that they believed would help them clarify their current point of greatest confusion. Through regular practice writing and categorizing such questions, we hoped to improve students' abilities to ask questions that require higher-level thinking. We collected data from 35 students in courses at two institutions. Over the course of the semester, students had the opportunity to write and categorize twenty of their own questions. After the semester, the faculty member categorized student questions using the taxonomy to assess the appropriateness of the taxonomy and whether students used it accurately. Analysis of the pilot data indicates three issues to be addressed: 1) Student compliance in writing and categorizing their questions varied. 2) Some students had difficulty correctly coding their questions using the taxonomy. 3) Some student questions could not be clearly characterized using the taxonomy, even for faculty raters. We will address each of these issues with appropriate refinements in our next round of data collection: 1) Students may have been overwhelmed with the request to write a question after each class period. In the future, we will require students to write and categorize at least one question per week, with more frequent questions encouraged. 2) To improve student use of the taxonomy in future data collection, students will receive more practice with the taxonomy when it is introduced and more feedback on their categorization of questions during the semester. 3) We are reformulating our taxonomy to accommodate questions that may straddle more than one category, such as a question about how to extend a mathematical operation to a new situation (which could be categorized as either a level 3 or 5). We are hopeful that these changes will improve accuracy and compliance, enabling us to use the intervention as a means to promote metacognitive regulation and measure changes as a result, which is the intent of the larger scope of the project.

Citation
Format

Goldberg, S. R., & Venters, C., & Masnick, A. (2021, July), Refining a Taxonomy for Categorizing the Quality of Engineering Student Questions Paper presented at 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference. 10.18260/1-2--37649

TY - CPAPER
AB - The ability to identify one’s own confusion and to ask a question that resolves it is an essential metacognitive skill that supports self-regulation (Winne, 2005). Yet, while students receive substantial training in how to answer questions, little classroom time is spent training students how to ask good questions. Past research has shown that students are able to pose more high-quality questions after being instructed in a taxonomy for classifying the quality of their questions (Marbach‐Ad &amp;amp; Sokolove, 2000).

As pilot data collection in preparation for a larger study funded through NSF-DUE, we provided engineering statics students training in writing high-quality questions to address their own confusions. The training emphasized the value of question-asking in learning and how to categorize questions using a simple taxonomy based on prior work (Harper et al., 2003). The taxonomy specifies five question levels: 1) an unspecific question, 2) a definition question, 3) a question about how to do something, 4) a why question, and 5) a question that extends knowledge to a new circumstance.

At the end of each class period during a semester-long statics course, students were prompted to write and categorize a question that they believed would help them clarify their current point of greatest confusion. Through regular practice writing and categorizing such questions, we hoped to improve students&#39; abilities to ask questions that require higher-level thinking.

We collected data from 35 students in courses at two institutions. Over the course of the semester, students had the opportunity to write and categorize twenty of their own questions. After the semester, the faculty member categorized student questions using the taxonomy to assess the appropriateness of the taxonomy and whether students used it accurately. Analysis of the pilot data indicates three issues to be addressed:
1) Student compliance in writing and categorizing their questions varied.
2) Some students had difficulty correctly coding their questions using the taxonomy.
3) Some student questions could not be clearly characterized using the taxonomy, even for faculty raters.

We will address each of these issues with appropriate refinements in our next round of data collection:
1) Students may have been overwhelmed with the request to write a question after each class period. In the future, we will require students to write and categorize at least one question per week, with more frequent questions encouraged.
2) To improve student use of the taxonomy in future data collection, students will receive more practice with the taxonomy when it is introduced and more feedback on their categorization of questions during the semester.
3) We are reformulating our taxonomy to accommodate questions that may straddle more than one category, such as a question about how to extend a mathematical operation to a new situation (which could be categorized as either a level 3 or 5).
We are hopeful that these changes will improve accuracy and compliance, enabling us to use the intervention as a means to promote metacognitive regulation and measure changes as a result, which is the intent of the larger scope of the project.

AU - Saryn R. Goldberg
AU - Chris Venters
AU - Amy Masnick
CY - Virtual Conference
DA - 2021/07/26
PB - ASEE Conferences
TI - Refining a Taxonomy for Categorizing the Quality of Engineering Student Questions
UR - https://strategy.asee.org/37649
DO - 10.18260/1-2--37649
ER -