Board 14A: Work in Progress: Integrating Information and Data Literacy Skills into Biomedical Engineering Laboratory Courses

Alexander James Carroll; Joshua Daniel Borycz; Sheldon Salo; Amanda R. Lowery; Eric Spivey

Download Paper | Permalink

Conference: 2024 ASEE Annual Conference & Exposition
Location: Portland, Oregon
Publication Date: June 23, 2024
Start Date: June 23, 2024
End Date: June 26, 2024
Conference Session: Biomedical Engineering Division (BED) Poster Session
Tagged Division: Biomedical Engineering Division (BED)
Page Count: 5
DOI: 10.18260/1-2--46709
Permanent URL: https://peer.asee.org/46709
Download Count: 69

Paper Authors

biography

Alexander James Carroll Vanderbilt University orcid.org/0000-0003-0248-3811

visit author page

Alex Carroll, MSLS, AHIP, is the Associate Director of the Science and Engineering Library (SEL) at Vanderbilt University. Alex leads the SEL's liaison program, designing and delivering services to support the research enterprise and the teaching mission of the School of Engineering and STEM academic units within the College of Arts and Science. He received his MSLS degree from the University of North Carolina at Chapel Hill’s School of Information and Library Science, and his BA from James Madison University.

Alex is an Associate Editor of the Journal of the Medical Library Association (JMLA) and is a Distinguished member of MLA's Academy of Health Information Professionals (AHIP). His research interests include studying the information seeking behaviors and data practices of STEM researchers, designing library liaison services, and improving information literacy instruction for students in the sciences. His work in these areas has been recognized by the ALA Library Instruction Round Table with "Top Twenty" awards in 2018 and 2019, and by ASEE's Engineering Library Division with Best Publication Awards in 2020 and 2022.

visit author page

biography

Joshua Daniel Borycz Vanderbilt University orcid.org/0000-0002-1505-148X

visit author page

At Vanderbilt University I help graduate and undergraduate students learn how to do research and succeed academically by introducing them to a range of tools, developing new tools, creating educational programs, and advocating for the use of library servi

visit author page

author page

Sheldon Salo Vanderbilt University Library orcid.org/0009-0002-0272-0784

biography

Amanda R. Lowery Vanderbilt University

visit author page

Amanda Lowery is a Professor of Practice at Vanderbilt University Department of Biomedical Engineering. She has been working in the field of accessibility and service learning for 8 years. Her specific interests are in early childhood mobility and toy accessibility.

visit author page

author page

Eric Spivey Vanderbilt University

Download Paper | Permalink

Abstract

Background: Undergraduate engineering programs train students to develop, conduct, interpret experiments as well as foster an ability to acquire and apply new knowledge as needed [1]. While engineering educators recognize the importance of equipping students with the ability to find, evaluate, and synthesize technical information, limited classroom time inhibits instructors’ ability to teach students these skills [2], [3]. Engineering librarians are well-suited to partner with engineering educators in designing instructional interventions that promote student development of these crucial skills [4]. Senior design courses have been identified within the literature as rich opportunities for integrating information literacy instruction into the biomedical engineering (BME) curriculum [5]. While single interventions within senior design courses are helpful, students may experience greater benefits from a holistic information literacy training program that is scaffolded across the undergraduate curriculum [6], [7].

Purpose: We explored whether laboratory courses were an effective context for integrating information literacy throughout the undergraduate biomedical engineering curricula. We scaffolded instructional interventions across the undergraduate curriculum, anticipating that after participation in this information literacy program, students would demonstrate an increased understanding of the breadth of technical information sources available.

Instructional Methods: Students at [Institution Redacted] completed a required BME laboratory course as a sequential series of one credit courses in their sophomore (BME 2900W), junior (BME 3900W), and senior (BME 4901) years. In addition to gaining expertise in conducting experiments methods, collecting data, and interpreting findings, students were expected to demonstrate progressive growth in their ability to write laboratory reports that placed their results in context with previously published primary literature.

To promote students' skill growth, engineering librarians provided guest lectures in this three-course sequence, introducing information sources and how to access them. In BME 2900W, librarians demonstrated how to find experimental protocols, engineering handbooks, and methods papers. In BME 3900W, students learned how to find patents and review articles. In BME 4901W, librarians provided an overview of managing research data, including best practices for organizing files and designing machine-readable tabular data. Methods: Beginning in the Spring 2022 semester, we initiated a longitudinal assessment of this information literacy program. Students completed pre-tests prior to each of the interventions that established their baseline knowledge, while also tracking skills gained and retained between each course. Following their completion of BME 4901, student also completed an identical post-test. The first cohort of students that experienced the full training sequence matriculated through the program in the Fall 2023 Semester. Data collection will continue into the Spring 2025 semester.

The pre-tests and post-test, which included a mix of objective and open response questions, are available on the Open Science Framework [9]. This protocol was reviewed by the [Institution Redacted] Institutional Review Board and was approved as a Quality Improvement project (IRB #232075).

The survey responses to the multiple-choice questions from each class were recorded in a spreadsheet with response identifiers, course information, and answers. The Analysis of Variance (ANOVA) method was used to compare the proportion of correct answers from each biomedical engineering class (2900W, 3900W, 4901W) to the objective questions in these tests. The Tukey Honest Significant Difference (TukeyHSD) test was performed to make pairwise comparisons between individual variables. These were computed using the stats package in R-4.3.1.

Preliminary Results: Table 1 reports results from student performance on nine objective questions. Q3-Q7 asked students to identified what source of information they should consult when completing different tasks. Q8-Q11 asked students to identified which library-licensed resource they should utilize to find different types of technical literature documents. Figure 1 shows the proportion of correct responses divided by class, question, and topic. The pre- and post-test results for BME 4901W were combined because the content taught in this course (research data management) was not covered in the multiple-choice questions analyzed in this work. The ANOVA and TukeyHSD results indicate a significant increase in the number of correct responses to all test questions within each successive course (Mean square = 4.6; F-statistic = 30.4; p-value<0.001). The ANOVA results for course and question interaction term indicated a significant difference for questions across courses (Mean square = 0.43; F-statistic = 2.80; p-value<0.001). The TukeyHSD results for each individual question compared across courses showed significant results for question 6 (2900W-4901W: p<0.05), question 10 (2900W-3900W: p<0.001; 2900W-4901W: p<0.001), and question 11 (2900W-3900W: p<0.01).

Since questions 1 to 7 each address resource types, 8 to 11 address tool knowledge, and 12 is on article structure, these questions were combined into the topic categories Resource, Tool, and Reading for comparison. ANOVA results for the course and topic interaction term showed a significant difference for topics across courses (Mean square = 0.63; F-statistic = 3.64; p<0.01). The TukeyHSD test for topics across courses showed significant increases in correct responses for Resource (2900W-4901W: p<0.01) and Tool (2900W-3900W: p<0.001; 2900W-4901W: p<0.001) related questions.

Discussion: These results show that information literacy instruction can have a lasting impact on student outcomes. Figure 1A demonstrates an overall increase in knowledge on research information. The test questions indicated that most biomedical engineering students were already familiar with patents, research articles, and protocols (Q3, Q4, and Q5), as well as the Derwent Innovation Index and Web of Science (Q8 and Q9) (Figure 1B), which all had correct response rates of over 75% in the BME 2900W pre-tests. The instructional focus on the importance of review articles for understanding fields and connecting to important journal articles helped students increase their familiarity with this resource type (Q6). While most students in 2900W had not heard of Cold Spring Harbor (Q10) or AccessScience (Q11), after attending a didactic on how to use these tools, students remembered the types of information they could be used to find. Overall, the students’ knowledge of research tools and research types increased significantly after each session (Figure 1C).

Future Work: The objective questions in these tests do not measure how well students can use technical literature; rather, they test recognition of resource types, tools, and article structure. At the 2024 Annual Meeting, we plan to share preliminary data from a sentiment analysis of a random sample of open response questions from students who completed all four tests. The text responses from students may reveal more improvement in student research workflows and confidence levels.

In additional future work, we plan to assess a random sample of students’ laboratory reports using rubrics to measure student achievement of learning outcomes and citation analysis to measure extent of information use [10]–[13]. The complete rubric can be viewed online on the Open Science Framework [15]. To isolate the effects of this information literacy training program, lab reports generated by students that participated in this information literacy training will be compared to a sample of laboratory reports created by previous cohorts of BME students who did not receive any specialized information literacy training within their laboratory course. These assessments of student learning artifacts will provide clearer depictions of whether the skill gains reflected in the objective questions translated into improved practices.

Citation
Format

Carroll, A. J., & Borycz, J. D., & Salo, S., & Lowery, A. R., & Spivey, E. (2024, June), Board 14A: Work in Progress: Integrating Information and Data Literacy Skills into Biomedical Engineering Laboratory Courses Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. 10.18260/1-2--46709

TY - CPAPER
AB - Background: Undergraduate engineering programs train students to develop, conduct, interpret experiments as well as foster an ability to acquire and apply new knowledge as needed [1]. While engineering educators recognize the importance of equipping students with the ability to find, evaluate, and synthesize technical information, limited classroom time inhibits instructors’ ability to teach students these skills [2], [3]. Engineering librarians are well-suited to partner with engineering educators in designing instructional interventions that promote student development of these crucial skills [4]. Senior design courses have been identified within the literature as rich opportunities for integrating information literacy instruction into the biomedical engineering (BME) curriculum [5]. While single interventions within senior design courses are helpful, students may experience greater benefits from a holistic information literacy training program that is scaffolded across the undergraduate curriculum [6], [7].

Purpose: We explored whether laboratory courses were an effective context for integrating information literacy throughout the undergraduate biomedical engineering curricula. We scaffolded instructional interventions across the undergraduate curriculum, anticipating that after participation in this information literacy program, students would demonstrate an increased understanding of the breadth of technical information sources available.

Instructional Methods: Students at [Institution Redacted] completed a required BME laboratory course as a sequential series of one credit courses in their sophomore (BME 2900W), junior (BME 3900W), and senior (BME 4901) years. In addition to gaining expertise in conducting experiments methods, collecting data, and interpreting findings, students were expected to demonstrate progressive growth in their ability to write laboratory reports that placed their results in context with previously published primary literature.

To promote students&#39; skill growth, engineering librarians provided guest lectures in this three-course sequence, introducing information sources and how to access them. In BME 2900W, librarians demonstrated how to find experimental protocols, engineering handbooks, and methods papers. In BME 3900W, students learned how to find patents and review articles. In BME 4901W, librarians provided an overview of managing research data, including best practices for organizing files and designing machine-readable tabular data.
Methods: Beginning in the Spring 2022 semester, we initiated a longitudinal assessment of this information literacy program. Students completed pre-tests prior to each of the interventions that established their baseline knowledge, while also tracking skills gained and retained between each course. Following their completion of BME 4901, student also completed an identical post-test. The first cohort of students that experienced the full training sequence matriculated through the program in the Fall 2023 Semester. Data collection will continue into the Spring 2025 semester.

The pre-tests and post-test, which included a mix of objective and open response questions, are available on the Open Science Framework [9]. This protocol was reviewed by the [Institution Redacted] Institutional Review Board and was approved as a Quality Improvement project (IRB #232075).

The survey responses to the multiple-choice questions from each class were recorded in a spreadsheet with response identifiers, course information, and answers. The Analysis of Variance (ANOVA) method was used to compare the proportion of correct answers from each biomedical engineering class (2900W, 3900W, 4901W) to the objective questions in these tests. The Tukey Honest Significant Difference (TukeyHSD) test was performed to make pairwise comparisons between individual variables. These were computed using the stats package in R-4.3.1.

Preliminary Results: Table 1 reports results from student performance on nine objective questions. Q3-Q7 asked students to identified what source of information they should consult when completing different tasks. Q8-Q11 asked students to identified which library-licensed resource they should utilize to find different types of technical literature documents.

Figure 1 shows the proportion of correct responses divided by class, question, and topic. The pre- and post-test results for BME 4901W were combined because the content taught in this course (research data management) was not covered in the multiple-choice questions analyzed in this work. The ANOVA and TukeyHSD results indicate a significant increase in the number of correct responses to all test questions within each successive course (Mean square = 4.6; F-statistic = 30.4; p-value&amp;lt;0.001). The ANOVA results for course and question interaction term indicated a significant difference for questions across courses (Mean square = 0.43; F-statistic = 2.80; p-value&amp;lt;0.001). The TukeyHSD results for each individual question compared across courses showed significant results for question 6 (2900W-4901W: p&amp;lt;0.05), question 10 (2900W-3900W: p&amp;lt;0.001; 2900W-4901W: p&amp;lt;0.001), and question 11 (2900W-3900W: p&amp;lt;0.01).

Since questions 1 to 7 each address resource types, 8 to 11 address tool knowledge, and 12 is on article structure, these questions were combined into the topic categories Resource, Tool, and Reading for comparison. ANOVA results for the course and topic interaction term showed a significant difference for topics across courses (Mean square = 0.63; F-statistic = 3.64; p&amp;lt;0.01). The TukeyHSD test for topics across courses showed significant increases in correct responses for Resource (2900W-4901W: p&amp;lt;0.01) and Tool (2900W-3900W: p&amp;lt;0.001; 2900W-4901W: p&amp;lt;0.001) related questions.

Discussion: These results show that information literacy instruction can have a lasting impact on student outcomes. Figure 1A demonstrates an overall increase in knowledge on research information. The test questions indicated that most biomedical engineering students were already familiar with patents, research articles, and protocols (Q3, Q4, and Q5), as well as the Derwent Innovation Index and Web of Science (Q8 and Q9) (Figure 1B), which all had correct response rates of over 75% in the BME 2900W pre-tests. The instructional focus on the importance of review articles for understanding fields and connecting to important journal articles helped students increase their familiarity with this resource type (Q6). While most students in 2900W had not heard of Cold Spring Harbor (Q10) or AccessScience (Q11), after attending a didactic on how to use these tools, students remembered the types of information they could be used to find. Overall, the students’ knowledge of research tools and research types increased significantly after each session (Figure 1C).

Future Work: The objective questions in these tests do not measure how well students can use technical literature; rather, they test recognition of resource types, tools, and article structure. At the 2024 Annual Meeting, we plan to share preliminary data from a sentiment analysis of a random sample of open response questions from students who completed all four tests. The text responses from students may reveal more improvement in student research workflows and confidence levels.

In additional future work, we plan to assess a random sample of students’ laboratory reports using rubrics to measure student achievement of learning outcomes and citation analysis to measure extent of information use [10]–[13]. The complete rubric can be viewed online on the Open Science Framework [15]. To isolate the effects of this information literacy training program, lab reports generated by students that participated in this information literacy training will be compared to a sample of laboratory reports created by previous cohorts of BME students who did not receive any specialized information literacy training within their laboratory course. These assessments of student learning artifacts will provide clearer depictions of whether the skill gains reflected in the objective questions translated into improved practices.
AU - Alexander James Carroll
AU - Joshua Daniel Borycz
AU - Sheldon Salo
AU - Amanda R. Lowery
AU - Eric Spivey
CY - Portland, Oregon
DA - 2024/06/23
PB - ASEE Conferences
TI - Board 14A: Work in Progress: Integrating Information and Data Literacy Skills into Biomedical Engineering Laboratory Courses
UR - https://peer.asee.org/46709
DO - 10.18260/1-2--46709
ER -