Assessing Engineering Students’ Information Literacy Skills: An Alpha Version of a Multiple-Choice Instrument

Ruth Wertz; Meagan C. Ross; Senay Purzer; Michael Fosmire; Monica E. Cardella

Download Paper | Permalink

Conference: 2011 ASEE Annual Conference & Exposition
Location: Vancouver, BC
Publication Date: June 26, 2011
Start Date: June 26, 2011
End Date: June 29, 2011
ISSN: 2153-5965
Conference Session: They're Not "Soft" Skills!
Tagged Division: Educational Research and Methods
Page Count: 11
Page Numbers: 22.237.1 - 22.237.11
DOI: 10.18260/1-2--17518
Permanent URL: https://peer.asee.org/17518
Download Count: 517

Paper Authors

biography

Ruth Wertz P.E. Purdue University, West Lafayette orcid.org/0000-0002-9698-3392

visit author page

Ruth E. H. Wertz is a graduate student in Engineering Education at Purdue University. She is a Professional Engineer in the State of Indiana, and holds B.S. and M.S. degrees in Civil Engineering from Trine University and Purdue University.

visit author page

biography

Meagan C. Ross Purdue University, West Lafayette

visit author page

Meagan Ross is passionate about providing awareness of engineering to K-12 educators, enabling them to actively and confidently influence students to become the next generation of world changing engineers. Through her consulting business, Meagan regularly develops and facilitates workshops on STEM careers for K-12 educators, parents, and students, with the personal objective to help close the gender gap in engineering.

Meagan Ross is a Ph.D. student in the School of Engineering Education at Purdue University, and is a recipient of a National Science Foundation Graduate Research Fellowship. She received a B.S. in Computer Science from Texas Woman’s University, and a M.S. in Electrical Engineering from Texas Tech University. Prior to beginning her doctoral studies, she worked as a microelectromechanical-systems (MEMS) engineer for Texas Instruments.

visit author page

biography

Senay Purzer Purdue University, West Lafayette orcid.org/0000-0003-0784-6079

visit author page

Purzer is an Assistant Professor in the School of Engineering Education and is the Co-Director of Assessment Research for the Institute for P-12 Engineering Research and Learning (INSPIRE) at Purdue University. Purzer has received her M.A. and Ph.D. degrees in Science Education at Arizona State University. She has a B.S. degree in Physics Education and a B.S.E. in Engineering. She has journal publications on instrument development, teacher professional development, and K-12 engineering education. Her creative research focuses on design problem-solving, collaborative learning, and assessment research.

visit author page

author page

Michael Fosmire Purdue University Libraries, West Lafayette orcid.org/0000-0002-5138-8178

biography

Monica E. Cardella Purdue University, West Lafayette orcid.org/0000-0002-4229-6183

visit author page

Monica E. Cardella is an Assistant Professor of Engineering Education and is the Co-Director of Assessment Research for the Institute for P-12 Engineering Research and Learning (INSPIRE) at Purdue University. Dr. Cardella earned a B.Sc. in Mathematics from the University of Puget Sound and an M.S. and Ph.D. in Industrial Engineering at the University of Washington. At the University of Washington she worked with the Center for Engineering Learning and Teaching (CELT) and the LIFE Center (Learning in Informal and Formal Environments). She was a CASEE Postdoctoral Engineering Education Researcher at the Center for Design Research at Stanford before beginning her appointment at Purdue. Her research interests include: learning in informal and out-of-school time settings, pre-college engineering education, design thinking, mathematical thinking, and assessment research.

visit author page

Download Paper | Permalink

Abstract

Are Engineering Students Life-Long Learners?: Results from A Two-Tier Multiple-Choice InstrumentOne of the most important skills students can take away from a technical education is the abilityto become curious, persistent, and life-long learners, and information literacy is crucial todeveloping those skills. This aligns with ABET accreditation criterion 3.i. Few standard, easilygradable assessment tools exist to assess the information literacy components of lifelong learningskills, especially in the context of an engineering education. The existing instruments generallyinvolve self-reports rather than objectively measuring life-long learning skills. In this paper, wepresent results from a two-tiered multiple choice instrument designed to quickly measurerelevant information literacy competencies. We collected data from first-year engineeringstudents (N=366), who were asked to read a technical memo and, based on the memo’sarguments, answer eight multiple choice and two open-ended response questions. While theaverage score on the multiple choice portion was 43%, the majority (96%) identified the mainpoint of the memo correctly. However, only 49% of students were able to correctly identify theassumptions and unreliable sources used in the memo, despite identifying, in the open-endedresponses, assumptions and unreliable sources as reasons why certain answers were incorrect. Aqualitative analysis of the open-ended responses provides more detailed insights into students’abilities to identify and resolve conflicts between information sources, evaluate the reliabilityand relevancy of information sources, and use reliable information sources. Additional conceptsprobed in the multiple choice questions assess student ability to: identify implicit and explicitassumptions, evaluate overall quality of a written document, extract the main points from theinformation sources, and to interpret and present information accurately. The qualitativeresponses support the quantitative scores: information literacy skills among first-yearengineering students need to improve for students to become competent lifelong learners.

Citation
Format

Wertz, R., & Ross, M. C., & Purzer, S., & Fosmire, M., & Cardella, M. E. (2011, June), Assessing Engineering Students’ Information Literacy Skills: An Alpha Version of a Multiple-Choice Instrument Paper presented at 2011 ASEE Annual Conference & Exposition, Vancouver, BC. 10.18260/1-2--17518

TY - CPAPER
AB - Are Engineering Students Life-Long Learners?: Results from A Two-Tier Multiple-Choice InstrumentOne of the most important skills students can take away from a technical education is the abilityto become curious, persistent, and life-long learners, and information literacy is crucial todeveloping those skills. This aligns with ABET accreditation criterion 3.i. Few standard, easilygradable assessment tools exist to assess the information literacy components of lifelong learningskills, especially in the context of an engineering education. The existing instruments generallyinvolve self-reports rather than objectively measuring life-long learning skills. In this paper, wepresent results from a two-tiered multiple choice instrument designed to quickly measurerelevant information literacy competencies. We collected data from first-year engineeringstudents (N=366), who were asked to read a technical memo and, based on the memo’sarguments, answer eight multiple choice and two open-ended response questions. While theaverage score on the multiple choice portion was 43%, the majority (96%) identified the mainpoint of the memo correctly. However, only 49% of students were able to correctly identify theassumptions and unreliable sources used in the memo, despite identifying, in the open-endedresponses, assumptions and unreliable sources as reasons why certain answers were incorrect. Aqualitative analysis of the open-ended responses provides more detailed insights into students’abilities to identify and resolve conflicts between information sources, evaluate the reliabilityand relevancy of information sources, and use reliable information sources. Additional conceptsprobed in the multiple choice questions assess student ability to: identify implicit and explicitassumptions, evaluate overall quality of a written document, extract the main points from theinformation sources, and to interpret and present information accurately. The qualitativeresponses support the quantitative scores: information literacy skills among first-yearengineering students need to improve for students to become competent lifelong learners.
AU - Ruth Wertz P.E.
AU - Meagan C. Ross
AU - Senay Purzer
AU - Michael Fosmire
AU - Monica E. Cardella
CY - Vancouver, BC
DA - 2011/06/26
PB - ASEE Conferences
TI - Assessing Engineering Students’ Information Literacy Skills: An Alpha Version of a Multiple-Choice Instrument
UR - https://peer.asee.org/17518
DO - 10.18260/1-2--17518
ER -