Exploring Undergraduate Students’ Computational Literacy in the Context of Problem Solving

Camilo Vieira; Alejandra J. Magana; Anindya Roy; Michael L. Falk; Michael J. Reese

Download Paper | Permalink

Conference: 2015 ASEE Annual Conference & Exposition
Location: Seattle, Washington
Publication Date: June 14, 2015
Start Date: June 14, 2015
End Date: June 17, 2015
ISBN: 978-0-692-50180-1
ISSN: 2153-5965
Conference Session: Simulation
Tagged Division: Computers in Education
Page Count: 16
Page Numbers: 26.744.1 - 26.744.16
DOI: 10.18260/p.24081
Permanent URL: https://peer.asee.org/24081
Download Count: 609

Paper Authors

biography

Camilo Vieira Purdue University orcid.org/0000-0001-8720-0002

visit author page

Third year PhD student in Computing Education - Purdue University
Master of Engineering in Educational Technologies - Eafit University
Systems Engineer - Eafit University

visit author page

biography

Alejandra J. Magana Purdue University, West Lafayette orcid.org/0000-0001-6117-7502

visit author page

Alejandra Magana is an Assistant Professor in the Department of Computer and Information Technology and an affiliated faculty at the School of Engineering Education at Purdue University. She holds a B.E. in Information Systems, a M.S. in Technology, both from Tec de Monterrey; and a M.S. in Educational Technology and a Ph.D. in Engineering Education from Purdue University. Her research is focused on identifying how model-based cognition in STEM can be better supported by means of expert technological and computing tools such as cyberinfrastructure, cyber-physical systems, and computational modeling and simulation tools.

visit author page

author page

Anindya Roy Johns Hopkins University

biography

Michael L. Falk Johns Hopkins University orcid.org/0000-0002-8383-4259

visit author page

Michael Falk is a Professor in the Department of Materials Science and Engineering at Johns Hopkins University's Whiting School of Engineering where he has served on the faculty since 2008 with secondary appointments in Mechanical Engineering and in Physics and Astronomy. He holds a B.A. in Physics (1990) and a M.S.E. in Computer Science (1991) from Johns Hopkins University and a Ph.D. in Physics (1998) from the University of California, Santa Barbara. He has been twice selected as a visiting Chaire Joliot at the École Supérieure de Physique et de Chimie Industrielles at Paris Tech and has organized extended workshops on the physics of glasses and on friction, fracture and earthquakes at the Kavli Institute for Theoretical Physics. He has received several awards for his educational accomplishments, and in 2011 he received an award from the university's Diversity Leadership Council for his work on LGBT inclusion. His education research focuses on integrating computation into the undergraduate core curriculum. Falk also serves as the lead investigator for STEM Achievement in Baltimore Elementary Schools (SABES) an NSF funded Community Enterprise for STEM Learning partnership between JHU and Baltimore City Schools.

visit author page

author page

Michael J. Reese Jr. Johns Hopkins University

Download Paper | Permalink

Abstract

Exploring Undergraduate Students’ Computational Literacy in the Context of Problem SolvingBackgroundComputational Science and Engineering (CSE) is a modern practice in workplace engineeringused to solve complex problems in disciplinary areas using computational tools1. In educationalsettings, there has been a call to integrate computational tools and methods into the differentdisciplinary engineering curricula sooner and often2. In Materials Science and Engineering, a subdiscipline has been established called Computational Materials Science3. Attending this call, thedepartment of Materials Science and Engineering at a Mid-Atlantic University started acurricular innovation consisting of one computational course for material scientists (CPMSE)and the integration of several computational modules into subsequent six core courses.This study focuses on understanding how computational tools and methods were applied duringthe problem solving episodes to support the solution of computational challenges in a freshmencourse. The research questions addressed in this study are: ● How does students’ performance in solving computational challenges relate to their disciplinary learning outcomes in a computational material science course? ● How do specific steps of the problem solving process relate to student overall performance in a computational materials science course? ● How does students’ problem solving approaches relate to students’ programming skills?MethodsThe CPMSE course was designed using the How People Learn framework4. Hence, it isKnowledge Centered, Learner Centered, and Community Centered. It uses MATLAB as theprogramming environment and the learning objectives are5: (1) Write MATLAB programs to execute well-deﬁned algorithms. (2) Design algorithms to solve engineering problems by breaking these into small tractable parts. (3) Model physical and biological systems by applying linear systems and ordinary and partial differential equationsThe course employs an inverted classroom approach where students are required to watch therecorded lectures before coming to class for practice exercises. During the semester, the studentsare required to complete five computational projects related to their core courses. The projectswere organized using the four steps problem solving process6: (1) Understand the Problem; (2)Devise a Plan; (3) Carry out the Plan; and (4) Review/Extend. The disciplinary andcomputational learning outcomes for each of the projects are described in Appendix A.Twenty three freshmen and sophomore material science and engineering students enrolled in theCPMSE course in spring 2014 participated in the study. The participants solved fivecomputational projects following the four problem solving steps. The projects were evaluatedusing an assessment rubric presented in Appendix B. The rubric has three major components: (1)Problem Solving Steps; (2) Disciplinary Outcomes; and (3) Computational Literacy. Theseindividual components and each rubric criteria were first analyzed using descriptive statistics.Afterwards, the three components were correlated among each other and to project scores, andcourse score. This process was carried out for the individual projects and for the averageperformance in each of the rubric criteria.ResultsTable 1 depicts average scores for all projects grouped by the three different criteria. The highestscores were obtained in the computational literacy skills constructs for all the projects but projectone. This result suggests that students took some time to acquire these skills, but they were ableto use them in different disciplinary contexts. Table 1. Descriptive Statistics of student overall scores for each computational project Problem Solving Steps Disciplinary Computational Literacy Outcomes Skills Planning/Program Specification Deploy Coding Execution Total Design (%) (%) Concepts (%) Style (%) (%) (%) Project 1 (N=21) Mean 87.81 59.20 100 84.05 66.81 74.99 Std. Dev. 22.49 30.47 0 20.36 31.21 19.11 Project 2 (N=21) Mean 100 98.43 96.62 100 96.10 97.68 Std. Dev. 0 3.84 4.79 0 9.05 3.14 Project 3 (N=21) Mean 72.24 76.05 100 86.29 91.57 86.14 Std. Dev. 30.36 25.41 0 21.18 22.76 14.14 Project 4 (N=21) Mean 68.95 97.57 89.90 98.24 98.29 93.46 Std. Dev. 44.38 5.76 22.10 2.82 5.71 8.21 Project 5 (N=20) Mean 69.5 77.40 75.95 94 91.25 81.72 Std. Dev. 45.55 16.44 33.75 6.05 17.63 17.84Table 2 shows the Pearson correlation for the three major components of the rubric as well as the finalproject score. Results suggest that in average, problem solving is closely related to the computationalliteracy skills. These two constructs are also determinants for the total projects scores and for the coursescore. Table 3. Pearson correlation for rubric criteria Problem Disciplinary Computational Total Course Solving Steps Outcomes Literacy Skills Project Score Problem Solving 1 Steps Disciplinary 0.27 1 Outcomes Computational 0.86 0.09 1 Literacy Skills Total Project 0.97 0.28 0.94 1 Course Score 0.83 0.08 0.88 0.87 1Conclusions and SignificanceThe significance of this study lies on the understanding of the extent computational literacysupports or hinders problem solving processes. Preliminary analysis suggests that students maybe able to acquire and maintain computational literacy skills throughout a computationaldisciplinary course. Another relevant result suggests that there is a close relationship betweenadequately following the problem solving steps and writing an organized program that executescorrectly.References1 Turner, P., Petzold, L., Shiflet, A., Vakalis, I., Jordan, K., & St. John, S. Undergraduate computational science and engineering education. Society for Industrial and Applied Mathematics Review (SIAM Rev.), 53, pp. 561-574 (2011).2 NSF. National Science Foundation Advisory Committee for Cyberinfrastructure Task Force on Grand Challenges Final Report (2011).3 Hafner, J. Atomic-scale computational materials science. Acta Materialia 48, 71–92 (2000)4 Bransford, J. How People Learn: Brain, Mind, Experience, and School. National Academies Press (2000)5 Magana, A. J., Falk, M. L., Reese, JR. Introducing Discipline-Based Computing in Undergraduate Engineering Education (2013)6 Pólya, G. How to Solve It. Garden City, NY: Doubleday (1957). Appendix A Computational and disciplinary learning outcomes per project Computational Learning Disciplinary Learning Outcomes OutcomesProject 1  The student demonstrates the ability to apply the  The student graphically represents and techniques of modeling and simulation to a range of calculates the phases present in a binary problem areas. phase diagram.  The student uses MATLAB to create visual displays of data, including graphs, charts, tables, and histograms.Project 2  The student implements algorithms for solving  The student models for the progress of the differential equations. HIV infection in a patient that is being treated  The student models biological systems by applying with a drug of a given effectiveness linear systems and ordinary and partial differential equationsProject 3  The student demonstrates the ability to apply the  The student models crystal structures cleaved techniques of modeling and simulation to a range of along a plane and generates a three- problem areas. dimensional representation of them.  The student uses MATLAB to create visual displays of data, including graphs, charts, tables, and histogramsProject 4  The student models biological systems by applying  The student simulates the cardiac tissue and linear systems and ordinary and partial differential the ventricular fibrillation process equations  The student demonstrates the ability to apply the techniques of modeling and simulation to a range of problem areas.Project 5  The student demonstrates the ability to apply the  The student creates and interprets a molecular techniques of modeling and simulation to a range of dynamic simulation in terms of kinetic problem areas. energies. Appendix B Rubric for Project and Application CPMSE Component STUDENT ID: Poor (0-2) … Excellent (9-10) Planning/Program Design (10%) - No strategy is - All four areas (designing, Evaluates the student’s plan for completing articulated for the coding, testing, debugging) are the project. design, coding, addressed clearly in the context Student instructions: testing or debugging. of the project. Summarize the nature of the algorithm - The summary references the … briefly, identifying the most relevant project description and identifies information from the project description. relevant aspects of the project. Articulate a well thought-out strategy for - The strategy is articulated Problem designing, coding, testing and debugging clearly and is logical and well Solving your work thought-out. Specification satisfaction (30%) - The solution - The solution produces correct Evaluates the degree to which the solution produces wholly output in all cases with only satisfies the specification. incorrect output minor exceptions. Is the solution accurate and robust? under all of the tests … - All output meets specifications Does it conform to the problem run. regarding format, order and specifications regarding format, order and presentation. presentation? Deployment of Disciplinary Concepts - No solution - A solution is provided that is (20%) provided. correct, clear and well Evaluates whether the student can use the documented. Disciplinary solution to approach a disciplinary problem. … Outcomes Can the student use their code to address the disciplinary issue or to solve a related problem? Coding Style (10%) - Code is entirely - Code is well commented. Measures the extent to which the code is uncommented. - Code is properly indented and presented in a manner that is clearly - Global variables are variable and function names are readable by others. used without well chosen. Is the code indented, commented and are justification due to - Code is well structured. variable and function names chosen to exceptional … enhance readability? circumstances. Does the code appropriately deploy - Code is not language capabilities to avoid redundant differentiated intoComputational structures, global variables and functions or m-files;Literacy Skills unnecessarily lengthy blocks of code? i.e. spaghetti code. Program Execution (30%) - Program does not - Program is free of syntax errors Evaluates the extent to which the program run at all. that impede execution. functions in a way that conforms to - Program takes the expected specifications. … input parameters and returns the Does the program execute? expected output as required in Is the input and output of the expected the specification in all respects. form?

Citation
Format

Vieira, C., & Magana, A. J., & Roy, A., & Falk, M. L., & Reese, M. J. (2015, June), Exploring Undergraduate Students’ Computational Literacy in the Context of Problem Solving Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.24081

TY - CPAPER
AB - Exploring Undergraduate Students’ Computational Literacy in the Context of Problem SolvingBackgroundComputational Science and Engineering (CSE) is a modern practice in workplace engineeringused to solve complex problems in disciplinary areas using computational tools1. In educationalsettings, there has been a call to integrate computational tools and methods into the differentdisciplinary engineering curricula sooner and often2. In Materials Science and Engineering, a subdiscipline has been established called Computational Materials Science3. Attending this call, thedepartment of Materials Science and Engineering at a Mid-Atlantic University started acurricular innovation consisting of one computational course for material scientists (CPMSE)and the integration of several computational modules into subsequent six core courses.This study focuses on understanding how computational tools and methods were applied duringthe problem solving episodes to support the solution of computational challenges in a freshmencourse. The research questions addressed in this study are: ● How does students’ performance in solving computational challenges relate to their disciplinary learning outcomes in a computational material science course? ● How do specific steps of the problem solving process relate to student overall performance in a computational materials science course? ● How does students’ problem solving approaches relate to students’ programming skills?MethodsThe CPMSE course was designed using the How People Learn framework4. Hence, it isKnowledge Centered, Learner Centered, and Community Centered. It uses MATLAB as theprogramming environment and the learning objectives are5: (1) Write MATLAB programs to execute well-deﬁned algorithms. (2) Design algorithms to solve engineering problems by breaking these into small tractable parts. (3) Model physical and biological systems by applying linear systems and ordinary and partial differential equationsThe course employs an inverted classroom approach where students are required to watch therecorded lectures before coming to class for practice exercises. During the semester, the studentsare required to complete five computational projects related to their core courses. The projectswere organized using the four steps problem solving process6: (1) Understand the Problem; (2)Devise a Plan; (3) Carry out the Plan; and (4) Review/Extend. The disciplinary andcomputational learning outcomes for each of the projects are described in Appendix A.Twenty three freshmen and sophomore material science and engineering students enrolled in theCPMSE course in spring 2014 participated in the study. The participants solved fivecomputational projects following the four problem solving steps. The projects were evaluatedusing an assessment rubric presented in Appendix B. The rubric has three major components: (1)Problem Solving Steps; (2) Disciplinary Outcomes; and (3) Computational Literacy. Theseindividual components and each rubric criteria were first analyzed using descriptive statistics.Afterwards, the three components were correlated among each other and to project scores, andcourse score. This process was carried out for the individual projects and for the averageperformance in each of the rubric criteria.ResultsTable 1 depicts average scores for all projects grouped by the three different criteria. The highestscores were obtained in the computational literacy skills constructs for all the projects but projectone. This result suggests that students took some time to acquire these skills, but they were ableto use them in different disciplinary contexts. Table 1. Descriptive Statistics of student overall scores for each computational project Problem Solving Steps Disciplinary Computational Literacy Outcomes Skills Planning/Program Specification Deploy Coding Execution Total Design (%) (%) Concepts (%) Style (%) (%) (%) Project 1 (N=21) Mean 87.81 59.20 100 84.05 66.81 74.99 Std. Dev. 22.49 30.47 0 20.36 31.21 19.11 Project 2 (N=21) Mean 100 98.43 96.62 100 96.10 97.68 Std. Dev. 0 3.84 4.79 0 9.05 3.14 Project 3 (N=21) Mean 72.24 76.05 100 86.29 91.57 86.14 Std. Dev. 30.36 25.41 0 21.18 22.76 14.14 Project 4 (N=21) Mean 68.95 97.57 89.90 98.24 98.29 93.46 Std. Dev. 44.38 5.76 22.10 2.82 5.71 8.21 Project 5 (N=20) Mean 69.5 77.40 75.95 94 91.25 81.72 Std. Dev. 45.55 16.44 33.75 6.05 17.63 17.84Table 2 shows the Pearson correlation for the three major components of the rubric as well as the finalproject score. Results suggest that in average, problem solving is closely related to the computationalliteracy skills. These two constructs are also determinants for the total projects scores and for the coursescore. Table 3. Pearson correlation for rubric criteria Problem Disciplinary Computational Total Course Solving Steps Outcomes Literacy Skills Project Score Problem Solving 1 Steps Disciplinary 0.27 1 Outcomes Computational 0.86 0.09 1 Literacy Skills Total Project 0.97 0.28 0.94 1 Course Score 0.83 0.08 0.88 0.87 1Conclusions and SignificanceThe significance of this study lies on the understanding of the extent computational literacysupports or hinders problem solving processes. Preliminary analysis suggests that students maybe able to acquire and maintain computational literacy skills throughout a computationaldisciplinary course. Another relevant result suggests that there is a close relationship betweenadequately following the problem solving steps and writing an organized program that executescorrectly.References1 Turner, P., Petzold, L., Shiflet, A., Vakalis, I., Jordan, K., &amp; St. John, S. Undergraduate computational science and engineering education. Society for Industrial and Applied Mathematics Review (SIAM Rev.), 53, pp. 561-574 (2011).2 NSF. National Science Foundation Advisory Committee for Cyberinfrastructure Task Force on Grand Challenges Final Report (2011).3 Hafner, J. Atomic-scale computational materials science. Acta Materialia 48, 71–92 (2000)4 Bransford, J. How People Learn: Brain, Mind, Experience, and School. National Academies Press (2000)5 Magana, A. J., Falk, M. L., Reese, JR. Introducing Discipline-Based Computing in Undergraduate Engineering Education (2013)6 Pólya, G. How to Solve It. Garden City, NY: Doubleday (1957). Appendix A Computational and disciplinary learning outcomes per project Computational Learning Disciplinary Learning Outcomes OutcomesProject 1  The student demonstrates the ability to apply the  The student graphically represents and techniques of modeling and simulation to a range of calculates the phases present in a binary problem areas. phase diagram.  The student uses MATLAB to create visual displays of data, including graphs, charts, tables, and histograms.Project 2  The student implements algorithms for solving  The student models for the progress of the differential equations. HIV infection in a patient that is being treated  The student models biological systems by applying with a drug of a given effectiveness linear systems and ordinary and partial differential equationsProject 3  The student demonstrates the ability to apply the  The student models crystal structures cleaved techniques of modeling and simulation to a range of along a plane and generates a three- problem areas. dimensional representation of them.  The student uses MATLAB to create visual displays of data, including graphs, charts, tables, and histogramsProject 4  The student models biological systems by applying  The student simulates the cardiac tissue and linear systems and ordinary and partial differential the ventricular fibrillation process equations  The student demonstrates the ability to apply the techniques of modeling and simulation to a range of problem areas.Project 5  The student demonstrates the ability to apply the  The student creates and interprets a molecular techniques of modeling and simulation to a range of dynamic simulation in terms of kinetic problem areas. energies. Appendix B Rubric for Project and Application CPMSE Component STUDENT ID: Poor (0-2) … Excellent (9-10) Planning/Program Design (10%) - No strategy is - All four areas (designing, Evaluates the student’s plan for completing articulated for the coding, testing, debugging) are the project. design, coding, addressed clearly in the context Student instructions: testing or debugging. of the project. Summarize the nature of the algorithm - The summary references the … briefly, identifying the most relevant project description and identifies information from the project description. relevant aspects of the project. Articulate a well thought-out strategy for - The strategy is articulated Problem designing, coding, testing and debugging clearly and is logical and well Solving your work thought-out. Specification satisfaction (30%) - The solution - The solution produces correct Evaluates the degree to which the solution produces wholly output in all cases with only satisfies the specification. incorrect output minor exceptions. Is the solution accurate and robust? under all of the tests … - All output meets specifications Does it conform to the problem run. regarding format, order and specifications regarding format, order and presentation. presentation? Deployment of Disciplinary Concepts - No solution - A solution is provided that is (20%) provided. correct, clear and well Evaluates whether the student can use the documented. Disciplinary solution to approach a disciplinary problem. … Outcomes Can the student use their code to address the disciplinary issue or to solve a related problem? Coding Style (10%) - Code is entirely - Code is well commented. Measures the extent to which the code is uncommented. - Code is properly indented and presented in a manner that is clearly - Global variables are variable and function names are readable by others. used without well chosen. Is the code indented, commented and are justification due to - Code is well structured. variable and function names chosen to exceptional … enhance readability? circumstances. Does the code appropriately deploy - Code is not language capabilities to avoid redundant differentiated intoComputational structures, global variables and functions or m-files;Literacy Skills unnecessarily lengthy blocks of code? i.e. spaghetti code. Program Execution (30%) - Program does not - Program is free of syntax errors Evaluates the extent to which the program run at all. that impede execution. functions in a way that conforms to - Program takes the expected specifications. … input parameters and returns the Does the program execute? expected output as required in Is the input and output of the expected the specification in all respects. form?
AU - Camilo Vieira
AU - Alejandra J. Magana
AU - Anindya Roy
AU - Michael L. Falk
AU - Michael J. Reese Jr.
CY - Seattle, Washington
DA - 2015/06/14
PB - ASEE Conferences
TI - Exploring Undergraduate Students’ Computational Literacy in the Context of Problem Solving
UR - https://peer.asee.org/24081
DO - 10.18260/p.24081
ER -