Asee peer logo

FPAvisual: A Tool for Visualizing the Effects of Floating-Point Finite-Precision Arithmetic

Download Paper |


2014 ASEE Annual Conference & Exposition


Indianapolis, Indiana

Publication Date

June 15, 2014

Start Date

June 15, 2014

End Date

June 18, 2014



Conference Session

Computers in Education General Technical Session 2

Tagged Division

Computers in Education

Page Count


Page Numbers

24.627.1 - 24.627.19



Permanent URL

Download Count


Request a correction

Paper Authors


Yi Gu Michigan Technological University

visit author page

Yi Gu is a PhD candidate in Computer Science at Michigan Technological University and received his MSc Degree in 2010. His main research interest area is the development of tools that help scientists understand the underlying relations in scientific data and aid students understand concepts through visualizations. Further information can be found at:

visit author page


Nilufer Onder Michigan Technological University

visit author page

Nilufer Onder is an associate professor of computer science at Michigan Technological University. She received her PhD from the Computer Science Department at the University of Pittsburgh. Her research areas are artificial intelligence planning systems, decision making under uncertainty, student persistence, and computer science education. She received the Michigan Tech university wide Distinguished Teaching Award in 2013. Her web page can be found via

visit author page

author page

Ching-Kuang Shene Michigan Technological University


Chaoli Wang Michigan Technological University

visit author page

Chaoli Wang is an assistant professor at the Department of Computer Science, Michigan Technological University. He received the PhD degree in computer and information science from The Ohio State University in 2006. From 2007 to 2009, he was a postdoctoral researcher at the University of California, Davis. Dr. Wang's research interests include scientific visualization, large-scale data analysis and visualization, user interface and interaction, information visualization, and visualization in education. At Michigan Tech, he co-directs the Graphics and Visualization Laboratory and participates in research activities at the Immersive Visualization Studio. His research has been supported by Michigan Technological University, the National Science Foundation, and the Dave House Family Foundation.

visit author page

Download Paper |


FPAvisual: A Tool for Visualizing the Effects of Floating-Point Finite-Precision ArithmeticAbstract:Many students in science and engineering do not realize how program correctness may beimpacted when floating-point finite precision arithmetic is used. In this paper, we presentFPAvisual, a visualization tool that helps instructors teach the reasons for the inaccuracies causedby floating-point arithmetic (FPA), their impact and significance in programs, and the techniquesto prevent errors from propagating. FPAvisual contains four components, namely Associative Law,Pentagon, Roots, and the Sine Function. Associative Law demonstrates how algebraicallyequivalent formulas computed by changing the order of operations can yield different results.Pentagon demonstrates that accumulation of errors emanating from finite precision in geometriccomputations may result in large positional errors. Roots shows that the solution for a quadraticequation will be incorrect when two numbers that need to be subtracted are very close inmagnitude or when one is much larger than the other. The program presents possible solutions tothese subtraction problems. The Sine Function component shows that results vary when the sameinfinite series for sine is used but computed in different ways. These four components allow theusers set up parameters of the specific problem represented, trace the results step by step, seewhen differences in results start to occur, and visualize how errors accumulate. They help studentsunderstand the ubiquity of issues with FPA, realize the significance of FPA in a multitude ofcontexts, and compare the methods to minimize the negative effects of FPA. FPAvisual has beenclassroom tested and evaluated by computer science students. We report our findings in this paper.

Gu, Y., & Onder, N., & Shene, C., & Wang, C. (2014, June), FPAvisual: A Tool for Visualizing the Effects of Floating-Point Finite-Precision Arithmetic Paper presented at 2014 ASEE Annual Conference & Exposition, Indianapolis, Indiana. 10.18260/1-2--20518

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2014 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015