Asee peer logo

Utilizing Think-Aloud Protocols to Assess the Usability of a Test for Ethical Sensitivity in Construction

Download Paper |


2014 ASEE Annual Conference & Exposition


Indianapolis, Indiana

Publication Date

June 15, 2014

Start Date

June 15, 2014

End Date

June 18, 2014



Conference Session

Methodological & Theoretical Contributions to Engineering Education 2

Tagged Division

Educational Research and Methods

Page Count


Page Numbers

24.1355.1 - 24.1355.16



Permanent URL

Download Count


Request a correction

Paper Authors


Kenneth Stafford Sands II Virginia Tech

visit author page

Kenneth S. Sands II is a doctoral candidate and graduate assistant in Environmental Design and Planning at Virginia Tech. His research focus is on professional ethics and its pedagogy.

visit author page


Denise Rutledge Simmons Virginia Tech Orcid 16x16

visit author page

Denise R. Simmons, Ph.D., is an assistant professor in the Myers-Lawson School of Construction & Civil and Environmental Engineering at Virginia Polytechnic Institute and State University. She holds a B.S., M.S., and Ph.D. in civil engineering and a graduate certificate in engineering education – all from Clemson University. Until 2012, she was the director of the Savannah River Environmental Sciences Field Station. Dr. Simmons has nearly fourteen years of engineering and project management experience working with public utility companies, a project management consulting company, and a software company. She is a registered professional engineer, project management professional and LEED accredited professional.. Her research interests are in investigating students' development of leadership skills and other professional competencies and in student involvement in co-curricular activities. Dr. Simmons is a NSF CAREER award winner for her research entitled, “Investigating Co-Curricular Participation of Students Underrepresented in Engineering.”

visit author page

Download Paper |


Utilizing a Think-Aloud Protocol to Assess the Usability of a Test for Ethical Sensitivity in ConstructionERM Paper Type: OtherProfessional societies such as the American Society of Civil Engineers (ASCE) and theConstruction Management Association of America have codes of ethics to guide constructionpractitioners on professional practice. Additionally, ABET accredited civil and constructionengineering programs are expected to provide professional ethics instruction to its students tomeet the standards established by the profession. However, no assessment tool measuring astudent’s ethical sensitivity to professional issues in construction exists to determine whether thisinstruction improves student recognition of ethical issues in construction. The purpose of thispaper is to outline the use of a think aloud protocol as a part of the instrument development of atest for ethical sensitivity in construction (TESC). The paper details the process taken to developand administer the think aloud protocol, the results of the think aloud protocol, and how theresults assisted refinement of the TESC.The TESC is a structured qualitative assessment instrument that requires student participants toread, recognize, and list ethical issues embedded in vignettes. Vignettes are helpful inunderstanding a person's attitudes, perceptions, and beliefs. The “think-aloud” protocol allowedfor usability evaluation of the TESC by requiring participants to verbalize their thoughts,processes, and reasoning as they complete the items of the instrument. Prior research hasassociated the think aloud administration process with cognitive validation however, it alsoallows researchers to evaluate the affective processing of students when completing the test.Different from other uses of the think aloud protocol as a tool of verbal cognitive validation, thethink aloud protocol is also used as a means to understanding both the cognitive response andaffective responses of student participants of the TESC, observing their moral imagination,cognitive recognition and affective processing.Results from administering the think aloud protocol assisted the TESC’s clarity, administration,and approach to scoring participants’ responses. Problems of the original TESC included itssimplicity to complete and ease of understanding vignettes. The think aloud providedunderstanding of administration parameters of the test, which included time ofadministration/completion of the TESC, and strain required of participants to complete theTESC. The think aloud added qualitative understanding of responses and assisted the authors indeveloping an argument for construct validity. Additionally, the think aloud protocol providedadditional means of items analysis. Lastly, which is considerably important to the researchers,the think aloud assisted in identifying what qualitative responses generated by participants‘actually’ meant, adding to refinement of the scoring manual for the TESC. The authorsanticipate that results will assist researchers in understanding the benefits of using think aloudprotocols for refining an instrument, while adding validity arguments.

Sands, K. S., & Simmons, D. R. (2014, June), Utilizing Think-Aloud Protocols to Assess the Usability of a Test for Ethical Sensitivity in Construction Paper presented at 2014 ASEE Annual Conference & Exposition, Indianapolis, Indiana. 10.18260/1-2--23288

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2014 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015