Memarian, B., & McCahan, S. (2020, June), Outcomes-based Assessment Instrument for Engineering Problem-solving Skills  Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual On line . 10.18260/1-2--35023

\bibitem{asee_peer_35023}
  Memarian, B., \& McCahan, S. (2020, June), \emph{Outcomes-based Assessment Instrument for Engineering Problem-solving Skills}
  Paper presented at 2020 ASEE Virtual Annual Conference Content Access, Virtual On line .
  10.18260/1-2--35023

Bahar Memarian and Susan McCahan.  "Outcomes-based Assessment Instrument for Engineering Problem-solving Skills".  2020 ASEE Virtual Annual Conference Content Access, Virtual On line , 2020, June.  ASEE Conferences, 2020.  https://peer.asee.org/35023 Internet.  13 Mar, 2025

\bibitem{asee_peer_35023}
  Bahar Memarian and Susan McCahan.
  "Outcomes-based Assessment Instrument for Engineering Problem-solving Skills".
  \emph{2020 ASEE Virtual Annual Conference Content Access, Virtual On line , 2020, June}.
  ASEE Conferences, 2020.
  https://peer.asee.org/35023 Internet.  13 Mar, 2025

@INPROCEEDINGS{asee_peer_35023
author = "Bahar Memarian and Susan McCahan"
title = "Outcomes-based Assessment Instrument for Engineering Problem-solving Skills"
booktitle = "2020 ASEE Virtual Annual Conference Content Access"
year = "2020"
month = "June"
address = "Virtual On line "
publisher = "ASEE Conferences"
note = {https://peer.asee.org/35023}
number = {10.18260/1-2--35023}
}

TY  - CPAPER
AB  - The work presented is intended for a poster presentation. In assessment of engineering problem solving skill, grading schemes and sometimes rubrics are used. Common notion of reliability is largely based on the inter-rater rating (IRR), rather than inter-coder rating (ICR) of the feedback delivered. Evaluation culture is however shifting from a measurement of knowledge perspective to an assessment for learning domain. Reliability should therefore be understood in the realm of this new domain and through ICR methods. The ICR of data collected on evaluations made by a novel formative feedback instrument is investigated in this study. This instrument was previously proposed and tested by the authors and the findings are drawn from the same data set. The ICR of the instrument’s indicators are analyzed. In addition, the correlations between each of solution quality and assessor sample size is examined against the ICR of instrument’s indicators. Data used was collected from counter-balanced simulated evaluation trials (n = 33 assessors). The collected data is coded and grouped based on identical solution profiles. Following findings of literature, only solution profiles having percentage agreement of 55% or higher are used in the analysis. Findings show that 4 out of 6 indicators of CAIR have a percentage agreement of 55% or higher across a large collection of evaluated electrical engineering solution profiles examined. This is in agreement with literature which reports ICR of 55% to 75% as acceptable. The results of correlational analysis confirms that percentage agreement on most of instrument’s criteria is uncorrelated with both assessor sample size and quality of solutions.
AU  - Bahar Memarian
AU  - Susan McCahan
CY  - Virtual On line 
DA  - 2020/06/22
PB  - ASEE Conferences
TI  - Outcomes-based Assessment Instrument for Engineering Problem-solving Skills
UR  - https://peer.asee.org/35023
DO  - 10.18260/1-2--35023
ER  -