Frank, B. M., & Kaupp, J. A., & Simper, N., & Scott, J. (2015, June), Multi-method Longitudinal Assessment of Transferrable Intellectual Learning Outcomes  Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.24513

\bibitem{asee_peer_24513}
  Frank, B. M., \& Kaupp, J. A., \& Simper, N., \& Scott, J. (2015, June), \emph{Multi-method Longitudinal Assessment of Transferrable Intellectual Learning Outcomes}
  Paper presented at 2015 ASEE Annual Conference \& Exposition, Seattle, Washington.
  10.18260/p.24513

Brian M Frank, James A. Kaupp, Natalie Simper, and Jill Scott.  "Multi-method Longitudinal Assessment of Transferrable Intellectual Learning Outcomes".  2015 ASEE Annual Conference & Exposition, Seattle, Washington, 2015, June.  ASEE Conferences, 2015.  https://peer.asee.org/24513 Internet.  28 Dec, 2024

\bibitem{asee_peer_24513}
  Brian M Frank, James A. Kaupp, Natalie Simper, and Jill Scott.
  "Multi-method Longitudinal Assessment of Transferrable Intellectual Learning Outcomes".
  \emph{2015 ASEE Annual Conference \& Exposition, Seattle, Washington, 2015, June}.
  ASEE Conferences, 2015.
  https://peer.asee.org/24513 Internet.  28 Dec, 2024

@INPROCEEDINGS{asee_peer_24513
author = "Brian M Frank, James A. Kaupp, Natalie Simper, and Jill Scott"
title = "Multi-method Longitudinal Assessment of Transferrable Intellectual Learning Outcomes"
booktitle = "2015 ASEE Annual Conference \& Exposition"
year = "2015"
month = "June"
address = "Seattle, Washington"
publisher = "ASEE Conferences"
note = {https://peer.asee.org/24513}
number = {10.18260/p.24513}
}

TY  - CPAPER
AB  -      Multi-method longitudinal assessment of transferrable intellectual learning                                     outcomes[University  X,  (blinded)]  is  part  of  a  six-­‐institution  consortium  in  Canada  committed  to  developing  assessment   techniques   for   generic   learning   outcomes   and   cognitive   skills.   These   skills,   including  critical   thinking,   problem   solving,   communication   and   lifelong   learning,   are   the   subject   of  discussion   in   higher   education   generally   (e.g.   the   American   Associate   of   Colleges   and   Universities  (AAC&U)   Essential   Learning   Outcomes)   and   engineering   in   the   form   of   accreditation   requirements.  The   three-­‐year   project   at   [University   X,   (blinded)]   is   using   multiple   methods   to   assess   the  longitudinal   development   of   these   skills   in   engineering,   humanities,   physical   science,   and   social  science   sectors.      Methods   are   aimed   at   sustainable   assessment   achieved   within   standard   course  contexts  and  developing  internal  processes  for  the  implementation,  management,  and  assessment  of  university-­‐wide  learning  outcomes  that  recognize  and  enhance  disciplinary  expectations.    We  have  taken  a  three-­‐pronged  approach  to  the  longitudinal  assessment  of  general  learning  outcomes  across  targeted  programs,  including  engineering:  A.         Assessment  using  standardized  quantitative  instruments  and  qualitative  processes  B.         Working  with  course  instructors  to  align  teaching,  learning  and  assessment  of  complex             cognitive  skills,  embedding  course-­‐based  “authentic  problem  tasks”  for  course  grading  and             assessment  C.         Assessment  of  course  artefacts  using  meta-­‐rubrics  scored  independently  of  course  grading  The   project   is   using   multiple   instruments   to   measure   identified   learning   outcomes,   including   two  standardized   instruments:   Collegiate   Learning   Assessment   Plus   (CLA+)   Test,   Critical   Thinking  Assessment   (CAT)   Test,   think-­‐aloud   sessions,   AAC&U   VALUE   rubrics,   and   a   new   triangulated  qualitative/quantitative  measure  of  Transferable  Learning  Orientations  (TLO)  based  on  the  VALUE  rubric  for  lifelong  learning  and  the  Motivated  Strategies  for  Learning  Questionnaire  (MSLQ).  In  the  first  year  of  the  project  a  double  cross-­‐sectional  study  of  first  and  fourth  year  students  was  conducted  to  pilot  the  instruments  and  identify  key  themes.  Over  2000  first  and  fourth  year  students  from  the  Faculty  of  Arts  and  Science  (Psychology,  Drama  and  Physics),  and  from  the  Faculty  of  Engineering  and  Applied  Science  (Chemical  Engineering,  Civil  Engineering,  Geological  Engineering,  and  Mechanical  Engineering)  have  consented  to  participate  in  the  project.    Preliminary  analysis  indicates  significant  correlations  in  student  scores  on  specific  CLA+  and  VALUE  rubrics  dimensions,  and  significant  changes  in  learning  outcomes  from  first  year  to  fourth  year  in  engineering.  The  TLO  was  piloted  among  approximately  1000  students  in  Engineering  and  Applied  Science  over  two  years.  The  instrument  was  refined  through  quantitative  analysis  and  focus  groups,  and  found  significant  improvement  between  first  to  fourth  year  learning  orientations.  This  paper  will  present  results  from  the  first  year  of  the  study  including  descriptive  statistics  and  factor  analysis  for  the  first  year  and  fourth  year  student  samples,  correlations  between  the  CLA+  and  assessment  course  artefacts  using  VALUE  rubrics,  approach  to  calibration  and  scoring,  and  demographic  analysis.    It  will  discuss  observations  about  instructor  buy-­‐in  and  feedback,  and  broader  impact.  It  will  also  present  the  development  and  dimensions  of  the  TLO,  alignment  with  the  VALUE  rubrics,  and  results  of  the  pilot  study.  
AU  - Brian M Frank
AU  - James A. Kaupp
AU  - Natalie Simper
AU  - Jill Scott
CY  - Seattle, Washington
DA  - 2015/06/14
PB  - ASEE Conferences
TI  - Multi-method Longitudinal Assessment of Transferrable Intellectual Learning Outcomes
UR  - https://peer.asee.org/24513
DO  - 10.18260/p.24513
ER  -