To Build or to Buy, That is the Question

Wanju Huang; Margaret Wu

Download Paper | Permalink

Conference: 2024 ASEE Annual Conference & Exposition
Location: Portland, Oregon
Publication Date: June 23, 2024
Start Date: June 23, 2024
End Date: July 12, 2024
Conference Session: Continuing Professional Development Division (CPD) Technical Session 1
Tagged Divisions: Continuing, Professional, and and Online Education Division (CPOED)
Permanent URL: https://peer.asee.org/48158

Request a correction

Paper Authors

biography

Wanju Huang Purdue University

visit author page

Dr. Wanju Huang is a Clinical Associate Professor of Learning Design and Technology at Purdue University. Her research interests focus on online learning, professional development in STEM, augmented reality/virtual reality, and the integration of artificial intelligence in education. She has contributed to three NSF-funded projects as co-PI and key personnel. Additionally, she has been a co-PI for grants funded by the Susan Harwood Training Grant Program of the Occupational Health and Safety Administration (OSHA) and Substance Abuse and Mental Health Services Administration (SAMHSA) under the U.S. Department of Health and Human Services. Currently, she is leading the professional learning evaluation for the Micro-Electronics Security Training (MEST) initiative, which is part of the Scalable Asymmetric Lifecycle Engagement (SCALE) funded by the Department of Defense.

visit author page

biography

Margaret Wu Purdue University

visit author page

Marge Wu is a Senior Research Analyst with the Micro-electronics Security Training (MEST) Initiative. She has a decade’s worth of diverse experiences working in assessment, instructional technology support and development, and information technology units in higher education. Her professional interests include the intersection of technology and pedagogy, online learning, outcomes and program evaluation, and survey methodology. She holds a M. S. Ed in Literacy and Language Education and a BA in English Secondary Education, both from Purdue University.

visit author page

Download Paper | Permalink

Abstract

The technology learners use to access online professional development learning experiences can impact their learning, persistence, and satisfaction. The instructional technology market offers a plethora of learning management systems and other tools that allow organizations to deliver various learning opportunities (e.g., degree programs, non-credit certificates). But how can organizations determine which tool offers the features that are critical to learners’ unique needs? When exploring learning management systems (LMSs) suitable for professionals/adult learners’ learning preferences and needs, existing LMSs (e.g., Canvas, Brightspace, edX) might not be the most appropriate. Features that are crucial to organizations offering professional development to adult learners may not be as salient to the K-12 and post-secondary environment, the LMS market’s primary customer. This paper discusses the process of identifying and prioritizing the essential LMS features critical to high-level learning and teaching needs in the context of a professional development program for adult learners in specialized STEM fields. At the same time, the platform must allow instructors to deliver high-quality learning experiences using evidence-based pedagogical practices for online learning. In conducting a needs analysis, three tiers of features of an LMS ideal for professionals/adult learners were identified. The team first focused on the robust grading, assessment, discussion, and content management features learners and instructors expect from a modern LMS. Next, the team identified features that may be present to some extent within existing LMSs but are accomplished through third-party integrations rather than available to learners seamlessly within the platform. These features include video-conferencing, instant messaging, document collaboration and storage tools for group work, and data-informed grouping options for instructors to create learner groups based on responses to questionnaires. In considering this tier, the team also identified specialized technology, such as modeling software and IT security measures, as features worthy of future exploration. Finally, the team identified emerging or nonexistent features in the LMS market. The focus was on features that could leverage data generated through learner actions to create visualizations, dashboards, and reports that provide learners with information they can use to enhance their professional development paths and provide instructors with data they can use to improve course and program quality. Features belonging to this tier include Social Network Analysis (SNA) that enables learners to see graphs of their interactions with other learners and instructors, Natural Language Processing (NLP) that allows learners to see themes emerging from discussions, visualizations that offer learners insight on their learning journey based on completion of past experiences and progress in ongoing experiences as well as recommendations for future courses, and easily exportable reports on granular clickstream data. This needs assessment and LMS market analysis enabled the team to make an informed decision on whether to purchase a prepackaged LMS (i.e., to Buy) or to develop a customized LMS for professional development programs (i.e., to Build). Given the existing resources available, such as technical infrastructure and personnel, it became evident that building an LMS was a feasible goal that would best support adult learning and teaching excellence and also ensure program sustainability.

Citation
Format

Huang, W., & Wu, M. (2024, June), To Build or to Buy, That is the Question Paper presented at 2024 ASEE Annual Conference & Exposition, Portland, Oregon. https://peer.asee.org/48158

TY  - CPAPER
AB  - The technology learners use to access online professional development learning experiences can impact their learning, persistence, and satisfaction. The instructional technology market offers a plethora of learning management systems and other tools that allow organizations to deliver various learning opportunities (e.g., degree programs, non-credit certificates). But how can organizations determine which tool offers the features that are critical to learners’ unique needs?  
When exploring learning management systems (LMSs) suitable for professionals/adult learners’ learning preferences and needs, existing LMSs (e.g., Canvas, Brightspace, edX) might not be the most appropriate. Features that are crucial to organizations offering professional development to adult learners may not be as salient to the K-12 and post-secondary environment, the LMS market’s primary customer. This paper discusses the process of identifying and prioritizing the essential LMS features critical to high-level learning and teaching needs in the context of a professional development program for adult learners in specialized STEM fields. At the same time, the platform must allow instructors to deliver high-quality learning experiences using evidence-based pedagogical practices for online learning. 
In conducting a needs analysis, three tiers of features of an LMS ideal for professionals/adult learners were identified. The team first focused on the robust grading, assessment, discussion, and content management features learners and instructors expect from a modern LMS. Next, the team identified features that may be present to some extent within existing LMSs but are accomplished through third-party integrations rather than available to learners seamlessly within the platform. These features include video-conferencing, instant messaging, document collaboration and storage tools for group work, and data-informed grouping options for instructors to create learner groups based on responses to questionnaires. In considering this tier, the team also identified specialized technology, such as modeling software and IT security measures, as features worthy of future exploration. 
Finally, the team identified emerging or nonexistent features in the LMS market. The focus was on features that could leverage data generated through learner actions to create visualizations, dashboards, and reports that provide learners with information they can use to enhance their professional development paths and  provide instructors with data they can use to improve course and program quality.  Features belonging to this tier include Social Network Analysis (SNA) that enables learners to see graphs of their interactions with other learners and instructors, Natural Language Processing (NLP) that allows learners  to see themes emerging from discussions, visualizations that offer learners insight on their learning journey based on completion of past experiences and progress in ongoing experiences as well as recommendations for future courses, and easily exportable reports on granular clickstream data.  
This needs assessment and LMS market analysis enabled the team to make an informed decision on whether to purchase a prepackaged LMS (i.e., to Buy) or to develop a customized LMS for professional development programs (i.e., to Build). Given the existing resources available, such as technical infrastructure and personnel, it became evident that building an LMS was a feasible goal that would best support adult learning and teaching excellence and also ensure program sustainability.  
AU  - Wanju Huang
AU  - Margaret Wu
CY  - Portland, Oregon
DA  - 2024/06/23
PB  - ASEE Conferences
TI  - To Build or to Buy, That is the Question
UR  - https://peer.asee.org/48158
ER  -