Return on investment for digital learning
Pugliese and the Arizona State University Action Lab has an upcoming report that offers data from six large educational institutions over 18 months with a focus on finding what improves return on investment (ROI) in the areas of access, outcomes, and economics when providing digital learning. These notes are from articles written in advance of that report (the actual report is available in mid-April).
Bates, T. (2018) ‘Some thoughts on scaling online and digital learning’, online learning and distance education resources, February 14. Available at https://www.tonybates.ca/2018/02/14/some-thoughts-on-scaling-online-and-digital-learning/.
Chatlani, S. (2018) ‘How to effectively scale a digital learning model’, Education Dive, January 31. Available at https://www.educationdive.com/news/how-to-effectively-scale-a-digital-learning-model/515936/.
The ROI of digital learning with Lou Pugliese (2018) YouTube video, added by Bryan Alexander [Online]. Available at https://youtu.be/qtJt8ThrUE8?t=6m51s.
Schaffhauser, D. (2017) ‘Finding the ROI of Online Programs’, Campus Technology, July 19. Available at https://campustechnology.com/articles/2017/07/19/finding-the-roi-of-online-programs.aspx.
Pugliese works at Arizona State University’s Action Lab . They conducted research to find out if schools are getting a good return on their investment into the development and delivery of online courses. There have been other research projects with similar goals, but Pugliese felt that most findings did not lead to actionable conclusions. “It’s more than a best practices report,” he noted. “It’s really specific mechanics of the economic and operational efficiencies in these institutions that we surveyed that would directly translate [to other institutions].
‘”It’s more than a best practices report,” he noted. “It’s really specific mechanics of the economic and operational efficiencies in these institutions that we surveyed that would directly translate [to other institutions].”‘ (Schaffhauser, 2017).
Online learning has been around for 20 years or so, and so we can know investigate what works at scale, with a broad and diverse array of learners.
They looked at six schools which represented a range of institutions.
- Public universities:
- Arizona State University (72,000 students)
- University of Central Florida (66,000 students)
- Georgia State University (32,000 students)
- Technical and community college:
- Kentucky Community & Technical College System (16 colleges and over 70 campuses, with a total of 107,000 students of which 55,000 are online)
- Community colleges:
- Houston Community College (23 campuses, with a total of 69,000 students)
- Rio Salado College (60,000 students, with most courses offered online)
The question about ROI focuses on three dimensions:
- Access: Enrollment growth, socioeconomic diversity
- Academic outcomes: Degree completion, retention, course-level outcomes
- Institutional economics: Periodic investments needed to sustain digital environments, infrastructure needs, course development costs and how to maximize content development space after making the initial investment, needed student support
Pugliese emphasizes that the report pulls together a lot of data, and institutions and learning how to organize themselves to maximize their similar data.
Case study: Rio Salado College
The official report will include case studies of six institutions, including Rio Salado College which is a community college in Arizona. Rio Salado expanded the number of online courses they offer while keeping costs to a minimum by ‘using adjunct faculty for online classes and compensating them… [in a plan in which] teachers are paid per enrollee’ (Chatlani, 2018). This now involves ‘more than 1,500 adjunct faculty members with limited course-design input, and 23 full-time faculty chairs crafting courses to help adjuncts learn best ways of delivering the content, creating and accessing to open educational resources and spearheading curriculum’ (Chatlani, 2018). They keep faculty happy and quality high by having many staff meetings, maintaining reasonable standards for interacting with students, and providing meaningful metrics.
Case study: Arizona State University (ASU)
ASU has a large number of non-traditional learners and their online program is designed to entice and support these learners. They mostly design their courses in-house, but use Blackboard for an LMS and contract with Pearson for student acquisition support. One key to their success in their online learning programs is their use of learning analytics. They have a team of about 50 instructional designers who work with the faculty to design the courses and create plug-ins that will work with their LMS to provide the affordances found in learning technology (Chatlani, 2018).
Findings: Things that have changed
Online learning used to be focused on students working toward their Bachelor’s degree, but now the student population for these courses is more diverse — which means the programs similarly are more diverse.
Also, schools used to outsource their online program management (OPM), but now they mostly develop and promote the programs in-house because this provides them with higher quality material that can be scaled to meet their needs. Pugliese mentions that they also looked at how open educational resources (OER) was used in some schools, but he did not provide any details.
Findings: Areas for improvement
Schools can improve cost and efficiency by adopting a centralized approach to delivery.
Near future: Official report and workshops
Pugliese’s research group is still putting together their report, which will provide details of their findings. (It is scheduled for release in early April 2018.) They also are planning to host workshops at different schools over the next few years with discussions about how the schools can take actionable steps to improve ROI based on the research group’s findings.