Authors

1. Sinclair, Peter BN(Renal), MPhil
2. Kable, Ashley Dip Teach Nurse Ed, RN, Grad Dip Health Serv Man, PhD, MACN
3. Levett-Jones, Tracy PhD, RN, MEd & Work, BN, DippAppSc (Nursing)

Abstract

Review question/objective: The objective of this systematic review is to identify, appraise and synthesize the best available evidence for the effectiveness of internet-based e-learning programs on health care professional behavior and patient outcomes.

Background: Technological innovation has not only impacted social change in recent years but has been the prime driver of educational transformation.1The newest consumers of post-secondary education, the so-called 'digital natives', have come to expect education to be delivered in a way that offers increased usability and convenience.2 Health care professionals (HCPs) in the clinical setting, particularly those in rural and remote communities, are no different.3-5 Today's health workforce has a professional responsibility to maintain competency in practice through achieving a minimum number of hours of continuing professional development.6 Consequently, HCPs seeking professional development opportunities are reliant on sourcing these independently according to individual learning needs.7 However, difficulties exist in some health professionals' access to ongoing professional development opportunities, particularly those with limited access face-to-face education8,9due to geographical isolation or for those not enrolled in a formal program of study.10,11These issues challenge traditional methods of teaching delivery; electronic learning (e-learning) is at the nexus of overcoming these challenges.

The term e-learning originated in the mid-1990s as the internet began to gather momentum.1Electronic learning can be broadly defined as any type of educational media that is delivered in an electronic form.12Terms such as computer-assisted learning, online learning, web-based learning and e-learning are often used synonymously but all reflect knowledge transfer via an electronic device. This broad definition allows for a gamut of multimedia to be used for the purpose of constructing and assessing knowledge. Multimedia typically used in e-learning range from the now archaic Compact Disc Read-Only Memory (CD-ROMs), to the simple Microsoft PowerPoint, or the more advanced and complex virtual worlds such a second life. Electronic learning can be delivered in asynchronous or synchronous formats, with the latter (for example interactive online lectures via platforms such as BlackboardCollaborate or WebEx) more commonly used in formal educational settings according to set timetables of study.1

Person-to-person interactivity is an important enabler of knowledge generation and while functionalities such as web 1.0 (discussion board and email) and more recently web 2.0 (Wikis and blogs) allow for this to occur both synchronously and asynchronously, it is usually utilized in formal educational contexts only. However, the economy of formal education does not allow for free access to courses which proves challenging for HCPs seeking quality educational opportunities who choose not to undergo a formal program of study or are just looking to meet a specific learning need. Alternatively, asynchronous e-learning is a more learner-centred approach that affords the opportunity to engage in learning at a time and location that is convenient and enables the learner to balance professional development with personal and work commitments.13These learning opportunities are self-directed and do not require a human to facilitate learning, rather, technology officiates/facilitates the learning process and, in the asynchronous e-learning context, the learner negotiates meaning independently.14

Health-related e-learning research has focused on several domains including media comparative designs15,16, self-efficacy17,18, user satisfaction19,20, instructional design21, knowledge outcomes9,22-28, clinical skills development17,29,30, and facilitators/barriers to its use.31The benefits of e-learning are well documented in terms of increased accessibility to education, efficacy, cost effectiveness, learner flexibility and interactivity.32However, some fundamental methodological and philosophical flaws exist in e-learning research, not least the use of comparative design studies. Comparison between e-learning and traditional teaching methods are illogical and methodologically flawed because comparison groups are heterogeneous, lack uniformity and have multiple confounders that cannot be adjusted for.33,34As early as 1994, researchers34in computer-assisted learning were citing these limitations and called for a fresh research agenda in this area. Cook33,35repeated this call in 2005 and again in 2009 and noted a paucity of research related to patient or clinical practice outcomes. Electronic learning is not an educational panacea and research needs to progress from pre- and post-interventional and comparative designs that evaluate knowledge increases and user satisfaction. It is time to move towards determining whether improved self-efficacy or knowledge gained through e-learning improves patient outcomes or influences clinical behavior change and whether these changes are sustained. In order to develop the empirical evidence base in e-learning, research needs to be guided by established theoretical frameworks and use validated instruments to move from assessing knowledge generation towards improving our understanding of whether e-learning improves HCP behavior and more importantly, patient outcomes.

One suitable framework that is congruent with e-learning research is Kirkpatrick's36four levels of evaluation. Kirkpatrick's model is hierarchically based with level one relating to student reaction and how well the learner is satisfied with the education program. Level two pertains to learning and the evaluation of knowledge, level three expands on this and considers whether the education has influenced behavior. In the context of this review, behavior change is any practice that is intrinsically linked with the outcomes of the e-learning program undertaken. Finally, level four evaluates the impact on outcomes such as cost benefit or quality improvements.36,37The majority of e-learning research has focused on participant experience and knowledge acquisition, outcomes that correspond with the first two levels of Kirkpatrick's model.38To date, few studies have examined the effectiveness of internet-based e-learning programs on HCP behavior, which aligns with Level 3 of Kirkpatrick's model.

Studies exist that use self-reported measures of intention to change behavior,39,40 however self-reported intention to change does not necessarily translate into actual behavior change.41 Studies that have not used self-reported measures of behavior change have used objectively measured evaluation criteria including objective structured assessment of technical skills (OSATS) using various methods including simulation task trainers42 and clinical simulations using standardized patients43 scored by a panel of experts using standardized assessment tools. Carney et al.44 used a national reporting and data system to measure the impact of a single one hour e-learning program undertaken by radiologists (n=31) aimed at reducing unnecessary recall during mammography screening. Carney et al. reported a null effect and attributed this to the complexities of behavior change, suggesting that longer term reinforcement of principles relating to mammography recall was required to effect behavior change. These findings also suggest that a multi-modal intervention may be required in order to reduce excessive recall rates in this area, rather than a single intervention. Contrary to Carney et al., Pape-Koehler et al.42 and Smeekins et al.43 reported positive findings using randomized controlled designs to test the efficacy of e-learning interventions on individual's surgical performance42 and the detection of child abuse43, respectively. Pape-Koehler et al. used a 2x2 factorial design to demonstrate that an e-learning intervention significantly improved novice surgeon (n=70) surgical performance of a laparoscopic cholecystectomy (change between pre-post test OSATS p 0.001) when used in isolation or in combination with a practical training session compared to practical training alone. Smeekins et al. demonstrated that a 2 hour e-learning program improved nurses' (n=25) ability to detect child abuse in an emergency department. The nurses in the intervention (n=13) group demonstrated significantly better (p=0.022) questioning techniques and consequently, higher quality history taking, to determine children at risk of child abuse when compared with the control group who received no training at all.

These three exemplar studies demonstrate the broad range of applications e-learning has in HCP education, as each study used different designs, had different subject areas and target health care professionals. This reflects the conceptual and practical challenges of the area of research that addresses levels three of Kirkpatrick's model. For this reason, the e-learning research agenda in health should focus on whether knowledge generated through e-learning is able to be re-contextualized into clinical practice, and influence sustained clinical behavior change and patient outcomes.

A preliminary search of PubMed, CINAHL, The Cochrane Library, The JBI Database of Systematic Reviews and Implementation Reports, ERIC and PROSPERO was conducted to determine if a systematic review on the topic of interest already existed. This search identified four systematic reviews that specifically reviewed outcome measures of knowledge and skill improvement in the domain of e-learning. Two38,45 examined research conducted in nursing, with the other two46,47 in orthodontics. Lahti et al.38 systematic review examined the impact of e-learning on nurses' and nursing students' knowledge, skills and satisfaction. Lahti et al.38 were unable to demonstrate a statistical difference between cohorts undertaking e-learning compared to conventional teaching methods, findings that were not replicated by Du et al.45 This may be due to the decision by Lahti et al. to include studies that utilized CR-ROM and that, despite being published in 2014, the actual review took place in 2010 and did not capture several significant studies published after this date. The systematic review by Du et al.45 examined the efficacy of online distance education in terms of knowledge acquisition and retention, and skill performance in employed nurses and nursing students. This review identified nine randomized controlled trials (RCTs) of which five studies considered skill performance. Four of these studies demonstrated superior or equivalent improvement in skills compared to control groups. Similar findings were noted in two other systematic reviews conducted by Al-Jewair et al.46,47 who concluded that computer-aided learning was at least as efficacious as conventional teaching methods in improving knowledge in undergraduate and postgraduate orthodontic students and educators.

These reviews included blended and fully online studies of varying instructional design quality. Seven out of the nine studies identified by Du et al. utilized interactive elements to facilitate communication between students and teachers which would confound the results of the review given that interactivity in this manner is known to improve the user experience and knowledge outcomes.48 To date, there are no database-indexed systematic reviews that identify, appraise and synthesize the best available evidence for the effectiveness of internet-based e-learning programs on HCP behavior and patient outcomes using objectively administered evaluation criteria.