Organizations are implementing and using electronic health records (EHRs) as they strive to meet Meaningful Use criteria enacted through the Health Information Technology for Economic and Clinical Health Act, part of the American Recovery and Reinvestment Act.1 According to the Institute of Medicine (IOM), EHRs have eight core functions, and one of those is to provide clinical decision support (CDS) tools.2 Clinical decision support equips clinicians, staff, patients, or other individuals with knowledge and person-specific information, intelligently filtered or presented just in time, to enhance health and healthcare.3 In support of the application of evidence to healthcare decisions, the IOM recommends the use of computerized CDS.4 Clinical decision support can include a variety of tools such as alerts, reminders, order sets, care plans, protocols, enhanced displays, and documentation forms.5,6
BACKGROUND
Clinical Decision Support
Systematic reviews studying outcomes of health information technology and its impact on quality of care have shown that reminders, a form of CDS, significantly improve clinician adherence to practice guidelines.7-9 Most of the randomized controlled trials were conducted in outpatient environments, primary care settings specifically, with interventions targeting providers (physicians predominantly and sometimes advanced practice providers) and measuring processes of care and less frequently outcomes of care.7,8 Frequent targets for CDS intervention include health maintenance, prescription and medication dosing, and clinical test ordering.10 Clinical decision support effectiveness is linked with integration within an EHR requiring no additional clinician data entry, providing decision support as part of the clinical workflow, providing recommendations, promoting action, and providing decision support at the time and location of decision making.10
A series of systematic reviews was conducted on CDS systems (CDSSs) and updated over an 11-year period.11-13 These reviews looked at controlled trials and effects on clinician performance and patient outcomes. The most recent review of 100 studies (only two were nursing focused) reported 62 of 97 studies (64%) had an increase in clinician performance and seven of 52 studies (13%) improved patient outcomes with CDSSs.13 A synthesis of systematic reviews on CDSSs included 17 studies (only one nursing focused); 12 of 16 reviews had strong evidence for improved clinician performance, and only three of 16 reviews found strong evidence for positive patient outcomes.14 There is strong evidence that CDSSs improve process adherence and clinician performance.7-14
Clinical Decision Support for Nurses
A systematic review of CDSS in nursing identified several benefits of CDS, including improved interprofessional communication, increased access to best practice information, and more consistent quality of care.15 Randell et al16 conducted a systematic review of CDSS use in nursing with eight studies and found an inconsistent effect of the CDSS on nursing performance and patient outcomes. A Cochrane review evaluated the effects of on-screen computer reminders on processes and outcomes of care and reported median process improvements of 4.2% across all processes (interquartile range, 0.8%-18.8%).17
Dowding et al18 reviewed four case sites and nursing use of CDSSs; nurses used CDSSs for recording information, monitoring patient progress, and confirming the accuracy of decisions previously made. Nurses were observed making decisions and communicating with patients and then only later returning to the CDSS to confirm their decisions. Nurses' "familiarity with the patient, the patient's condition, and the decision support technology" appeared to affect CDSS use and whether they overrode recommendations made by the CDSS.18(p1164)
A study examined nurses' perceptions of reminders on missed nursing care and found correlations between the reported reminder usage and reported missed nursing care.19 A literature review of CDSS in nursing practice focused on features useful to nurses including assessment, problem diagnosis, plans of care, interventions, and evaluation of outcomes; 27 studies supported two or more of these nursing features.20 Studies on CDSS use in nursing are limited.20,21
Electronic health records provide opportunities to implement CDS tools such as alerts and reminders. One organization was implementing a new EHR and with it CDS for nurses; this example discusses the use of CDS for nurses to support fall risk identification and fall prevention plan of care.
FALL RISK AND PREVENTION
Patient falls are the most frequently reported adverse event in hospitals.22,23 Patient falls occurring in hospitals can lead to injury, increased length of stay, and increased costs.24 In fact, costs for hospitalized patient falls with serious injury have been estimated at $13 316 per patient, and the length of stay for patients experiencing a fall has been shown to be 6.3 days longer than that for nonfallers.24 In addition, falls contribute to mortality rates that are up to 50% higher than those for patients who do not fall.25 The 2012 average rate of falls with injury in US hospitals for Medicare patients was 0.527 per 1000 patient-days (range, 0-11.628); Duke University Hospital's 2012 rate was 0.425 per 1000 patient-days.26 In 2008, the Centers for Medicare & Medicaid Services categorized falls with injury as hospital-acquired conditions, and costs related to these falls became nonreimbursable.27
Falls are measured as a nursing-sensitive indicator by the National Quality Forum28 and Press Ganey's National Database of Nursing Quality Indicators.29 Clinical practice guidelines recommended by the Agency for Healthcare Research and Quality,30 the Institute for Healthcare Improvement,22 and the Institute for Clinical Systems Improvement31 consider a fall risk assessment as a standard component of a nursing admission assessment for adults in US hospitals and specify when a fall prevention plan of care is needed. In accordance with these guidelines, Duke University Hospital requires RNs to complete and record a fall risk assessment within 24 hours of patient admission. The assessment includes a series of questions, and any positive answer indicates the patient is at high risk. If indicated by the risk assessment, a fall prevention plan of care is initiated and recorded. The fall risk assessment is repeated every 12-hour nursing work shift to determine changes in risk that may indicate an evolving need for a fall prevention plan of care.
PROJECT AIMS
This quality-improvement project implemented CDS tools for fall risk identification and fall prevention. Aims of the project included (1) improving documentation of fall risk assessments and for high-risk patients' fall prevention plans of care, (2) assessing nursing staff satisfaction to determine acceptance of the computerized fall risk program, and (3) improving clinical outcomes by reducing patient falls and patient falls with injury.
LITERATURE REVIEW: CLINICAL DECISION SUPPORT AND FALLS
The implementation of EHRs with embedded CDS functions has been shown to demonstrate improved documentation of fall risk.32,33 Electronic fall prevention "toolkits" can be in the form of stand-alone fall risk assessments recommending patient-specific interventions, including automated population of customized care plans, tailored bedside posters, and individualized patient education handouts.33,34 More specifically, studies have shown that electronic fall prevention toolkits improved documentation of fall risk assessment by 25% (P < .003),34 improved documentation of fall prevention plans of care for at-risk patients by 25% (P < .0001),33 reduced overall falls rates (P = .02), and reduced fall rates in patients 65 years or older (P = .003)35 compared with usual care with no CDS. Clinical decision support systems have demonstrated success in improving clinician performance and some improvement in clinical outcomes for fall prevention and management.32-35
METHODS
Intervention and Implementation
In August 2011, Duke University Hospital decided to implement an EHR with the capability to include CDS tools, specifically alerts and reminders. The EHR implementation involved many changes within the organization at one time; about 54 legacy software applications retired on implementation of the new EHR system. Nurses on medical and surgical units received more than 18 hours of training on the new EHR workflows and functions. Some functions, such as flow-sheet charting for shift assessments, transitioned from one computer application to another. Functions such as admission assessments, care plans, and bar-code medication administration were new electronic features for staff.
The organization implemented fall prevention-related CDS as a component of the Epic Systems (Madison, WI) EHR in June 2013. Prior to implementation of the EHR, the fall risk assessment was documented on a computerized nursing flow-sheet application, and plans of care were paper based. The legacy flow-sheet application did not include any CDS components such as alerts and reminders.
The CDS tools for fall prevention included three features: (1) an "admission documentation incomplete" fall risk assessment indicator, (2) a "shift documentation incomplete" fall risk assessment indicator, and (3) a "rules-based alert" for patients at high risk of falls and not on a fall prevention plan of care. The incomplete assessment indicators (the first two implementation features) provide links to appropriate documentation sections within the EHR. The third feature, the plan-of-care alert, presents a link to implement the fall plan of care. Collectively, these features for fall risk assessment and plan-of-care implementation constitute the CDS tools. The users of the tools are RNs caring for hospitalized adults.
Setting
The setting is Duke University Hospital, a 938-bed academic health center in Durham, NC. The project focused on 16 adult units ranging from 16 to 32 beds each, for general medical or surgical patients as well as specialty populations. At the start of the project, one medical unit and one surgical unit were performing below the target 90% documentation compliance rate for fall assessments and plans of care; these two units were selected for retrospective chart review relative to documentation of the fall risk assessments and fall prevention plans of care. These two units were also selected for review of alert action data in the post-CDS period and focus groups to evaluate nursing staff satisfaction.
Design and Timeline
The project used a pre/post quasi-experimental study design. The overall methods include comparison of quarterly audits, retrospective chart review, alert action data, fall safety reports, falls and falls-with-injury rates, and focus groups. Documentation of fall risk assessments and, for patients at high risk, fall plans of care were evaluated using quarterly audit data, retrospective chart review, and fall safety reports. Nursing staff satisfaction was evaluated using focus groups. Clinical outcomes were evaluated using falls and falls-with-injury rates. The overall pre-CDS observation period was October 2012 through May 2013. The post-CDS observation period was August 2013 through January 2014. The specific pre- and post-CDS periods vary by data source and are further outlined in the following section. This project was reviewed by the Duke University Health System's institutional review board and approved as exempt.
ANALYSIS AND RESULTS
Data were analyzed using IBM SPSS Statistics version 21 (Armonk, NY). The overall quantitative results are available in Table 1 and presented by data source, time period, and measure.
Quarterly Audits
Duke University Hospital performs a planned quarterly point prevalence chart audit to monitor the completion of fall risk assessments in the previous 24 hours and associated plans of care for patients at high risk. Unit-level fall reduction champions are responsible for ensuring completion of the quarterly fall process audits. In accordance with The Joint Commission recommendations,36 90% compliance with documentation standards is the minimum acceptable target. Documentation compliance with the fall risk assessment for 16 selected adult medical and surgical units (average, 30 beds per unit) at Duke University Hospital ranged from 73% to 100% compliance in October 2012: three of the selected units scored less than the organizational targeted compliance of 90%. Documentation compliance with fall prevention plans of care during the same period ranged from 70% to 100%, with five of 16 adult units scoring less than the target compliance of 90%.
Fall risk assessment and plan-of-care-documentation compliance rates for the 16 units were examined. Data from three quarterly point prevalence audits pre-CDS were compared with three quarters post-CDS. Data from the quarterly audits were not normally distributed and had a negative skew, with most numbers large and close to 100%; as a result, a Mann-Whitney U test was performed. Mean compliance with the fall risk assessment increased from 95.3% to 97.25% post-CDS (median, 96.5 to 100 post-CDS) and was statistically significant. Mean compliance with the fall plan of care increased from 92.33% to 92.58% post-CDS (median, 93.3 to 100 post-CDS) and was not significant. Further examination of the preselected medical unit and surgical unit indicated that mean documentation compliance for the fall risk assessment did increase from 88.95% to 98.27% (median, 90 to 100 post-CDS), which was statistically significant. Mean plan-of-care-documentation compliance for the two units increased from 82.35% to 93.45% (median, 82.7 to 100 post-CDS) and, although not statistically significant, did bring them within compliance requirements of 90%.
Retrospective Chart Review
Retrospective chart review was completed by the primary author for 60 patients per unit on the preselected medical and surgical units 2 months pre-CDS and 2 months post-CDS. Reviewed charts were randomly selected from a list of patients on each unit during the chosen time periods. Chart review included fall risk assessment documentation on admission if the patient was originally admitted to the study unit, documentation of fall plan of care upon admission for patients at high risk, and completion of the shift fall risk assessment and the fall plan of care (if indicated) for up to nine shifts (4.5 days based on the organizations' average lengths of stay). As a result, a completion percent was calculated for each patient based on the number of shifts reviewed.
Of the 240 charts reviewed pre- and post-CDS, only 55 patients were originally admitted to the study units pre-CDS, and 88 patients were admitted post-CDS review (all other patients were transferred to the units). Documentation compliance for the fall risk assessment on admission increased from 92.73% to 98.86% and was statistically significant. Thirty-five patients met criteria indicating that they were at high risk of falls on admission pre-CDS, and 65 patients met criteria post-CDS. Admission plan-of-care compliance for patients at high risk of a fall decreased from 77.1% to 61.5% post-CDS and was not statistically significant. On further examination, the medical unit had a statistically significant decrease in documentation of the fall plan of care on admission from 75% pre-CDS to 32.1% post-CDS.
Compliance for the shift fall risk assessment increased from 93.25% pre-CDS to 94.69% post-CDS (median, 100 to 100 post-CDS) but was not statistically significant. Compliance for the associated fall plan of care decreased from 75.22% pre-CDS to 60.35% post-CDS (median, 100 to 100 post-CDS) and was statistically significant.
Alert Action Data
Alert action data for the plan-of-care alert on the preselected medical and surgical units were reviewed for the post-CDS period in November and December 2013. No data were available about action on the fall risk assessment reminders (shift or admission). In November 2013, the plan-of-care alert triggered 1982 times on the two units, and in only 42 instances the care plan template was applied (2%). The medical unit applied the care plan 19 times, and the surgical unit applied the plan 23 times. In December 2013, the alert triggered 1671 times, and the care plan template was applied 42 times (2.5%): 20 times on the medical unit and 22 times on the surgical unit.
Fall Safety Reports
Patient falls are reported by the staff nurse caring for the affected patient via an electronic safety reporting system, in place before launching the EHR and used throughout this project. Patient falls reported through the safety reporting system are routed electronically to the unit management team for review. Using fall safety reports on the 16 units, data were reviewed to determine documentation compliance for the fall risk assessment, time since the last fall risk assessment, and (if indicated) documentation of the fall plan of care for patients who fell. The pre-CDS period was compared with the post-CDS period using [chi]2 test.
There were 39 patients with fall safety reports pre-CDS and 45 patients in the post-CDS period. All patients with falls both pre- and post-CDS had a fall risk assessment prior to the fall. The time since the last fall risk assessment was not significant pre- and post-CDS. The compliance rate for patients who fell, were at high risk of falls, and had a fall plan of care was 96.9% pre-CDS and 100% post-CDS, although this increase was not statistically significant.
Falls and Falls With Injury
Patient falls per 1000 patient-days and falls with injury per 1000 patient-days were examined for the 16 units. The pre-CDS period was compared with the post-CDS period. Falls and falls with injury were not normally distributed with a positive skew, as most numbers were small and close to zero; as a result, a Mann-Whitney U test was performed. Mean falls per 1000 patient-days increased from 3.13 pre-CDS to 3.35 post-CDS (median, 2.53 to 2.97), and this was not significant. Mean falls with injury per 1000 patient-days were 0.447 pre-CDS and 0.490 post-CDS (median, 0.0 to 0.0 post-CDS), which was not significant.
Focus Groups
Nurse satisfaction with the fall prevention CDS tools was assessed on the preselected medical unit and surgical unit using focus groups. Participation was voluntary with notice by flyers posted on the two preselected units and announcement at the end of a unit-based staff meeting. The focus groups were 30 minutes in length and semistructured, with the facilitator asking questions about satisfaction with the CDS components. Each participant was given a handout with screenshots of the shift assessment reminder, admission assessment reminder, and the plan-of-care alert to facilitate common discussion about each item. The focus groups were held on May 7 and 13, 2014.
On the medical unit, 15 of the 45 RNs participated in the focus group, as did three of the 55 RNs from the surgical unit. The reminder for the shift fall risk assessment was viewed as most helpful, and the admission reminder was somewhat helpful. Several staff reported not having seen the alert for patients at high risk who were not on a plan of care. Recommendations for changes to the CDS tools and the EHR related to falls included (1) having the fall risk assessment question that was specific to admission (fall within 3 months prior to admission) display only on admission and (2) adding a row for documentation of the bed exit alarms. In addition, staff felt there was redundancy between the fall plan of care and the patient education topics. Discussion about the fall risk assessment indicated there was confusion about the definition of high risk among the staff on the medical unit. Staff felt that even though some of the fall risk assessment answers were "yes" or positive the patient was not at high risk in their judgment.
DISCUSSION
Aim 1: Documentation of Fall Risk Assessments and Fall Prevention Plans of Care
Documentation of fall risk assessments upon patient admission and documentation of a fall risk assessment every 12-hour nursing work shift improved. Documentation of fall risk assessments improved significantly on the 16 units post-CDS, according to quarterly audit data (P = .05) as well as on the preselected medical and surgical units (P = .03). Retrospective chart review indicated that the preselected units had improvement in documentation of fall risk assessment on admission post-CDS (P = .05). Other studies also demonstrated improvement in fall risk assessment.21-23 Examination of safety reports for patients who fell found all patients pre- and post-CDS had a fall risk assessment documented. No statistically significant differences were found in safety reports pre- and post-CDS for the time since the last fall assessment.
Documentation of fall prevention plans of care for high-risk patients did not improve after our implementation and in some cases was less compliant post-CDS. Documentation of the fall prevention plan of care for patients at high risk did not change according to quarterly audit data. In contrast, retrospective chart review indicated documentation of the admission plan of care on the medical unit decreased post-CDS (P = .00), and the shift plan of care decreased on both preselected units (P = .01). Action on the plan-of-care alert was only 2% to 2.5%. No statistically significant differences were found in safety reports pre- and post-CDS for documentation of the fall prevention plans of care.
The differences in results between quarterly audit data and retrospective chart review could be related to the quarterly audit data having higher starting compliance percentages, with ranges from 70% to 100%, resulting in no difference found. The decrease in documentation-of-care plans on the two units could be related to the change from paper care plans pre-CDS to electronic documentation. Nurses' primary documentation each shift was completed on flow-sheets and the medication administration record. It could be that the change to electronic care plans was a process different enough for the care plan to be overlooked.
Aim 2: Nursing Staff Satisfaction
Nursing staff satisfaction with and acceptance of the computerized fall risk program was adequate. Staff reported favorably on the fall risk assessment reminders; this is similar to other alert focus group results found with use of direct action links.37 Some staff had not seen or benefited from the care plan alert; this is in line with the low alert action data of 2% to 2.5% and the low care plan documentation compliance. Staff suggestions for EHR and CDS tool changes were sent to Duke University Health System information technology governance for approval and prioritization.
Aim 3: Clinical Outcomes
Clinical outcomes were unchanged as evidenced by no change in patient falls and falls-with-injury rates. These are similar to results found by Dowding et al38 using an interrupted time-series design in which implementation of an EHR did not result in decreased fall rates. A 2011 study of nurse-sensitive patient outcomes with implementation of EHRs (not CDS) reported that there was a higher rate of falls in year 1 of EHR implementation (4.6% to 6.3%, P < .001), and injury falls increased by 16.4% in year 1 (P < .05).39 This is also reflective of systematic reviews of CDSSs that report more improvement in clinician performance than in patient outcomes.11-13
Changes Implemented and Further Study
In our project implementation, the fall risk assessment had a final flow-sheet row in which staff indicated the plan of care was implemented. It was noted during retrospective chart review that even though this field was often used patients often did not have a corresponding fall prevention plan of care documented. This flow-sheet row was changed at the request of the fall champion group in March 2014 to capture whether the patient was at high risk of falls with a "yes" or "no" answer. Additional chart review would be useful in determining whether this change also improved fall plan-of-care-documentation.
The plan-of-care alert was available for viewing by the RNs in the admission/transfer/discharge navigator within the EHR. This placement facilitated the admission fall risk assessment but not the shift fall risk assessment and potential ongoing changes in risk (as the shift fall risk assessment was documented using flow-sheets). A system change was implemented in February 2014 to display the fall alert as a popup so RNs would see it during flow-sheet charting for patients at high risk of falls and not on a fall plan of care. Additional data collection is needed to determine whether this change improved documentation of fall plans of care. A review of the plan-of-care alert action data and a discussion with RN users would also provide useful follow-up.
The alert data indicated only 2% to 2.5% action on the care plan alert, which was exceedingly low. It would be interesting to see whether other EHR-implemented alerts at Duke University Hospital had similar action percentages or whether differences in alert action existed across clinician groups, such as RNs versus respiratory care practitioners or other ancillary care providers.
Other Considerations
Anecdotal notes during retrospective chart review showed that some nurses indicated within the fall risk assessment flow-sheet that the plan of care was implemented, but no fall plan of care was documented. On some records, the shift fall risk assessment changed from positive to negative and back again across multiple days, leading the chart reviewer to question the accuracy of the assessments. There were also occasions where the fall risk assessment changed from positive to negative, and the plan of care was appropriately discontinued; however, the fall risk assessment was later changed again to positive, but the fall plan of care was not reinitiated. For some positive fall risk assessments, staff would enter a comment that the patient was not at high risk. This discrepancy between staff judgment of patients at high risk on the medical unit was noted during the medical unit focus group session.
It is possible that the volume of changes with implementation of a hospital-wide EHR made it difficult to focus on a single aspect such as the available CDS tools. It also brings into question the timing of post-CDS data collection given the volume of changes.
Implementation of EHRs can be complex and involve interactions among many different components such as the technology, people, processes, organization, and external environment.40 These interactions in a given implementation go beyond just the EHR vendor and the associated CDS tools implemented and result in differences across organizations.40 This could explain why the same EHR product can demonstrate different process outcomes and clinical outcomes when implemented in different organizations.
LIMITATIONS
Falls occurring in other areas of the hospital or facility were not included; the falls rates examined were unit-centric and not patient-centric. Collecting patient-centric data may have resulted in different outcomes. No consideration was given to possible seasonality of patient falls, and different seasons were included in the pre- and post-CDS periods. No consideration was given to the age of patients, and this may have affected the outcomes. Quarterly audit data were collected by unit staff, and there may be differences in collection methods across units and collectors. Quarterly audit data are a point-in-time audit method and include the most recent fall risk assessment and (if indicated) the current fall prevention plan of care only. Retrospective chart review evaluating this intervention included 4.5 continuous days of patients' stays and did not include the entire length of stay. Documentation of the plan of care focused on the planned interventions for fall prevention and did not include any review of intervention completion or goal attainment.
CONCLUSION
Overall, the implementation of CDS reminders and alerts for fall risk and prevention had mixed results. Improvements were seen in the documentation of the fall risk assessment after the implementation of the EHR and embedded CDS tools. No improvement was seen for the clinical outcomes, illustrated by the lack of change in the fall rates. Managers and those seeking quality improvements should not automatically reach for CDS tools such as alerts and reminders without plans to measure the impact and any associated improvements. Measurement is important to determine whether the expected results were in fact achieved. The design of CDS tools and evaluation measures should involve clinical staff and users. Further investigation of the differences in CDSS usage by nurses and improvements in processes and outcomes across sites are needed. This will enable CDS tools such as alerts to be designed and placed in all appropriate workflows for use by clinicians.
REFERENCES