Why It Matters
During the time of year often dreaded by people working in the Monitoring and Evaluation department of a hospital – the quarterly report submission period – my colleagues and I discovered a surprising discrepancy in our data. The number of deaths reported through the inpatient wards was low compared to the number reported through the hospital morgue unit.
On further exploration, we found this type of under-reporting (and sometimes over-reporting) had become a common problem in multiple data elements. It goes without saying that quality data is crucial for planning, resource allocation, and performance monitoring and evaluation. Poor quality limits the usefulness of such data for informed decision making. Our experience gave birth to our journey to improving data quality in the hospital.
Quality improvement team members and the hospital CEO gather together.
Laying the Foundation for Change
To identify opportunities and challenges, we began with the context of current practices at Zewditu Memorial Hospital in Addis Ababa, Ethiopia. We did an in-depth analysis of our performance by adapting a USAID tool, the Data Quality Assessment (DQA) Checklist. This tool defines several dimensions of quality data: completeness, validity, consistency, timeliness, and accuracy. These core attributes were used in a baseline assessment of our data with a result of 41 percent (out of 100 percent) across the dimensions.
Next, we set out to truly understand the contributory factors to this low score. We employed a Fishbone Diagram (a diagram visualizing the factors in an organized, simple, and easily understandable way), which helped determine the root causes of challenges by brainstorming with each data unit’s leaders. By dissecting the problem and recording its possible causes, the team illuminated potential solutions.
The teams used a Fishbone diagram to investigate possible causes of poor data quality.
Creating Solutions
The team, empowered by participating in this exploration, was ready to tackle the problems through these steps:
- Team formation and engagement: We developed a multidisciplinary team composed of case team leaders (nurses), department directors (general practitioners), health information technicians, a monitoring and evaluation expert, and a quality improvement advisor. The team created a flow chart of the existing data management process, from data capture to final entry into the database, as well as detailed charts for each individual service delivery unit. We matched the steps in the process with the owners and defined key process and outcome measures for tracking improvement.
The team developed flow charts to explore the existing data management process.
- Capacity building and system re-design: Data owners and data stewards were identified and trained in the new desired flow of data by experts from the Ministry of Health.Senior management updated data cycles and job descriptions, and the Monitoring and Evaluation leadership developed tracking tools for each domain of data quality. The team established mechanisms for monthly review of data by the stewards, followed by reporting to the Health Management Information System (HMIS) unit. We re-designed tools for data management and created protocols using human factors principles to enhance engagement and improve user-friendliness. These guides were made available to all participating units. Each team gave a self-assessment report and was verified by directors and performance monitoring team.
The team proposed a revised process for the data management cycle.
Reporting, communication, and feedback loops:The team established monthly forums for data presentation, discussion, and decisions on action items for each data unit. Dashboards were provided for each unit to track their performance. A hospital performance monitoring team, led by the Medical Director and CEO, gave feedback to each team on monthly basis and a system of accountability was put in place to check timeliness, completeness, and correctness of data. This data is verified by a random sampling method, lot quality assurance assessment.The multidisciplinary team shared responsibility for data management.
The team used multiple tools to track data quality.
- Recognition and reward: The team established a recognition scheme for data quality improvement based on the data quality assessment tool score. High performing departments were rewarded with local scholarships, certificates of appreciation, and trophies.This reward system created a sense of competition and motivation to further strengthening the data management system.
Results and Sustaining the Gains
Initially, the data ownership issue was a big challenge for this project. Clinicians believed the HMIS unit held responsibility for data management and did not pay attention to data, thus the data was not representative of reality.
Involving the senior management team, following up frequently, and demonstrating data utilization for decision making changed the culture. Now, a performance monitoring team tracks data quality and compares it to the hospital’s plans. The clinicians know the value of the data for decision making and pay attention to quality issues with the data. As a result, they decided data quality should be one of the criteria for the recognition program led by hospital management. Working on data quality is prioritized.
In the first three months, the data quality score improved from 41 percent to 77 percent. Three months later, the score further improved to 80 percent. Involving improvement teams and senior management in multiple interventions led to significant improvement in data quality. Encouraging end-users of data to recognize the gaps in quality and how the data affects their clinical decision fostered ownership and leadership of the process. Furthermore, tying data quality to a reward and recognition system boosted the motivation of staff and ensured sustainability of the improved data management system.
What started out as a daunting task has evolved into a source of joy and pride as we confidently provide data to management for use. We are assured that with this change in the data management process, we will provide better care for our patients thanks to quality data.
Ahlam Mahmoud is a Monitoring & Evaluation Senior Technical Advisor at Zewditu Memorial Hospital.