Researchers at the World Bank, University of Maryland and University of California Berkeley, have developed new techniques to study the supply side of health care—in particular, the quality of medical advice that doctors provide. Supply side research identifies the determinants of the quality and pricing of clinical advice, working hours, referral, and marketing decisions of the providers. The characteristics of providers identified through these tools can also be used to understand how households choose among multiple providers.
There are a couple of reasons why this research is important for policy. First, the quality of medical advice is likely an important determinant of the care; perhaps more so than (say) the availability of medicines or the number of rooms in a primary health care setting. Therefore, measuring the quality of medical advice can yield different insights and degrees of variation relative to other quality measures used in the literature, such as the quality of infrastructure. In fact, we find that there is little correlation between structural measures of quality and the quality of medical advice. Second, the quality of medical advice itself may be determined by several factors. For instance, the research thus far shows that provider effort (what providers do) may be more important than provider competence (what providers know) in determining the care a patient receives. This suggests that policy designed to improve effort (such as better informed patients or stronger incentive schemes for doctors) will have a greater impact on health care than those that increase competence (such as training).
The measurement of provider quality has proved quite informative. For example, doctors in Tanzania complete less than a quarter of the essential checklist for patients with classic symptoms of malaria, a disease that kills 63,000-96,000 Tanzanians each year. A public sector doctor in India asks one (and only one) question in the average interaction—“What’s wrong with you?” In Paraguay, the amount of time a doctor spends with a patient has nothing to do with the severity of the patient’s illness. These isolated facts actually represent common patterns; understanding these patterns can yield insights on the appropriate policy levers for improving health care delivery.
This website provides information on the techniques employed by the research team to further understand quality of medical care through measuring provider competence and provider effort, and understanding the gap between the two.
Measuring Quality of Care
In measuring the quality of care, the endpoint is the answer to a simple question: “For a patient with a set of symptoms and an underlying illness, what percentage of essential questions and what percentage of essential examinations does a doctor actually perform?” In further understanding patterns of variation, it has been useful to decompose differences into those arising from differences in competence and those arising from differences in effort. The former has been measured through clinical vignettes and the latter through direct clinical observation (DCO), combined in some cases with exit surveys. The clinical vignettes standardize the case-mix and the patient-mix while the direct clinical observation controls observationally for patient characteristics. More sophisticated tools such as simulated patients are currently being developed; these will allow us to answer the quality of care question more accurately in a clinical setting.
To date, these techniques for measuring quality of care have been used for field research by World Bank and other teams in Argentina, Jamaica, India, Indonesia, Paraguay, Rwanda, and Tanzania. Lessons learned from several country projects are detailed on this site:
Case Studies
Measuring Provider Competence
A fundamental element in measuring the quality of medical advice is provider competence. Provider competence is defined here as the percentage of essential questions and examinations that a doctor completes for a specific medical case and the likelihood of reaching a correct diagnosis. In parallel research Das and Hammer (2007) and Leonard (2007) developed the approach for measuring provider competence in low-income countries and validated their measures using psychometric tools and common-sense correlations, such as that between training and measured competence (positive). This experience has contributed to additional provider vignettes for studies in Argentina, Indonesia, Jamaica and Rwanda.
In addition to provider responses to vignettes, data collected using standard facility/provider questionnaires also offers more traditional structural quality measures that can be used independently, or in conjunction with provider vignettes.
Vignettes - Questionnaires - Program for Scoring Vignettes
Measuring Provider Effort
The most comprehensive methods for measuring provider effort are direct observation of the provider’s effort during consultations; combining these with exit interviews can be helpful in understanding the extent to which patient characteristics affect doctor behavior (and the extent to which doctor behavior affects patient satisfaction). Both methods involve collecting data on what the provider did, given the patient’s history, examination and treatment, during a consultation. In addition, exit surveys collect information on the patient’s background and (possibly) health status and satisfaction with the interaction.
Patient Exit Interviews - Direct Observation
Argentina
India
Rwanda
Tanzania
RSS