Monitoring & Evaluation - IHP+A strong country- led monitoring and review system is the foundation for policy dialogue, action and mutual accountability. In the IHP+ Global Compact, signatories commit to focus on results and increase the use of shared mechanisms for reporting on progress and reviewing performance. Country compacts almost always make explicit reference to increasing the use of national monitoring and review systems. A framework for monitoring, evaluation and review of national health strategies has been developed through a multi- partner collaboration to highlight what needs to be done and how. There are also signs of more readiness for collective action, and high level support from global health leaders. The challenge is to develop more unified, simpler approaches based on country monitoring platforms in which all parties have confidence. IHP+ encourages: Use of the national health strategy as the basis for information and accountability.
Clinical Mentoring Toolkit. Monitoring & Evaluation Tools. This section includes tools and resources that a mentor can use to assess the skills of health care.
Mechanisms such as annual performance reviews as the basis for joint review and action. Commitment to greater transparency in resource availability and use in country compacts. Country efforts to define roadmaps towards one monitoring platform, whose implementation is supported by all partners Progressive alignment of development partners.
MONITORING & EVALUATION: Some Tools, Meth ods & Approaches MONITO& RING EVALUATION: The Worl d Bank. SOME TOOLS, METHODS AND APPROACHES FOR MONITORING AND EVALUATION.
Better data is also the foundation for greater accountability. As part of efforts to promote mutual accountability, IHP+ is also encouraging the routine inclusion of aid effectiveness indicators for review during annual health sector reviews. Latest progress. At the informal meeting of global health leaders in New York on September 2. Working Group on Indicators & Reporting Burden was established.
It consists of senior focal points from the participating global health agencies, and is chaired by WHO. Under the leadership of WHO and the World Bank the technical work is being taken forward, with support from IHP+.
A meeting of the IHP+ Working Group on Information and Accountability (previously the M& E Working Group) and the Working Group on Indicators and Reporting Burden on 2. August 2. 01. 4 in Geneva issued an Outcome Statement and agreed on a list of global reference list of 1. They endorsed the meeting outcome statement, committed to promote the use of the list of core indicators and to make further progress on rationalising global reporting requirements.
This was followed by the Summit on Measurement and Accountability for Results in Health in Washington, June 2. WHO, World Bank and USAID. The Health Data Collaborative is taking forward the IHP+ principles to achieve a country- led platform for information and accountability. IHP+ has supported its establishment and operation from the onset, including being a co- sponsor. See also: Results& Evidence: Monitoring and Evaluation, Tools: M& E Platform.
Monitoring and Evaluation. Good monitoring and evaluation design during project preparation is a much broader exercise than just the development of indicators. Good design has five components. Clear statements of measurable objectives for the project and its components, for which indicators can be defined. A structured set of indicators, covering outputs of goods and services generated by the project and their impact on beneficiaries.
Provisions for collecting data and managing project records so that the data required for indicators are compatible with existing statistics, and are available at reasonable cost. Institutional arrangements for gathering, analyzing, and reporting project data, and for investing in capacity building, to sustain the M& E service. Proposals for the ways in which M& E findings will be fed back into decision making. Examples. 1. Project objectives.
Projects are designed to further long- term sectoral goals, but their immediate objectives, at least, should be readily measurable. Thus, for example, a health project might be designed to further the sectoral goals of a reduction in child mortality and incidence of infectious diseases, but have an immediate, measurable objective of providing more equitable access to health services.
Objectives should be specific to the project interventions, realistic in the timeframe for their implementation, and measurable for evaluation. India’s District Primary Education Project, for example, set out its objectives at the district level in clear statements linked directly to indicators: Capacity building: District sub- project teams would be fully functional, implementing sub- project activities and reporting quarterly on progress. In- service teams would be functioning, with augmented staff and equipment, providing support for planning and management, teacher in- service training, development of learning materials, and program evaluation. Reducing dropout and improving learning achievement: School/community organizations would be fully functional for at least half the schools, and dropout rates would be reduced to less than 1. Learning achievements in language and mathematics in the final year of primary school would be increased by 2. Improving equitable access.
Enrollment disparities by gender and caste would be reduced to less than 5 percent. Indicators. Input indicators are quantified and time- bound statements of resources to be provided. Information on these indicators comes largely from accounting and management records. Input indicators are often left out of discussions of project monitoring, though they are part of the management information system. A good accounting system is needed to keep track of expenditures and provide cost data for performance analysis of outputs. Input indicators are used mainly by managers closest to the tasks of implementation, and are consulted frequently, as often as daily or weekly. Examples: vehicle operating costs for the crop extension service; levels of financial contributions from the government or cofinanciers; appointment of staff; provision of buildings; status of enabling legislation.
Process indicators measure what happens during implementation. Often, they are tabulated as a set of contracted completions or milestone events taken from an activity plan.
Examples: Date by which building site clearance must be completed; latest date for delivery of fertilizer to farm stores; number of health outlets reporting family planning activity; number of women receiving contraceptive counseling; status of procurement of school textbooks. Output indicators show the immediate physical and financial outputs of the project: physical quantities, organizational strengthening, initial flows of services. They include performance measures based on cost or operational ratios. Examples: Kilometers of all- weather highway completed by the end of September; percentage of farmers attending a crop demonstration site before fertilizer top- dressing; number of teachers trained in textbook use; cost per kilometer of road construction; crop yield per hectare; ratio of textbooks to pupils; time taken to process a credit application; number of demonstrations managed per extension worker; steps in the process of establishing water users' associations. Impact refers to medium or long- term developmental change. Early indications of impact may be obtained by surveying beneficiaries' perceptions about project services.
This type of leading indicator has the twin benefits of consultation with stakeholders and advance warning of problems that might arise. Examples of impact: (health) incidence of low birth weight, percentage of women who are moderately or severely anemic; (education) continuation rates from primary to secondary education by sex, proportion of girls completing secondary education; (forestry) percent decrease in area harvested, percent increase in household income through sales of wood and non- wood products. Examples of beneficiary perceptions: proportion of farmers who have tried a new variety of seed and intend to use it again; percentage of women satisfied with the maternity health care they receive. Collecting Data and Managing Project Records. The achievement of project objectives normally depends on how project beneficiaries respond to the goods or services delivered by the project.
Evidence of their response and the benefits they derive requires consultation and data collection that may be outside the scope of management. It is important to identify how beneficiaries are expected to respond to project services, because managers will need evidence of that response if they are to modify their activities and strategy. Indications that beneficiaries have access to, are using, and are satisfied with project services give early indication that the project is offering relevant services and that direct objectives are likely to be met.
Such evidence - market research - may be available sooner and more easily than statistics of impact such as changes in health status or improvements in income. Market research information is an example of a leading indicator of beneficiary perceptions that can act as a proxy for later, substantive impact. Other leading indicators can be identified to give early warning about key assumptions that affect impact. Examples would include price levels used for economic analysis, passenger load factors in transport projects, and adoption of healthcare practices. When planning the information needs of a project there is a difference between the detail needed for day- to- day management by the implementing agency or, later, for impact evaluation, and the limited number of key indicators needed to summarize overall progress in reports to higher management levels. For example, during construction of village tubewells, project managers will need to keep records about the materials purchased and consumed, the labor force employed and their contracting details, the specific screen and pump fitted, the depth at which water was found, and the flow rate.
The key indicators however, might be just the number of wells successfully completed and their average costs and flow rates. Exogenous indicators are those that cover factors outside the control of the project but which might affect its outcome, including risks (parameters identified during economic, social, or technical analysis, that might compromise project benefits); and the performance of the sector in which the project operates. Concerns to monitor both the project and its wider environment call for a data collection capacity outside the project and place an additional burden on the project’s M& E effort. A recent example of a grain storage project in Myanmar demonstrates the importance of monitoring risk indicators. During project implementation, policy decisions about currency exchange rates and direct access by privately owned rice mills to overseas buyers adversely affected the profitability of private mills. Management would have been alerted to the deteriorating situation had these indicators of the enabling environment been carefully monitored.
Instead, a narrow focus on input and process indicators missed the fundamental change in the assumptions behind the project. The relative importance of indicators is likely to change during the implementation of a project, with more emphasis on input and process indicators at first, shifting to outputs and impact later on. This is a distinction between indicators of implementation progress and indicators of development results. Data collection Project field records. Indicators of inputs and processes will come from project management records originating from field sites.
The quality of record keeping in the field sets the standard for all further use of the data and merits careful attention. M& E designers should examine existing record- keeping and the reporting procedures used by the project authorities to assess the capacity to generate the data that will be needed.
At the same time, they should explain how and why the indicators will be useful to field, intermediate, and senior levels of project management. The design of field records about, say, farmers in extension groups, people attending a clinic, or villagers using a new water supply, will affect the scope for analysis later. The inclusion of simple socioeconomic characteristics such as age and sex may significantly improve the scope for analysis. A good approach is to structure reporting from the field so that aggregates or summaries are made at intermediate stages. In this way, field staff can see how averages or totals for specific villages or districts enable comparisons to be drawn and fieldwork improved. Surveys and studies.
To measure output and impact may require the collection of data from sample surveys or special studies (including, where appropriate, participatory methods).