Review Background
Afya Ugavi (AU) Activity is in collaboration with the Government of Kenya’s (GoK) Ministry of Health (MoH) and other stakeholders since October 2016 through to September 2021 to catalyze a new level of health commodities supply chain performance, achieved through supportive activities. This is aimed at strengthening pharmaceutical supply chain systems for HIV/AIDS, malaria, Family Planning/ Reproductive Health, Maternal, Newborn, Child and Adolescent Health (FP/RMNCAH) at the national level and in fourteen (14) high burden, under-resourced and underserved counties. The Activity goal is to ensure there is a well-functioning, high performing supply chain system that provides communities with adequate, safe, affordable, and reliable supply of health commodities, when and where needed. Activity coverage includes 4 counties (Homa Bay, Uasin Gishu, Isiolo and Mombasa) for HIV/AIDS activities, 8 lake malaria endemic counties (Kisumu, Homa Bay, Migori, Siaya, Busia, Bungoma, Vihiga and Kakamega) and 3 northern frontier counties (Baringo, Samburu and Turkana) for FP/RMNCAH. Similar technical support is extended at national level for the constituent programs which include National AIDS/STI Control Program (NASCOP), National Malaria Control Program (NMCP) and the Department of Family Health (DFH). Additional support in line with the Global Health Security Agenda (GHSA) plans to improve Kenya’s capacity to help create safe and secure environment from infectious disease threats and elevate global health security as a national and global priority. The Activity goal is to ensure “Strengthened health supply chain and commodity management systems” at national and county levels.
Expected outcomes aoutputs are described as follows;
Outcome 1: Strengthened national level systems for supply chain and commodity management, with two contributory outputs;
Output 1.1: Strengthened technical leadership and coordination for commodity management, and
Output 1.2: Improved supply chain logistics and commodity management.
Sub-Purpose 2: Strengthened county oversight and implementation of supply chain services through the following expected outputs;
Output 2.1: Increased human and institutional capacity for commodity management, and;
Output 2.2: Improved commodity security environment
At the same time, USAID has also been reviewing its support towards supply chain interventions to ensure a consistent focus on interventions across the health sector. A part of this midterm review, and in addition to evaluating if AU is working in accordance with its stated objectives, the mission is also intent to evaluate whether all the funded implementing partners are working in accordance with the mission’s new supply chain priority objectives which include;
Standardizing supply chain services at the facility level through a standard package of services
Reducing overlaps in supply chain support between the service delivery partners and the mission’s supply chain partner (AU)
Reducing the current supply chain main risks, especially in light of the results of recent commodity audit reports and increasing/improving supply chain governance
Evaluation of these priority objectives in detail alongside Afya Ugavi output areas will assist the mission to align its future support appropriately, with focus on areas of weakness and emphasis on collaboration on others that cut across the partners in line with the above stated objectives.
Purpose of the Mid Term Review
Afya Ugavi Activity has been implemented for two years already in its five-year implementation lifespan, which necessitates for an audit of its performance so far against its stated objectives. The purpose of this review is therefore to assess progress made towards achievement of the overall and specific objectives of this Activity, its impacts on the target stakeholders who have been directly involved in implementation, as well as with policy makers at county and national levels up to this implementation period. In addition, thisreview will generate recommendations on how the Activity can incorporate the mission’s new/emerging thinking around supply chain and potentially trigger a revision of the implementation process including development of new activities for inclusion in subsequent workplans. The assessment will also help determine any impact the project has produced in the targeted counties and national level bodies, as well as improvement of disaster preparedness. The evaluation will also gauge the level of beneficiaries and other stakeholders’ participation and ownership of the implementation process including identification of the intended and unintended outcomes, best practices and lessons learned as well as challenges arising from Activity implementation. In overall, the evaluation will come up conclusions and recommendations on the way forward which will be targeted for inclusion in subsequent workplans.
The Mid-Term Review therefore has four main objectives:
to evaluate progress towards meeting the stated goal, intermediate results, and objectives of the project;
to evaluate the systems in place to measure and report on results; and
recommend actions required in the remaining project period to ensure that the stated goal, intermediate results, sub-intermediate results and sub- objectives are fully met.
To provide recommendations to USAID on how they can use Afya Ugavi and other service delivery partners to implement their newly defined supply chain priorities.
The MTR will also determine the sufficiency, effectiveness, sustainability and impact of the interventions.
The MTR will be participatory, involving AU, Ministry of Health (MoH), USAID local mission staff, local implementing partners, County and Sub County health leadership and related health commodity management staff at facility level.
Specific Tasks of the Consultant
In order to realize the evaluation objectives stated above, the Consultant is required to carry out and report on the following tasks;
Evaluate progress towards meeting the stated goal, intermediate results, and objectives of the project;
Compare baseline with mid-term progress and assess whether the objectives and outcomes areon track to being achieved;
Assess each program intervention disease area and document progress against the Annual Work Plans (AWPs) and the MEL plans for 2016/17 and 2017/18.
Assess and comment on unplanned results, unexpected constraints and new interventions notincluded in the AWPs;
Assess tools, manuals, guidelines or approaches that the program developed or used, their suitability towards health supply chain development and their suitability in the current devolved environment;
Assess how the data or information emerging from, continuous assessments, health commodity system audits operational research and any other special studies carried out by the project has been impactful in implementation;
Assess achievements in other cross intervention areas such as data review forums, facility in charges meetings, and supportive supervision among others, and how these multi-sectoral approaches have led to a holistic improvement in supply chain and the health service delivery continuum;
Assess any modifications of interventions so far and comment on whether these modifications are justified and record their outcomes;
Document innovative interventions and strategies being used by the Activity and provide, as a highlight, an innovative idea, promising practice, or good practice that is being advanced by this program, providing as much evidence-base as possible at the time of the evaluation;
Document good practices and lessons learned in the implementation of the project; and
Comment on the sustainability and potential replicability of interventions, especially with in terms of integration into the health delivery system at facility level.
Evaluate existing systems for measuring and reporting on results:
Provide an assessment of the project’s ability to implement workplans to adequately produce expected results;
Review the current Monitoring & Evaluation Plan, and specifically the Activity indicators to determine their relevance and alignment to the scope and depth and provide suggestions for improvement.
Review monitoring tools and data collection methods to draw conclusions about their
effectiveness in measuring the project’s performance and make suggestions for improvement as needed; and
Assess the Activity’s implementation in relation to financial expenditure, existing gaps and measures towards financial security.
Identify issues, risks and challenges within the project, which can be obstacles in achieving project results, and provide recommendations to overcome these.
Recommend actions required in the remaining project period to ensure that the stated goal, intermediate results, sub-intermediate results and sub- objectives are fully met:
Suggest realistic modifications necessary for the remaining duration of the project based on perceived shortcomings and gaps in project financial planning, implementation and project management structure;
Present specific recommendations for USAID, the Ministry of Health, County Health Management Teams and collaborating partners regarding the remaining period of the project that will ensure that all objectives are met;
Provide recommendations for potential extension or future phase of the project beyond its current funding period; and
Provide recommendations to USAID on how Afya Ugavi and other implementing partners’ interventions can be realigned alongside the mission’s supply chain priority objectives.
Evaluation Methodology
The consultant will be responsible for defining and carrying out the overall MTR evaluation approach. This will include specification of the techniques for data collection and analysis, structured field visits and interactions with the beneficiaries and the implementation teams, and the preparation and presentation of draft findings and recommendations.
The evaluation methodology should include both quantitative and qualitative evaluation methods, with focus on interpretation of the results, internal review of the activities and outcomes based on the Activity AWPs, progress reports and other related databases, field visits/observations which may include discussions with stakeholders, and any other measures as deemed necessary by the consultant.
On overall the process must speak to the following broad evaluation questions;
Has the right implementation been done? (Has the implementation outcome so far been relevant, appropriate and strategic to national/county level supply chain development goals?
Has the implementation been done right? (Were the actions to achieve the outputs and outcomes effective and efficient?);
Are the results sustainable? (Will the achieved and projected outputs and outcome(s) lead to benefits beyond the life of the existing project(s)?);
How might we do things better in the future? (Which findings may have relevance for future programming or for other similar initiatives elsewhere?).
Responsibilities
The consultant will be responsible for:
Reaching an agreement with Afya Ugavi on the work plan of activities as early as possible, but no later than 1 week prior to the commencement of the review;
Developing all the data collection tools to be used in the review;
Conducting the mid-term review, including the day-to–day management;
Conducting periodic de-briefs with the Project Management Team and key staff
Present the draft findings to the Activity and other invited participants on the final day of the review; and
Preparing and submitting a final MTR report to Afya Ugavi.
Afya Ugavi will also be responsible for the following;
Facilitating the consultants travel and logistical support to the AU offices outside Nairobi
Providing copies of required reference materials including progress reports, inception reports, financial information among others
Periodically reviewing progress of the MTR, and participating in ad hoc review meetings an providing an advisory role where required
Deliverables
At the conclusion of the MTR process, the following deliverables will be expected from the consultant;
A detailed concept note/inception report for the evaluation with the consultant’s understanding on the following areas;The Activity’s goals, mandate and coverage USAID Framework and Strategic Objectives;
An elaborate description of the evaluation methodology;
Annexes which include tools or instruments created and/or used for the review; a copy of the review schedule; and a list of those proposed to be interviewed with their titles/organizations.
Debriefing meeting with Afya Ugavi and invited guests on preliminary findings at the end of the field assessment and draft report writing to share preliminary findings and gather feedback for writing the final report.
A draft MTR report prepared and shared with AU for review and comment within seven days after the debriefing meeting. The consultant report will address all points under section 3.0 of this TOR along with annexes which should include tools or instruments used for the evaluation, theevaluation schedule and a list of people interviewed or consulted.
A final project evaluation report due within five days of receiving feedback on the draft report from Afya Ugavi
MTR Implementation Time Frame
The MTR consultancy is planned for a period of 30 consultancy days which are detailed as follows;
No Task No. of consultancy days
Draft concept note/inception report – 3 days
Field visits, data collection, interviews, desk study and data analysis – 20 days
Debrief/validation meeting to present preliminary findings to AU – 1 day
Draft report writing – 4 days
Final MTR report after incorporating comments from AU review – 2 days
TOTAL 30
Required Expertise for the Consultant
The following are the qualification requirements for the MTR consultant;
Preferably a Kenyan citizen or possessing legal requirements to work in Kenya.
At minimum, a Masters degree in the health sciences, social sciences, behavioral sciences, health systems research, Health supply chains or a related field.
Previous experience in conducting of project/program evaluations/reviews, with evidence of work done.
At least 5 years’ experience in M&E, health systems strengthening, public health, heath supply chains management and/or capacity building of public sector entities
Excellent knowledge of the health sector in Kenya, particularly on supply chain systems for HIV, malaria and FP and essential medicines.
Excellent verbal, interpersonal and written communication skills.
Strong analytical, problem-solving and decision-making capabilities.
Team player with the ability to work in a fast-paced environment.
Strong skills in stakeholder engagement and consensus building.
Computer proficiency – Microsoft office suite applications.
Experience with a previous USAID-funded program implementation or assessment.
Supervision and Evaluation
The Consultant is supervised and evaluated by the Monitoring and Evaluation Specialist
