Purpose
The evaluation is intended to mainly contribute to internal learning (‘improving’) and accountability towards our stakeholders (‘proving’):
Learning: insights and knowledge gained through the evaluation will contribute to the steering and implementation of the ongoing and planning of the future phase of the KNOWHOW3000 programme 2019-2022. Examples of good practices that have helped maximise results is an integral part for such learning.
Accountability: the outcome of the evaluation will deliver insights on how the KNOWHOW3000 Programme achieves results. It will be crucially important for the accountability towards stakeholders, especially the HORIZONT3000 member organisations and the Austrian Development Agency.
Recommendations: The evaluation report will be valuable in terms of recommendations on how to increase positive impact in the field of knowledge management for local partner organizations as well as possible improvements of the management, steering and coordination processes involved in the implementation of the programme.
Objective
The main objective of the evaluation is to assess the fulfilment of goals and achievement of expected results of the KNOWHOW3000 Programme in East Africa (Uganda, Kenya, Tanzania). The evaluation shall assess the relevance, effectiveness, efficiency, impact and sustainability of the KNOWHOW3000 programme, as well as the involved (project) management processes for implementing the programme in the programme region in focus (East Africa).
Focus and Scope
The evaluation shall assess the design, implementation and results of the KNOWHOW3000 Programme in East Africa.
Due to the geographical focus, the evaluation will mainly focus on result 1: Learning & Sharing: HORIZONT3000 partner organisations and Member Organisations partner organisations are enabled to learn from their and/ or other organisations’ experiences. Accordingly it will focus on the main direct target group of Result 1: HORIZONT3000 partner organizations, who participated in KNOWHOW3000 activities under Result 1.
Assessments concerning Result 2 (Internet Platform) and Result 3 (Organizational Development) shall be included whenever possible. HORIZONT3000 Technical Advisors and HORIZONT3000 regional staff are considered the primary target group of activities under R2 and R3 in East Africa.
The evaluation shall assess the current phase of the programme (2016, 2017, first half of 2018) and also take into account the last year of the previous phase of the programme (2015). Thus, the period to assess is 3.5 years.
Main guiding questions
The evaluation shall assess the fulfilment of the main objective of the KNOWHOW3000 Programme, the intervention logic of the programme and the relevance, effectiveness, efficiency, impact and sustainability of the programme in East Africa by addressing the following main guiding questions:
Relevance
To what extent is learning and knowledge sharing important for improving partner organization’s performance?
· Are the topics addressed and instruments applied appropriate to achieve the desired objective?
To what extent does the planning and implementation process involve the partner network in East Africa?
To what extent are cross-cutting topics (gender equality, poverty reduction, democracy and environment) considered in the planning and implementation of activities?
Effectiveness
To what extent has the objective of the programme been achieved in East Africa? Which are relevant factors for (not) achieving the objective?
What kind of evidence exists concerning the contribution of the programme to strengthened capacities and improved performance of partner organizations?
Which instruments did (not) prove to be effective in order to support learning and sharing processes at and among partner organizations (R1)?
Do partner organizations use capacities developed by the programme to strengthen organizational learning processes? If yes How? If no why?
To what extent do the internet platform and the OD toolbox support their main target groups (Technical Advisors and programme officers) in East Africa in their work with partner organizations (R2+R3)?
How effectively are synergies with the Technical Assistance Programme used in East Africa (e.g. borrow-a-Technical Advisor, etc.)?
Efficiency
Have financial resources and inputs been used efficiently to achieve objectives and results?
How do participants who have attended training workshop/knowledge management activites transfer the knowledge and skills gained into their organisation?
Are there alternative, more cost-efficient measures to achieve envisioned results?
What are the strengths and weaknesses of the (project) management processes of the programme in East Africa (internal systems and responsibilities)?
Impact
What was the ultimate benefit of the participation in KNOWHOW3000 activities for the partner organisation?
To what extent was the programme exemplary, created structures and had an impact in terms of leverage (e.g. adaption at target groups)?
Sustainability
How far are attained improvements and changes sustainable? Does knowledge gained/ capacities developed stay in the organization; is it anchored in the organization?
To what extent are economic, social and cultural factors taken into account in order to support sustainability of the results?
To what extent has ownership and leadership for the programme been fostered among partner organizations and the regional office?
Methodology
It is up to the consultants, to suggest the methodology and action plan as part of the offer and it is expected that the consultants clearly define the intended approach.
For the desk study, the consultants will be provided with the following documents:
KNOWHOW3000 programme in general: Application of the framework programme and the programme intervention; yearly progress reports 2016 and 2017; Final report 2015; review report of the previous project phase (2013-2015); management response (to the previous review); access to the internet platform www.knowhow3000.org;
KNOWHOW3000 programme in East Africa: Documentation Reports and Survey Results of KNOWHOW3000 events (trainings, sharing events, etc.); Annual Planning of Activities 2018
Additional material: overview/ description of Technical Advisor-Programme; HORIZONT3000 Regional Strategy for Eastern Africa
Due to the characteristics of the programme, data collection shall focus on direct target groups, and not cover indirect beneficiaries (target groups of the partner organization’s projects). It is suggested that the following actors, stakeholders and beneficiaries are involved in visits, interviews and/or surveys:
Key programme staff in the Regional Office East Africa (Director and Programme Officers)
Programme Coordinators East Africa and Knowledge Management Team at HORIZONT3000 Vienna
Project desks at member organizations having participated / having had a role in the KNOWHOW3000 Programme in East Africa
Partner Organizations, who participated in KNOWHOW3000 activities during the period of assessment: not less than 9 Partners in Uganda, 3 Partners in Kenya and 3 Partners in Tanzania (Consultants will be provided with an overview of partner organizations and the activities they participated in)
Selected Technical Advisors, who participated in KNOWHOW3000 activities
Data should be collected and interpreted if possible in a gender-disaggregated manner. OECD DAC Evaluation Quality Standards are to be applied and the compliance of the latter needs to be comprehensible in the evaluation.
Plan of Activities and Deliverables
Besides appointments, meetings and interviews for the data collection according to the methodological approach of the consultants, the plan of activities shall include the following two meetings:
Kick-off meeting in the Regional Office East Africa (ROEA) in Kampala: After the desk review is conducted, a meeting shall take place in order to review the action plan, which includes a programme of activities as well as instruments and methods to be applied for the development of the evaluation report. The consultant is also required to prepare and submit an inception report prior to conducting field visits.
Feedback meeting in the ROEA Kampala: After completing the data collection, a meeting shall take place where provisional results are presented and discussed with the Regional Director and Programme Officers in the ROEA.
The consultants shall present a draft report at latest two weeks after the presentation of provisional results in the feedback meeting. The draft report will be assessed by HORIZONT3000 as to its comprehensibility and comprehensiveness and the consultants are expected to integrate feedback in the final report.
The final report shall have a maximum of 20 pages (without annexes and index) and include: title page, table of content, list of abbreviations, executive summary, background, introduction, methods, findings, conclusions, lessons learnt, recommendations and Annexes (Logframe, terms of reference and schedule of the evaluation, list of key informants, list of documents used, questionnaires or other instruments used; Reports prepared for the field study; Information regarding the evaluators, and other document the consultants consider relevant).
Coordination
The general coordination of the evaluation including the selection, contracting and feedback processes is assumed by Kristina Kroyer, Project Coordinator Knowledge Management at HORIZONT3000 Vienna
The regional office in Kampala is responsible for the local and operative coordination of the evaluation and will provide back-up services for logistical and technical issues. Person in charge in the regional office is Jennifer Okusia, Senior Programme Officer for financing projects
Time Frame
The evaluation shall take place between June and September, with the final report completed by latest end of September 2018.
Budget
The total budget available for the evaluation is max EUR 15.000.
This value must cover all costs relating to the assignment including fees, taxes, food, accommodation, national/international transportation, office materials, insurances, etc.;
Payment of the fee will be made in two tranches:
50% at signing of the contract
50% after submission of final report[1] Development cooperation agency of the Austrian Catholic Children’s Movement – DKA, Austrian Catholic Men Movement – KMBÖ, Austrian Catholic Women Movement – kfb, Bruder und Schwester in Not – Diocese Innsbruck, Bruder und Schwester in Not – Katholische Aktion Kärnten, Caritas Austria, Welthaus of the Diocese Graz-Seckau, plan: g, as well as Referat Mission und Entwicklung of the Archdiocese of Vienna
