Terms of Reference for Midline study
Background
The goal of the ADSI programme is to consolidate and develop a holistic expansion of the demonstrably successful aspects of its Strengthening Innovative Practice in Secondary Education (SIPSE) pilot model (2013-2015) in relation to its innovation practice, whole school approach, horizontal (geographic) and vertical (institutional) outreach and impact, and shared ownership. The ADSI model presents a portfolio of system- wide ICT innovation elements that can address policy coherence needs for ICT integration in teaching and learning – inclusive of: a blended learning teacher development approach, whole school involvement, school leadership capacity building, converging technologies of e- and m-learning, use and development of open education resources, an online repository of materials, digital school awards, accreditation and certification to incentivize ICT integration and progression, policy dialogues to raise awareness and influence new policy formulation and institutionalization of new/good practice. Currently, ADSI covers 80 schools (20 in each of the four counties of Kiambu, Nyamira, Narok & Taita Taveta), 80 secondary school principals, 800 teachers and 80 Boards of Management chairpersons and 80 parents association chairpersons.
In Kenya, a baseline study was carried out in quarters 3 &4 of 2016 to inform the project and set the benchmarks upon which the project would be monitored and evaluated. The project has been running for more than one and a half years now in Kenya which is an opportune time to carry out a midline study. The study will help ADSI with the evidences, insights on the extent of achievement of objectives, factors that facilitate and impede the progress, and provide recommendations for midcourse corrections, if any.
Midline objectives
The overall objective of the midline study is to assess whether and/or the extent to which targets have been achieved thus far in the project, understand the determinants enabling and/or disabling achievement of targets, the extent of ownership built among the target groups, capture key learnings from the implementation experience and recommend doable actions for the remaining project period.
Specific evaluation objectives are to:
Assess the project progress/changes (since baseline) of the project so far and its likelihood of achieving its stated objectives (more than a year since implementation began) on the following aspects;
Institutionalization: Investigate the current level of institutionalization of national strategies for the pedagogical integration of ICT use in STEM and other subject teaching as related to policy in teacher development, curriculum, pre and –in service, school support and management practices
Digital Schools Development: Establish the current conditions, needs, resources and priorities of the schools in relation to ICT in STEM teaching and learning
Teacher Development: Establish changes in teacher competencies and practices for ICT use in STEM since baseline
Student Learning: Measure learner 21st century skills and other learner achievements in relation to STEM subjects and gather feedback on their experiences and attitudes towards the use of ICT in STEM learning inside and outside of schools.
Assess the ADSI programme in terms of relevance, effectiveness, efficiency of project implementation and level of ownership by project beneficiaries and other actors
Assess the ADSI programme in terms of effectiveness of networking with the different stakeholders for optimal leveraging of resources and sustainability benefits.
Document lessons on what works/doesn’t work in the implementation of ADSI approach and Theory of Change.
Generate practical, action-able recommendations that can be implemented by the project actors in the remaining phase of the project.
The proposed Methodology and Approach
As was the case with the baseline, a mixed method approach employing both qualitative and quantitative techniques in the collection of data will be used:
Data collection will be carried out using five different approaches:
Desk Review: A thorough assessment of relevant existing project documentation including project conceptual documents, progress reports, work plans, activity reports, available data, monitoring frameworks, etc. will be carried out.
Observation: Observations will be carried out at the school level to collect administrative data, audit the level of ICT infrastructure and equipment at the selected schools and observe classroom practices.
Quantitative survey (a representative sample of the population of students, teachers): A representative survey to help provide a clearer/specific picture from the target group. Surveys allow for a comprehensive acquisition and aggregation of statistical data that is easy to analyze and interpret. The statistical representation will also allow for segmentation and sub cluster analysis of the data collected.
Focus Group Discussions (At least 6 in each of the counties): To complement the data, this study will also employ a qualitative approach to data collection, and fieldwork in the form of focus group discussions (FGDs) with carefully selected members of the populations (teachers, students, EWG members, BoM members, etc.).
Key Informants In-Depth Interviews (Approximately 10 each of the counties): It will also be important to understand the perspectives of the various stakeholders in an in-depth manner. Key Informant Interviews will be carried out with: SBCs, HTs, representatives of key agencies/partners, government representatives, etc. The decision on who to be interviewed will be made jointly with ADSI project team.
GESCI is seeking a Consultant/Team to conduct the midline evaluation study including; data collection, analysis and reporting. The experts are expected to carry out the assignment in consultation with the GESCI-ADSI technical team comprising of MERL Specialist, Programme Manager, the Kenya Project Manager and the Education Specialist.
Roles and Responsibilities
Under the guidance of the MERL Specialist, the role of the Consultant is to:
Mobilize and provide a team of 16 qualified research assistants who will carry out data collection and digitization (estimated to take about 5 days for each).
Support the conceptualization, design and refining of tools for the midline study.
Present an inception report with a proposal for carrying out data collection, analysis and reporting.
Support training of the enumerators and piloting of the instruments.
Carry out midline data collection from 10 schools per county jointly with the GESCI MERL Specialist and Project Coordinators as per the guidelines provided during the training.
Manage the digitization of the data in survey monkey and Excel matrix sheets as per the guidelines provided.
Hand over the field data in hard copy format and digitized formats as per the guidelines
Carrying out the data analysis inclusive of
Midline data cleaning
Midline data analysis
Report write-up integrating
A draft midline report
A final report integrating revisions based on technical review from GESCI/ADSI.
GESCI will provide the following:
Travel from Nairobi to the counties and all the schools where data will be collected and back.
Provide US$50 per day (inclusive of accommodation, meals and honorarium) for the data collectors. The data collectors staff time cost is to be considered by the consultants as part the proposal.
Online
First draft of baseline study report – with preliminary findings for review by GESCI-ADSI-technical team
October 15th – 18th, 2018
Final study report (integrating revisions based on feedback form GESCI-ADSI technical team) and consolidated toolkit (on baseline study approaches and instruments)
Online
Final midline report
Indicative Schedule of key Deliverables
Inception report – with preliminary desk and data collection review and enhanced methodology and plan for carrying out baseline data analysis and report write-up – 10 August, 2018
Fieldwork report – mixed methods analysis of interview, focus group, survey and classroom observation data sets – 18 September 2018
First draft baseline study report presentation for ADSI management feedback – 28 September 2018
Final baseline study report integrating revisions based on feedback and the baseline toolkit of approaches and instruments – 12 October 2018
Required skills and Experience
To be considered for this role you must:
Possess an advanced Degree in research, monitoring and evaluation, education, education research or related fields.
Have a minimum of six (6) years’ experience with similar tasks preferably in the international development sector. Experience with conducting research on education thematic/ICT in Education areas will be an added advantage.
Have good knowledge of the Kenyan socio-political, cultural context and basic education landscape.
Possess advanced analysis, interpretation and writing skills.
Duration of task completion/scheduling
The assignment will take place over the months of August-September 2018 with following tentative allocation of working days (either consecutively or divided into two periods):
Desk study / preparation: 2 days
Consultations in Nairobi: 1 day
Writing Inception Report: 1 day
Support training workshop in Nairobi: 2 days
Data collection: 6 days
Analysis and draft report: 5 days
Discussion with GESCI/ADSI team: 1 day
Incorporation of comments and Final report: 2 days
Payment schedule:
20% upon approval of Inception report
30 % upon completion of data collection
30 % upon submission of first draft
20% upon completion and approval of the final report
Confidentiality and Copyright: The consultancy team will be required to sign a confidentiality clause whereby information accessed through the ADSI midline assignment can only be disclosed and shared with authorized members of the ADSI programme. GESCI will have ownership over any Intellectual Property developed for or derived from the engagement.
