Based on international guidelines [1] [2], this section outlines a comprehensive mechanism for monitoring and evaluation (M&E), which is also aligned with the EPIS framework discussed in Section 6. It utilizes a mixed-methods approach for data collection, verification, and analyses by integrating both qualitative (QL) and quantitative (QN) methods.
Monitoring is defined as a continuous and systematic process of data recording and targeted assessments throughout the life of the MHPSS program. The monitoring activities include tracking stakeholders’ engagement, capacity-building components, and feedback loops from both service providers and users. Similarly, evaluation is conceptualized as the systematic assessment of the MHPSS program’s achievements at various levels, such as training assessments and in-depth analyses of service provision. This structured M&E framework not only depicts the effectiveness and impact of the MHPSS model but also provides valuable insights for strategic decision-making and ensuring sustainability and scalability.
Table 7.1 presents a systematic M&E framework with distinct domains (and subdomains, where needed), corresponding indicators, and means for data collection and verification. To ensure accurate monitoring at all levels, the framework has also been organized into tiers (1–4), where needed. This tiered approach meets the specific needs of different groups of service providers (e.g., community mental health workers (Hamdard Force), clinical psychologists, primary care physicians, and specialists) by tailoring unique indicators to monitor and evaluate their performance.
TABLE 7.1: AN M&E FRAMEWORK | |||||
---|---|---|---|---|---|
Domains | (Sub-domains in italics) | Tiers | Indicators | Sources of data | Format |
1 EXPLORATION PHASE | 1.1 Situational analysis | All tiers | A comprehensive situation analysis report | Desk review / FGDs | Qualitative |
1.2 Financial support |
| Desk review | |||
1.3 Inter-sectoral collaboration |
| FGDs / KIIs | |||
1.4 Access to MHPSS digital solution |
| Portal data analysis | QN | ||
1.5 Mental health education campaign | A plan for a mental health education campaign | KIIs / Desk review | QL | ||
2 PREPARATION PHASE | 2.1 Capacity building: Recruitment | All tiers | Selection and registration of mental health workforce | MHPSS web portal | QN |
2.2.1 Trained workforce | All tiers |
| LMS - Courses completion report / portal registration data | Quantitative | |
2.2.2 Training outcomes — (1) | 1 | Training courses: Knowledge assessment | LMS - Courses assessment / feedback report | ||
2.2.2 Training outcomes — (2) | 2 | Competency in MyCare+ management | LMS - assessment report | ||
2.2.2 Training outcomes — (3) | 3 | Competency in knowledge, skills, attitude, and confidence | LMS - training assessment reports [3] | ||
2.2.2 Training outcomes — (4) | 4 | Competency in teaching methodology | LMS - training assessment report | ||
2.2.3 Training evaluation | All tiers | Feedback surveys by the trained mental health workforce | LMS - feedback report / FGDs | QL | |
2.3 Pilot MHPSS services | A pilot evaluation report that includes:
| FGDs / portal data | QL / QN | ||
2.4 Mental health campaign | A report of campaign evaluation | KIIs / portal data / campaign report | QL / QN | ||
3 IMPLEMENTATION PHASE | 3.1.1 Service users | All tiers |
| Portal / HF app / MyCare+ / mhGAP app data analysis | QL / QN |
3.1.2 Service outcome | All tiers |
| Portal data analysis | QN | |
3.1.3 Service evaluation | Feedback by service users | Portal data analysis / KIIs | QL | ||
3.2.1 Cases supervised | All tiers | Number of cases supervised (routine or urgent) | Portal data analysis | Quantitative | |
3.2.2 Supervision outcome | Number of cases where supervision is provided for: assessment, management, referral | Portal data analysis | Quantitative | ||
3.3 Program fidelity | Fidelity assessment report [4] | FGDs / KIIs | Qualitative | ||
4 SUSTAINABILITY PHASE | 4.1 Evidence gathering | All tiers | Review and evaluation of services in all phases | [5] | Qualitative |
4.2 Prepare a sustainability report | Perform data analysis and present insights from the implementation | A sustainability report on MHPSS service outcomes including:
| |||
4.3 Re-evaluate the services to scale up | Conduct regular meetings with stakeholders and service users to evaluate the services | A revised plan (developed through a collaborative approach) for scaling or replicating the MHPSS model at new sites |
WHO, 2023 https://applications.emro.who.int/docs/9789292740986-eng.pdf?ua=1
Adapted assessment tools: mhGAP-HIG knowledge questionnaire; ENACT; MICA-4; confidence assessment questionnaire
Fidelity assessment will be based on qualitative feedback from both service users and providers
Based on Program Sustainability Assessment Tool (PSAT) and Clinical Sustainability Assessment Tool (CSAT)
Mental Health Strategic Planning & Coordination Unit Health Section Ministry of Planning, Development & Special Initiatives