Public Policy Impact Assessment – Measuring the Impact of Employment Inclusion Initiatives Linked to the Active Solidarity Income (RSA)

The objective : Evaluating the impact of RSA-related employment programs in a French department

Significant public funding is allocated to supporting the social and professional integration of RSA recipients. In 2022, this amounted to €245.2 million in the department’s budget, despite a reduction in national funding for RSA.

In 2017, the department launched an initiative to assess the effectiveness of support programs for RSA beneficiaries. The aim was to quantify the impact of funded actions intended to promote social and professional inclusion.

To support this initiative, the department engaged our team to design an impact evaluation framework that would help guide employment support strategies across the territory. Our mission involved assessing the effectiveness of three innovative support programs compared to the current standard services offered to RSA recipients. These programs ranged in nature—from a job-matching platform with local employers, to an incubator-style mentorship program, and a more traditional coaching service.

Challenges: Designing a robust impact assessment framework for employment programs

The core challenge was to develop a reliable impact measurement protocol that accounted for on-the-ground realities. From scoping the evaluation to specifying the required data and implementing the methodology, several critical hurdles emerged:

• Selection bias: Avoiding any selection bias that could skew the evaluation results was paramount. We had to ensure that program participants were not fundamentally different from non-participants in the comparison group.
• Sample size and operational constraints: With a total of 450 RSA recipients spread across three programs, the relatively small sample sizes for each cohort posed issues in terms of statistical power and representativeness.
• Parallel experimental design: Isolating treatment groups for the evaluation of each individual program introduced complexity in both data management and comparative analysis.
• Contextual complexity: The evaluation needed to reflect the diversity of personal and socioeconomic situations, adding further nuance to the analysis.

Our solution: Optimizing the evaluation protocol and minimizing selection bias

The study was conducted across 15 municipalities with a total population of 278,000. Four distinct groups of RSA recipients were formed. One of these was a randomized control group receiving standard support services through the department’s existing inclusion programs.

• Scoping and framework definition: We began by clearly outlining the experiment’s objectives and the operational constraints defined by the departmental council. We established precise evaluation criteria, identified required datasets, detailed data collection methods, and highlighted the study’s limitations.
• Protocol optimization: Throughout the project, we refined the methodology, focusing on key elements such as distinguishing between supported and unsupported individuals, and assessing the relevance of segmentation strategies. To reduce selection bias, we applied a Wald estimator to compare program participants against the control group. This ensured an objective assessment, independent of the individual’s choice to participate or not.
• Ongoing monitoring and coordination: Data collection was conducted at regular intervals (6, 12, 18, and 24 months), in close coordination with the departmental council to support knowledge transfer. Biannual data analyses provided interim insights into the employment outcomes of participants.

The outcome: Actionable insights to optimize resource allocation across the department

After 18 months, aggregated results showed that one of the partner organizations outperformed the control group in terms of employment outcomes, albeit modestly. This incremental improvement nonetheless provided valuable insights to inform the department’s strategic decisions.

The departmental council has since fully integrated impact evaluation into its RSA governance and inclusion policy. This shift has enabled more efficient resource allocation toward high-performing solutions, while maintaining continuous oversight of their relevance and effectiveness for program beneficiaries.