Impact Evaluations

Measuring, Learning, Sharing

Impact evaluations provide evidence of concrete changes seen in project beneficiaries attributed to project activities. For the MIF, they offer an opportunity for continuous learning and improvement. For other private or public organizations implementing similar programs, these evaluations can provide valuable insights and evidence to help expand, enhance or redirect their programs.

Recent evaluations

August 2015

Planting the seeds: The impact of training on mango producers in Haiti

This paper evaluates the short-term impacts of a development project that aims to increase mango yields, sales of mango products, and the income of small mango farmers in rural Haiti. Various matching methods, in combination with difference-in-difference (DID), are used to deal with the potential selection bias associated with nonrandom treatment assignment. Our results show that in a 16-month period, the project increased the number of young Francique trees planted–a type that has greater market and export potential than traditional mango varieties–and encouraged the adoption of best practices. But the project has not yet led to a noticeable increase in total sales. The adoption of improved production practices is too recent to translate into significant changes in production and sales.

Download paper

July 2015

Community Savings and Credit Groups

This paper evaluates Community Savings and Credit Groups (GACC, for its Spanish acronym), groups that promote savings and allow members access to low-cost credit and “insurance" through a social fund fed regularly by their members. The GACC have generated a habit of saving among its members, persons in extreme poverty, increasing their volume of savings. The group members manage to accumulate savings larger than they would have done without a GACC. In addition, the groups have allowed them to generate greater social cohesion. Members of the groups are more involved in income generating community activities (businesses without family members) and an increase in the number of hours that households are willing to contribute to community activities. The GACC has proven to be a first step to financial inclusion for people in poverty, creating habits and discipline to manage savings and credit.

Download paper (in Spanish)

May 2015

From candles to light: The impact of rural electrification

This paper studies the impact of access to electricity via solar-powered home systems (SHSs) in rural communities in Peru. Applying propensity score matching at the community as well as at the household level, the authors find that households with SHSs spend less on traditional sources of energy - candles and batteries for flashlights - and that the subsequent savings are commensurate to the fee for SHS use. People in households with SHSs spend more time awake, and women in particular change patterns of time use: they spend more time taking care of children, cooking, doing laundry, and weaving for their families, and less time in productive activities outside their homes (farming). Children spend more time doing homework, which has translated into more years of schooling (among elementary school students) and higher rates of enrolment (in secondary school). Although women spend less time farming and men more time on home business activities in households with SHSs than in those without, these changes have had no evident impact on income or poverty.

Download paper
Download brief

May 2015

Creating opportunities for rural producers

This paper studies the impact of a pilot program that treated 57 small organizations of agricultural producers with high risk of getting involved in illegal drug production in Colombia. The program supported producers mainly by facilitating the commercialization of their new licit alternative sources of income. The paper combines propensity score matching, regression discontinuity, and Bayesian decision theory, with unique and rich panel data to assess the economic impact of the program. The results suggest that the program was successful in increasing total sales and improving the product’s quality for the treated producers. The intervention was more successful when combined with other programs that gave producers incentives to abandon illegal drug production definitely.

Download paper

March 2015

Can arts-based interventions enhance labor market outcomes among youth?

Using a randomized trial, the paper looks at employment and earnings of a youth training program in Brazil that uses arts and theater-based pedagogic tools. The evidence shows youth benefit both in the short- and medium-term. The impacts are economically large, compared to those typically found in the literature. There is no evidence of significant program impacts on other outcomes, including personality-related traits, suggesting that these traits may not be malleable for young adults in the short-run. The paper argues that the estimated labor market impacts are due to a combination of both skills formation and signaling of higher quality workers to employers.

Download paper

January 2015

Youth Training Programs Beyond Employment: Experimental Evidence from Argentina

The paper looks at the impact of entra21, a job training program for low income youth in Cordoba, Argentina. The program included life-skills and vocational training, as well as internships with private sector employers. The results indicate gains of about 8 percentage points in formal employment in the short term, although these effects tend to dissipate in the medium term. Contrary to what has been found for similar programs in the region, the effects of entra21 are substantially stronger for men, for whom the effects persist in the medium run. Program participants also exhibit earnings up to 50% higher than those in the control group, and an analysis of bounds indicates that these gains result from both higher employment levels and higher wages. With respect to results beyond employment, women selected for the program exhibit lower levels of welfare dependency – younger participants (aged 18 to 24) are less likely to receive child-related public cash transfers over the whole period of analysis; and program participants exhibit a higher probability of having requested consumer credit, and a higher probability of holding bank debts in good standing. These results indicate that training and internship programs directed at disadvantaged youth can provide other indirect benefits that are not usually accounted for in existing evaluations.

Download paper

December 2014

Soap Operas for Female Micro Entrepreneur Training

This paper analyzes the impact of the Strengthening Women Entrepreneurship Program in Peru (SWEP). SWEP trained female micro entrepreneurs in business management practices (such as accounting and marketing). The training, which was provided in 4- to 5-hour sessions, used soap operas and practical exercises specifically designed for the program. The results show that the program positively affected the adoption of business practices taught by the program. In particular, those who received the training were 4 to 6 percentage points more likely to assign themselves a fixed salary (rather than taking cash from their businesses based on personal needs) and 6 to 11 percentage points more likely to keep better records of potential business contacts. Some positive impacts were found on the adoption of bookkeeping practices (4 to 6 percentage points), although this result is not significant across all of the specifications. Although these changes in adoption rates were large compared with their baseline levels, they were rather small in absolute terms. Therefore, the study did not find any impact on average business performance, household expenditures, or women's empowerment in the household. Qualitative information suggests that micro entrepreneurs were satisfied with the training, but considered that many of the practices taught by the program were difficult to follow because of time constraints.

Download paper

July 2014

Communal Credit Committees: a convenient source of credit

Making credit available to rural communities in Costa Rica through Communal Credit Committees (CCCs) has proven to be an appropriate source of credit for these communities and appears to have displaced the use of other more expensive and less flexible sources, such as informal lenders and banks. CCCs are organizations managed by members of rural communities with logistical, technical, and administrative support from the Fundación Integral de Desarrollo Rural (FIDERPAC) and the Fundación Unión y Desarrollo de las Comunidades Campesinas (FUNDECOCA). This model, based on interpersonal relationships among community members, has helped to improve clients’ payment performance and to strengthen the social fabric in beneficiary communities.

Download brief

March 2014

Supply Chain Development in the Province of Cordoba

This paper estimates the direct and indirect effects of the Supply Chain Development Program implemented in the city of Cordoba, between 2003 and 2007 (with a focus on the cluster formed by firms working in information and communications technology). To estimate the indirect effects, the paper considers two complementary approaches: externalities associated with labor mobility and externalities associated with geographic proximity. The estimated direct and indirect effects were carried out using a random effects model in which selection bias is controlled by introducing lagged outcome variables. The results indicate that the program had positive and significant direct and indirect effects on sales, employment, wages and exports.

Download paper

November 2010

Improving the Competitiveness of Cattle Farmers through ICT Technologies (TRAZ.AR)

The paper studies the impact of the TRAZ.AR program conducted between 2004 and 2006 in Santa Fe, Argentina. The program promoted the adoption of radio frequency identification (RFID) technology which allows the identification and registration of cattle, so that farmers could add value to their product and eventually sell to niche high value markets as the European Union (traceability is a prerequisite to export there). The evaluation shows that program beneficiaries were able to obtain higher returns by accessing the European market. They experience a smaller reduction of cattle stock (Santa Fe was affected by the worst drought in the last 30 years during the implementation of the project), and an expansion of qualified workers. They were also more likely to adopt best practices like keeping records, satisfying standards for quality and food safety, using technology for traceability as well as adopting commercialization best practices.

Download paper

Other studies

Psychometric tests as a tool to improve screening and access to credit

This paper studies the use of psychometric tests, designed by the Entrepreneurial Finance Lab (EFL), as a tool to screen out high credit risk and potentially increase access to credit for small business owners in Peru. We use administrative data covering the period from June 2011 to April 2014 to compare usage of debt and repayment behavior across entrepreneurs who were offered a loan based on the traditional credit scoring method vs. the EFL tool. We find that the psychometric test can lower the risk of the loan portfolio when used as a secondary screening mechanism for already banked entrepreneurs, i.e. those with a credit history. For unbanked entrepreneurs, i.e. those without a credit history, using the EFL tool can increase access to credit without increasing portfolio risk.

Download paper
Download brief