“However beautiful the Strategy, you should occasionally look at the results." – Winston Churchill
Our MIF team – General Manager Nancy Lee, Claudia Gutierrez, Tetsuro Narita and me – recently spent two days with development experts and academics discussing and debating the importance of impact evaluation on Financial Inclusion programs. The retreat, co-sponsored by CGAP and DFID, was held in the shadow of Buckingham Palace at DFID’s offices, and included about 60 experts from development banks (ADB, CDC, DFID, MIF, EBRD, EIB, FMO, IFC...), foundations (Citi, Mastercard and Ford) and well-known scholars from IPA, Harvard and other institutions. The meeting was the brainchild of Alexia Latortue from CGAP, who saw a need to get a common understanding among funders and donors of the best evaluation methods, and also to spend time creating a “new impact narrative for Access to Finance.”
The diverse group of seasoned practitioners and researchers made for lively debate, with some arguing on behalf of rigorous experimental and quasi-experimental designs, others in favor of qualitative evaluation, and others primarily using market success and sustainability as a validation of their programs. Regardless of preferred methodology, all participants were heavily committed to the use of evaluation in choosing how to allocate resources, and thus the main debate was not whether to conduct research but how to use, interpret and disseminate it, what research gaps still exist, and how to coordinate efforts, not only among practitioners, but also with private financial institutions and other actors.
Stefan Dercon, DFID Chief Economist, opened the retreat by discussing how decision-making is informed by evidence. He parsed a new taxonomy to suggest four ways that most practitioners make arguments in favor of their programs:
- the compelling case – get a compelling story based on theory and experience, get examples to prove the point
- the hard core case – make all spending decisions based on thorough cost-benefit analysis and high quality methods to determine outcomes and impacts
- the religious case – evidence from impact evaluation is flawed, so let’s attack evidence; make policy by example
- the Keynesian Case – “when the facts change, I change my mind” – change course if one fails to find impact in various attempts.
I liked this taxonomy because it explains the current confused dialogue on the impact of financial inclusion, particularly with regard to microfinance, where extensive evaluation of many kinds has been conducted, including a growing body of quantitative impact evaluations. Nevertheless, microfinance experts seem to speak past each other as they debate from entirely different perspectives and are often trying to prove entirely different things. This is why a practitioner making a case about a growing, profitable and responsible microfinance institution may not make any headway in trying to argue with a “hard core” impact evaluator pointing to no significant income effects on beneficiaries of microcredit, and “religious case” based experts advocate on behalf of the market with fervor.
At MIF, our work with microfinance started by building the compelling case. Over the last 20 years, we provided financing of over US$ 320 million to over 220 financial intermediaries of different institutional types (such as specialized microfinance institutions, credit cooperatives, NGOs, banks) throughout Latin America and the Caribbean that reach a total of close to 5,000,000 microentrepreneurs and low income individuals with sustainable, high growth examples of financial inclusion in areas ranging from microcredit to remittances to savings and payments. In fact, MIF contributed to the creation of the microfinance sector as it exists today in LAC, which is rapidly consolidating into an industry.
To complement this track record, shouldn’t we also seek to know more about the effects of microfinance on poor and vulnerable populations and how it affects business creation and growth? Shouldn’t we be interested in testing which models work best with different clients? And, shouldn’t we be concerned about understanding potential effects on gender and family dynamics (either positive or negative)? This is why, more recently, MIF has also invested in a series of impact evaluations in a few of our most innovative projects. We are conducting a Randomized Control Trial (RCT) on a MIBANCO training program for low-income women that use videos and written materials to improve the performance of their microenterprises. In Colombia, we are assessing the impact of savings groups in the socio-economic conditions of poor women using quasi-experimental methods. Finally, in Costa Rica and Paraguay two impact evaluations on the effects of the expansion of rural credit are being conducted, testing village banking and rural credit committees lending methodologies, applying a mixed method approach where we are combining qualitative technics with quasi experimental (in Costa Rica) and experimental methods (in Paraguay).
We hope to learn from these evaluations and commit to making adjustments if suggested by the evidence. And, still being in Anglophile mode, I close with another Churchill quote on very consequential matters: “The truth is incontrovertible. Malice may attack it, ignorance may deride it, but in the end, there it is. “