This summer I had the honor of participating as a Monitoring, Evaluation, and Learning (MEL) Fellow at MIT D-Lab, where I carried out an evaluation of the 2019-2020 D-Lab Scale-Ups Fellowship. Even in the time of COVID-19, this experience was highly gratifying in terms of what I learned regarding design, evaluation, and social entrepreneurship.
The D-Lab Scale-Ups Fellowship, aside from helping improve business strategy and techniques, uses design to help social entrepreneurs work through their main challenges and explore solutions. The 2019-2020 cohort was made up of six entrepreneurs from East Africa working in a diverse range of sectors, from construction of low-cost housing to the creation of mobile cold storage units for crops, to phone applications to increase access to health insurance. My role was to work with the program manager, Jona Repishti, and the Monitoring, Evaluation, and Learning Manager at D-Lab, Laura Budzyna, to design and carry out an evaluation that could serve as guidance for possible improvements of the program. This year was the first time that fellowship had had an exclusive focus on local entrepreneurs, a regional cohort, the inclusion of a co-design summit focused on scaling challenges, and a year-long learning lab. The hope was that this evaluation would help determine the effect of these pivots.
The stage of the MEL Fellowship was set during its orientation, where I learned about D-Lab´s approach to development, participatory design, and program evaluation. From there, the process required exploration of the project and potential evaluation methods; the design of the three data collection instruments; data collection, analysis, and “sensemaking” of the final conclusions.
The thing that left the biggest impression on me was how D-Lab approaches and teaches the design process depending on the type of design and user, and then adapts it to all the work that they do in development. While I’ve worked in participatory design processes at the community level while living in Haiti from 2010 to 2015, and explored it further within my graduate program, D-Lab’s approach was novel to me.
Participatory design techniques permeated every aspect of D-Lab’s work. The use of a participatory lens to design with, by, and for, was noticeable in every process that we as MEL Fellows went through. This included the support that we received as MEL Fellows, with trainings highly dependent on participation, rather than lecture; the design process for the evaluation, which depended on creating an evaluation with and for the program manager and the social entrepreneurs (guided by the MEL Manager); and the Scale-Ups Fellowship itself, which supported entrepreneurs to explore problems and ideate solutions. Meanwhile, one of the main takeaways from the report was how entrepreneurs have learned how to use design to gain better insight from their clients and their teams for making their businesses grow. During the COVID-19 pandemic in particular, entrepreneurs stressed how the design process helped them to pivot in the face of the emergency, and the changes to the market that it brought.
The MEL Fellowship experience also was transformative in terms of learning to carry out evaluations and incorporating an evaluation lens into the way that I look at project design. The evaluation design process began by understanding the original project’s design (such as its theory of change), the program manager’s needs, and the many approaches and techniques that are provided by the project evaluation literature. I chose to use the Qualitative Impact Assessment Protocol (QUIP) as one of the main guides of this evaluation. This method helped to establish causal relationships between the Scale-Ups Fellowship and improvements in each Fellow’s enterprise through in-depth interviews: a perfect fit given the distance and the small sample size. One takeaway from this experience was how viewing a project design from the evaluation lens allows the designer to challenge their own vision of how the project should be by forcing them to review how each action that they take strengthens the objectives. I have worked in project design and management for many years, but this process allowed me to not just understand how to do evaluations, but also see project design in a new way.
A third lesson came from working with social entrepreneurs. Interviewing them was a reminder that entrepreneurship is not only about hard work, but about having a clear objective, and reimagining your actions to reach that objective in the face of a changing reality. Speaking to the Scale-Ups Fellows over the summer taught me about how they have addressed the global emergency, and unanimously, theirs is a role of constant analysis, creativity, and searching for solutions to problems.
The experience was deeply gratifying, and I very much look forward to using these lessons to my project design and evaluation work in the future.
About the author
Julio is a graduate student at the School of International and Public Affairs at Columbia University. He has worked in the design and management of development projects at the community and national level and hopes to incorporate project evaluation into his social development work in the future.
Jona Repishti, MIT D-Lab Social Entrepreneurship Manager
Laura Budzyna, Monitoring, Evaluation, and Learning Manager; Associate Director, Innovation Practice