What did we want to find out?
As a Monitoring, Evaluation and Learning (MEL) Fellow this summer, I was tasked with evaluating MIT D-Lab student projects that have partnered with TEWDI Uganda, a non-governmental organization located in Soroti, Uganda. D-Lab class collaborations with TEWDI have ranged from advancing clean cooking technologies to teaching innovation workshops to youth. TEWDI Uganda was chosen due to its longstanding collaboration with MIT D-Lab, which gave me a large sample size of projects to work with. TEWDI is also seen as a strong collaborator, which meant we could look at it from a strengths-based perspective, pull out what factors make the relationship generally successful, and identify the factors that differentiate projects with the same organization.
How did we measure this?
The evaluation looked at projects that partnered with TEWDI Uganda between the fall of 2013 and the fall of 2019. The evaluation examined the outcomes that projects achieved for both TEWDI and D-Lab students, as well as the factors that contributed to these outcomes. To answer these questions, I conducted in-depth interviews with students, D-Lab staff, and TEWDI staff, and also conducted a survey. In order to formulate survey and interview questions, I did a lot of research and interviews with key stakeholders. It was important to have a clear understanding of what success looks like to the various stakeholders. In both the survey and the interviews we asked about: (1) Project Outputs & Outcomes, (2) Project Fit, (3) Course Structure, (4) Partner Collaboration, (5) Management & Logistics, (6) Student Learning, (7) Process, (8) Continued Engagement, (9) Career & Education.
What did we find?
One of the key findings of the evaluation was the factors that have the largest impact on project success. We found that the following were the most important elements of a successful project:
1. The ability to conduct fieldwork: Students frequently described the fieldwork experience as transformative and emphasized how it helped to ground their project in reality. TEWDI staff also mentioned how helpful it is to be able to collaborate with students in person.
2. People: Mentor, Partner, Team
- Strong mentors with relevant skills and experience: According to students, having a mentor with a prior relationship to the partner and who was knowledgeable about the conditions on the ground proved indispensable.
- A strong community partner with relevant skills and experience: TEWDI was also mentioned frequently as a major reason for success. Students talked about how welcoming, engaging, and friendly TEWDI was, and that they enjoyed collaborating with TEWDI overall.
- Team: Although not rated as highly important as the mentor or the partner, a student’s team was frequently mentioned in interviews and the survey. Students felt that their team’s ability to work together, problem-solve, and work towards a common goal were very important.
3. Skills and knowledge learned in class: Students attributed much of their project’s success to the specific competencies gained through the class itself. This included skills and knowledge such as understanding user needs and gathering feedback in low-resource settings (Introduction to Energy), prototyping skills (Applications of Energy), hands-on skills required to implement selected development projects (D-Lab: Development), and design for remote manufacturing (Design for Scale).
4. Clear goals and expectations upfront: This theme was mentioned frequently in interviews with students. In several instances, students mentioned that the goals of the project were unclear and vague at the beginning, which led to a large amount of time upfront trying to define the problem rather than work toward a solution. Students also mentioned that clear goals allowed them to better understand their impact because they had clear milestones and ways to measure success.
To learn more about the evaluation and key findings, check out the D-Brief.
Final Reflections
Although I was originally supposed to travel to Uganda this summer, due to Covid-19 my fellowship had to be conducted from my small apartment in Austin, Texas. Not being in the field or in Cambridge with other MIT D-Lab staff had its challenges, but it also had its advantages. One bright spot was the close bond our team was able to cultivate. D-Lab’s MEL staff Laura Budzyna and Artie Maharaj and MEL Fellows Julio Martinez and Khanh Vu made the summer so enjoyable and I learned so much from each of them. Had we all been in the field this summer, I don’t think we would have been able to build such a strong team relationship. We were able to lean on each other during the challenging parts of the summer as well as celebrate our successes. I was also able to interact closely with D-Lab instructors who work with TEWDI Uganda. Libby Hsu, Eric Verploegen, and Dan Sweeney were immensely helpful through each phase of the summer. TEWDI’s Executive Director Betty Ikalany and her staff also deserve a special shoutout for being active and engaging partners and participants and for dealing with the challenges of living in different time zones. This summer was a group effort, and I am immensely grateful and proud.
Working on this evaluation with TEWDI and D-Lab reinforced my passion for monitoring, evaluation, and learning. Prior to graduate school, I worked in MEL for the federal government, so it was beneficial to see how non-governmental and smaller organizations utilize and learn from evaluation. I was extremely impressed with how well MEL has been integrated into the day to day life at D-Lab. The staff I worked with were extremely bought into the importance of MEL and wanted to engage and learn. It was truly inspiring and something that I will definitely try to emulate in my career.
Something else I will take away from the experience is the importance of mutual collaboration and respect in the field of development. In a time when the aid world is reckoning with its legacy of colonization and racism, this is even more relevant. The respectful and collaborative relationship between TEWDI and MIT D-Lab staff is something that others should strive for. Both sides see the collaboration as a mutually beneficial partnership. This is something we can all learn a lot from.
About the Author
Samantha Jorgensen is a second-year graduate student at the LBJ School of Public Affairs at the University of Texas-Austin. She is pursuing her MA in Global Policy Studies with a specialization in Monitoring, Evaluation, and Learning (MEL) for International Development. Her research interests include MEL and the use of data for development, especially as it relates to gender and the SDGs.
More information
Contact
Libby Hsu, MIT D-Lab Associate Dirctor for Academics