Conducting Evaluations During the COVID-19 Pandemic
The COVID-19 pandemic has resulted in unprecedented disruption in industries and sectors around the world, including research and evaluation. Accordingly, conducting evaluations of development projects presents a number of exceptional challenges, including:
- Data collection: Since international travel is not allowed, data is being collected remotely using digital technology, such as Zoom, WhatsApp, LoopUp, Viber, Skype, and others. However, arranging virtual meetings is more difficult as many stakeholders are not available (e.g., senior people in the government) or have connectivity issues (e.g., stakeholders in small cities or rural areas). Availability challenges might result in selection bias and affect the depth of the evaluation.
- Duration: Conducting online interviews throughout the day is difficult and sometimes impossible for many evaluators. This, combined with limited availability of stakeholders, has reduced the number of meetings conducted during the day and increased the duration of evaluation missions.
- Security: Many stakeholders are reluctant to share sensitive information over the Internet because of concerns regarding data security.
- Shift in needs: The disruptions caused by COVID-19 have led to shifts in government priorities, which in turn has obliged donors to adjust project objectives, activities, and interventions. Considering the knowledge gaps that exist around the pandemic, these changes might not be well documented and can affect the results of the evaluation.
Given the above challenges, conducting evaluations in the time of COVID-19 requires constant communication with the client, rethinking of the methodologies used previously, and flexibility in adapting work plans and evaluation designs. Our monitoring, evaluation, and learning (ME&L) team is taking all these steps. In addition to conducting interviews remotely, we have relied more on using local consultants to conduct face-to-face interviews, when possible, and bring their knowledge and experience to the evaluation.
Moreover, we have emphasized the importance of desk review and of data collection, not only from primary but also secondary sources, including GIS or social media data. In collaboration with the clients, we have adjusted evaluation design and work plans to allow more time for the evaluation, and revised evaluation questions to make sure that they bring value to the clients and respond to their current needs. We have worked with our evaluation teams to ensure that they are committed and able to work remotely.
Once the pandemic ends, we will need to make decisions about adjustments made out of necessity – which adjustments we can and should continue and which not. One thing that we have learned, though, is that hybrid methodologies (online/face-to-face) can be cost effective and a valuable part of the overall tool kit.
- About the Author
- Latest Posts
Mirela McDonald, ME&A’s Vice President for Evaluations and Proposal Development, has more than 25 years of experience in project management, planning, and administration. She has worked with a number of bilateral and multilateral donors, including USAID, the World Bank, the Millennium Challenge Corporation, and the European Bank for Reconstruction and Development among others.