6 Takeaways From Remote Data Collection in Rural DRC During COVID-19 Pandemic
Helping USAID ensure accountability and results through monitoring, evaluation, and learning (ME&L) presented a multitude of challenges after the COVID-19 pandemic required physical distancing, travel restrictions, and mandatory telework for USAID ME&L contractors like ME&A.
Nowhere was the challenge greater than ME&A’s evaluations under the Feed the Future Global Program Evaluation for Effectiveness and Learning (PEEL) project. PEEL frequently works in fragile and conflict-affected areas with security risks and stakeholder access limitations. PEEL’s previous experience with remote data collection in complex environments, however, turned out to be an advantage when COVID-19 forced PEEL to rethink its first almost all remote performance evaluation in the South Kivu Province of the Democratic Republic of Congo (DRC) with limited Internet in many places.
“The DRC is a complex environment requiring extensive planning and great care for data collection in ordinary circumstances let alone with COVID-19. The country has poor Internet connectivity, especially in rural areas, and many farmers, particularly women, don’t have mobile phones,” said Gary Woller, Ph.D., PEEL Chief of Party and ME&A Director of Monitoring and Evaluation. “What we learned is that if you are open to integrating new kinds of data collection methods, you don’t need to send expatriates for face-to-face data collection. You can do a reasonably high-quality evaluation with local consultants relying mostly on remote methods even in complex environments like the DRC.”
Below are six PEEL takeaways from the largely remote DRC evaluation along with two performance evaluations that had to be adapted for remote supervision of data collection mid-course in Mali due to political violence. Like the DRC, Mali also has extremely low Internet penetration by international standards.
- Be flexible and open to change with multiple contingency plans. Largely remote data collection requires rethinking the entire workplan and why and how you do everything. It also requires planning a series of reflect-and-adapt moments to adjust and switch to contingency plans depending on how host country public health and safety mandates evolve. PEEL’s development of multiple contingency plans enabled the evaluation teams to pivot quickly to changing guidance and at times unexpected initial results.
- Find workarounds to obtain the data you need. Since March 2020, PEEL has primarily relied on online surveys and computer-assisted telephone interviewing (CATI) to gather quantitative data to limit in-person data collection in accordance with strict social distancing and safety guidelines. PEEL also has relied on Google Meet, Zoom, Skype, WhatsApp, and telephone interviews for one-on-one key informant interviews for qualitative data since in some cases stakeholders cannot meet in groups for focus group discussions and in most cases facilitators cannot travel to remote areas due to COVID-19 restrictions.
- Remote data collection may affect the timeline. PEEL found that no or limited need to travel doesn’t necessarily save time. Under workplans for traditional in-person evaluations, teams are in the field for data collection for a set, limited amount of time. Without this tight window for in-person data collection, it can be harder to get key informants to make themselves available on the schedule you seek. Moreover, it takes longer to interview stakeholders individually versus in groups, and additional time is also needed for training and overcoming communications technology challenges, including having to reschedule interviews if technologies fail.
“For me the biggest challenge in doing remote data collection is focus group discussions where we need more thought and innovation,” Woller said. “The benefit of focus groups if you facilitate them well is you engage people in discussions to flush out issues in sometimes a more participatory and comprehensive way than one on one. Focus groups also save time and can be more cost effective with each focus group reaching six to 10 people at a time.”
- Identify stakeholder groups, such as farmer cooperatives or financial institutions, to overcome digital divides. In remote parts of DRC or Mali, it is critical to identify ways to include women, the less educated/lacking in digital literacy skills, and stakeholders who do not own phones in digital data collection. PEEL’s solution was to identify leaders of producer or other community-based organizations, such as farmer cooperatives or village savings and lending associations, and work through them to identify and arrange interviews for otherwise hard-to-reach populations.
- Invest heavily in upfront training for local consultants. Even before COVID-19, PEEL relied on local consultants who speak local languages and are often best equipped to interview stakeholders. Travel restrictions and social distancing requirements, however, necessitate transitioning a larger share of responsibility for in-country data collection to local consultants. Without an expatriate team leader on the ground managing the evaluation in-country, PEEL invested extensively in training the local consultants remotely on workplan strategies, data collection instruments and technologies, reflect-and-adapt moments, safety guidelines, and potential contingencies before sending them out to the field.
- Be open to new data sources in whatever ways are possible. PEEL found that because you can’t do what is traditionally done, you have to be agile and open and prepared to do something entirely different. Each solution depends on the situation and options and technology available in that locale. PEEL, for example, has conducted focus group discussions by speaker phone when the facilitator couldn’t travel to remote areas. PEEL is also exploring new methods of data collection for future use – such as application data, sensor data, biometrics, and data from drones – to provide more access to larger volumes of remotely collected data than ever before.
“I think at the end of the day these takeaways open the door to rethinking how we do data collection. We need to rethink this for cost-efficiency reasons, flexibility reasons, and potentially expanding our reach on the ground to avoid tarmac bias [focusing data collection in more easily accessible locations],” Woller said. “It makes all the sense in the world for USAID to continue pushing the envelope on remote data collection to see how it can complement and potentially replace some of the standard methodologies for data collection once the pandemic is over.”
PEEL supports USAID’s Feed the Future Initiative through population-based surveys, performance and impact evaluations, and other evaluative studies. These services allow USAID to better understand how Feed The Future activities are being implemented, how well they are performing, and how they are perceived by beneficiaries. USAID awarded the PEEL task order to ME&A in 2016 under the Policy, Planning and Learning-Learning, Evaluation and Research (PPL-LER) Indefinite Delivery Indefinite Quantity Contract providing evaluation, monitoring, and assessment services for USAID Missions and Offices worldwide.
- About the Author
- Latest Posts
Monica Jerbi, ME&A’s Communications Advisor, has more than 30 years of experience, much of it as a USAID contractor. Her expertise includes designing and implementing behavioral, attitudinal, and informational communication campaigns as well as researching, writing, editing, and designing print and digital communications and knowledge management products.