I've been enjoying the blog/podcast Statecraft, which interviews powerful professionals in government and how they approach important problems, because its exposing me to many unfamiliar perspectives. In the spirit of Statecraft, but only being able to leverage my limited personal network, I wanted to interview my friend Abisola, who's done Capacity Building work as part of International Development efforts in Nigeria and Tanzania.

The implicit understanding I’ve gotten from reading GiveWell reports on International Development is that more direct interventions, like Unconditional Cash Transfers and Anti-Malaria Nets, are favoured over higher level Capacity Building interventions, like those done Oxfam or Partners in Health. This is because:

• Is too hard to measure the effectiveness of
• Even if you could measure it, it's probably going to be less effective in the long run, because higher levels of abstraction means more steps, which means more opportunities problems

Abi’s experiences helped me understand the incentives and structures at play in International Development.

My questions are in bold, with Abi's replies in unformatted text.

First some background. Where did you go and who were the organizations involved? Why do these orgs exist and what motivates them at a conceptual level?

I went to Tanzania to work with an organization we’ll pseudonymously call SEED to improve processes in Monitoring & Evaluation (M&E) via a Canadian organization we’ll pseudonymously call HELPER.

At a practical level, HELPER recruits professionals to volunteer in Africa, the Caribbean, and Northern Canada. HELPER uses the Capacity Building framework of International Development (ID), wherein they help organizations acquire skills they will use independently after HELPER is done.

At a conceptual level, HELPER exists for Canadian professionals to “engage” in development work. The Canadians gain experience working in another environment, while the requesting organization gains skills essential to the growth of their organization.

SEED is an agricultural cooperative. They aim to do a lot, with activities such as: 

• Training marginalized Tanzanian youth about farming methods. 
• Facilitating international internships for Tanzanian students to the USA, Europe, and Israel.
• Supporting entrepreneurs with business coaching.
• Providing agriculture focused business consulting to other organisations. 
• Venue rental of their outdoor and indoor spaces.

SEED’s inception came about from a group of students looking to improve their prospects for work after graduation. They believe that Tanzania could be a profitable “breadbasket” for itself and surrounding countries, but Tanzania’s resources are underutilized and Tanzanians are underskilled.

I sought placement with HELPER, because:

1. Capacity Building aligned with my personal belief that meaningful change can happen when driven in a bottom-up manner by empowering people. 
2. I had successfully done Capacity Building before in Nigeria. There, I saw how individual members of an organisation gained and deployed skills via Capacity Building to support their own organization. 
3. I was really sick in Canada, doctors couldn't trace a physical cause for my illness. I felt strongly that I needed to be elsewhere.

What did you expect to get done while you were there?

After a few days of observing the organization and talking to staff about their experiences, I came up with a work plan approved by the Country staff at HELPER and the C-Suite (CEO and COO) of SEED:

1. Improve M&E Capacity Within SEED.
2. Develop Effective M&E Tools For Accurate Project Evaluation.
3. Improve Evidence-Based Decision-Making At SEED.

Each of these pieces had activities, inputs and outputs defined. For example, hiring a local M&E employee.

Jumping ahead, you’ve told me this didn’t happen. Why did you think this was possible to get done? Has anyone else ever done anything like this?

Cheeky. Why does anyone think anything is going to work out? 

Because they have evidence. Or they’re delusional.

Luckily, both apply to me in this case. I had a good experience in Nigeria. I thought the Tanzanian work plan was possible at SEED in particular because they said they wanted it; I was brought to SEED because they were looking for an M&E person. 

I know M&E is hard for people because - aside from the technical aspects - the nature of the function is to examine critically and tell you that things can be better. AKA things aren’t good enough. This can be hard to hear. But when they checked over my work plan they appeared enthusiastic and signaled interest like suggesting timelines, people to collaborate with, and they would smile with raised eyebrows (as if they were delighted). 

I may have also been slightly delusional. I don’t know what other people have done in other places, so I had no reference for the feasibility of my goals. As unrealistic as it may have been - my prerogative was to try my best. I was willing to set a lofty goal so that if I didn’t achieve it exactly, there would be levels beneath that would be satisfactory.   

How much of this were you able to get done?

Painfully, almost nothing. For each part of my work plan I got - what felt like to me - one step forward, and then nothing more.

1. Improve M&E Capacity Within SEED lead to:
   1. Half a training, that was reduced from a larger curriculum.
   2. An attempt at hiring someone that was abandoned after a candidate was found because of supposed lack of funds. I guess they just wanted to pacify me and to seem productive?
2. Develop Effective M&E Tools For Accurate Project Evaluation resulted in:
   1. Developing a Google Form to collect annual performance data. This information was used to develop the first annual report they’d have. 
   2. I was able to do an evaluation for a key project.  
3. Improve Evidence-Based Decision-Making At SEED.
   1. One meeting - where people spoke about their successes and needs for support.

Aside from the one training and one preliminary meeting, I literally could not get people to meet with me.

What. People just didn’t meet with you? Why didn’t people want to have meetings? If they didn’t have meetings, when/how were decisions getting made?

I could get the C-suite to meet with me alone to talk about their aspirations, but seemed unable to get them to commit to any implementation. There were also ad-hoc proposal-chasing C-suite meetings. But I couldn’t find leverage to force the meetings I wanted to happen. I couldn't communicate the incentive of “your programs could be better and you could get more funding long-term.”

All other SEED meetings were purely operational. It was a space for the C-suite to tell the staff what they would do. It was like a school - the teacher tells you what to do and you do it. The only questions people seemed to be able to ask were around clarification.

This isn’t surprising, strong hierarchies are pretty routine in Africa. But I did not expect the level of authoritarianism that was present in the organization. The C-suite does not delegate, because they want to keep control. This shows up plainly in how basic financial information is regarded, there’s a refusal to share basic parts of budgets, and specific information on the revenue of crops that go to market. To put it nicely, I believe this is an overcommitment to appear indispensable. Additionally, staff would occasionally come to me with reasonable concerns with work, like the fact they hadn’t been paid on-time. When I would suggest potential ways to address this, which would involve talking with management - there was an immediate shut down or hushing around this not being the environment this kind of recourse takes place in. All this to say, I couldn’t use an uprising to incentivize change either.

Could HELPER have helped more?

I’m not sure. I wonder a lot about what more was needed to make SEED functional or if being cut off and redirecting the resources elsewhere would be more appropriate. HELPER’s jurisdiction is mainly over us as advisors, so they can pull us out. It's trickier to understand what leverage HELPER has with their partner organizations - also the framing of these relationships is meant to be more kumbaya-like, like a cooperative partnership. If HELPER had to intervene more, what level of handholding would we end up doing and to what extent is that right? It would be off-brand for an organization like HELPER to intervene in ways that I think may be necessary.  

At a higher level, what incentives/structures/culture exist within SEED and HELPER that prevented this work from getting done?

This is hard to answer for SEED -  they do enough of the work, and they have a lot of work that looks great to typical funders that they can get by with how they operate. They pull enough strings to make things work - just so. This may be unpleasant, and I could be very wrong, but I think the ID industry is one that depends in parts on the bigotry of low expectations, usually under the guise of cultural sensitivity. This prevents the will to push for certain concrete outcomes that would be required in other sectors, it maintains consistent underperformance. A common example of this is in program design: There’s an acceptance from partners with unclear goals such as “improve economic outcomes for women agri-entrepreneurs” without high quality measurable metrics. Oftentimes the reporting success for a goal would be training attended rather than testing knowledge and application of training content - as well as a lack of longitudinal evaluations to track impacts over time. In contrast, if you look at a college in Toronto, they do conduct follow ups with their alumni - asking questions around income and career pathways to better factually ground their claims of success.

Is this problem local to Tanzania and SEED? Or did you also feel it replicated in other places like Nigeria?

I don’t recall this level of difficulty in Nigeria. The organization I worked with in Nigeria was more receptive, I had two staff members assigned to me who were keen to learn M&E techniques - they scheduled me before I could schedule them, they showed up, they asked questions. When it came time to create or administer surveys - they were there. Those staff weren’t solely motivated by the organization, they were also hungry for personal change. One of the two people assigned to me founded her own organisation in the country. There were also other staff who weren’t assigned to me who were curious and asked questions about the function. Maybe I got lucky in my first placement.

The problems I saw in Nigeria at that organization were more around timeliness and maintenance. An issue with timeliness that was fairly typical, a meeting would be said to start at 10AM but would actually start at 1PM. This resulted in days and tasks stretching longer than anticipated. Although as it was a pattern of behaviour - I could begin to account for this in my work style. When it comes to maintenance, an example I saw in Nigeria was around the website of the organization I worked with. They weren’t keeping up with updating information about their programming. This was an issue because potential beneficiaries or funders wouldn’t be aware of the current work they were doing.  

I was also a lot younger - so some of the weariness the Nigerian organization had of me and skills, were totally founded, but they still were much more willing to try different methods and create a framework for many of their programs.

I would like to think this recent experience in Tanzania was localized to SEED, but in chatting with other advisors in other parts of the country, as well as talking to people who worked at other organisations in the same city as me - there appeared to be similar kinds of barriers in people’s work. Or at least a similar pattern of not trusting the benefits of longer-term cooperation and instead opting to defect.

There was a neighbouring international organization, called GOODWORLD, who’s staff I’d befriended. To make a long and convoluted story - not that: there was a missing sum of money. The Tanzania side that had access to money denied there was even an issue to be addressed. Which required GOODWORLD to have several meetings in an attempt to recover those funds, over the span of at least 6 months. Ultimately, the Tanzanian side did not cooperate, so GOODWORLD was left to consider other options around working in that environment.

This feels like a repeating theme. It seems like everyone has no problem defecting from an agreement, because they don't believe the long term benefits of cooperation/effort are real. SEED doesn't care about measuring their programs, because they don't believe their funding could increase in the long-term. GOODWORLD's collaborators don't care about accurate accounting, because they didn't think GOODWORLD was going to be helpful in the long term.

I think you’re assuming money is their top priority. However, feeling important or the thrill of exercising power over someone can be valued higher than money. That being said, we can't really know what’s going on in their souls, so this is all speculation.

Given the mixed track record of Nigeria going well, but Tanzania being bad, is Capacity Building itself ill-advised?

I’m not sure, and I really hope not. So much ID work is weaved together with Capacity Building. There is an ongoing conversation and research in the matter. Factors such as context of interventions and buy-in from target populations make ascertaining effectiveness challenging. 

One article that I would point people toward is a piece called “The Science and Practice of Evaluation Capacity Building” by Steffen Bohni Nielsen, Leslie A. Fierro, Isabelle Bourgeois, and Sebastian Lemire. The article makes suggestions regarding how evaluation Capacity Building can be improved, and advocates for research to track the effectiveness of these methods.

After all this, how did your beliefs update?

Over the last few years, I’d been falling out of love with ID and non-profits. When it comes to top-down vs bottom-up change, I’m still trying to learn what’s more true.