Leave a Reply

Your email address will not be published. Required fields are marked *

On Methodological Demons, Farmers, and Bridgebuilders

Collage of a computer monitor, laptop, calendar, people and charts..

Last week I wrote about my inner evaluation demon (you can read the post here). Today’s post is a response by Andrew Blum. Andrew is the Director of Learning and Evaluation at the US Institute of Peace. He also tweets about peacebuilding and internal evaluation here.

When we met at the Eastern Evaluation Research Conference in April, we realized that we work close enough to each other in DC that we can meet for lunch and talk about… yes!… evaluation. I’m very fortunate to have met someone like Andrew because he’s such a great resource for strategies about internal evaluation and building a learning culture.

— Ann


In a recent blog post, Ann asked what our evaluation demons are.  The one that torments me the most, one that will be familiar to most anyone who has conducted research, is the one that continues to ask, “yes, but how do you really know that?”

You patiently explain your methodology to the demon, your verification procedures, your triangulation strategies, but the demon always has a doubt to express, a potential flaw, something else that could be done. When you finally say, yes, “but cost!” the demon will chuckle.

The reason this demon is so powerful is the difficulty we have in identifying a “good enough” methodology. Not only is this a big, hard question, but I’ve noticed recently it is hard question to even have a productive debate about.

The question seems to split people into two groups that I have begun to think of as farmers and bridgebuilders. To understand how these two groups think, imagine an evaluation with a methodology that is exactly 50% as rigorous as you would like. In this situation, the farmer sees half-a-crop, still something to eat.

The bridgebuilder sees half-a-bridge, useless and potentially dangerous. It’s not hard to see how these two groups might talk past each other when discussing methodology and methodological rigor.

Perhaps this is because I work in the field of peacebuilding, where quality data are hard to come by, but I am a proud farmer. I am constantly telling my colleagues, get me something, gather me some information, let’s do a bit better. Or to perhaps abuse the metaphor, let’s eat what we have, and work to plant a bit more of the field. But frankly, if I am talking to a committed bridgebuilder this kind of activity is hard to explain. So I’m interested in your thoughts on navigating this divide and ways to create more productive conversations on “good enough” methodologies.

— Andrew Blum (@alb202)

More about Andy Blum
At the Kroc Institute for Peace and Justice, Andrew Blum leads efforts to develop powerful new approaches to end cycles of violence and build peace. He works to fulfill the mission of the Kroc IPJ – to co-create learning with peacemakers and share that learning locally and globally. Andrew is an expert in developing applied learning systems and strengthening program design, monitoring, evaluation, and knowledge management systems within organizations focused on peacebuilding and human rights. With a Ph.D. from the University of Southern California, Andrew is an accomplished researcher in the field of peacebuilding and post-conflict reconstruction who has published and presented both academic and applied policy research to a wide range of audiences. He is also skilled instructor, speaker, facilitator, and educational program designer. No longer a transplanted Californian, Andrew has finally come home to the motherland after almost thirty years away.

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Courses

Most “professional” reports are too long, dense, and jargony. Transform your reports with these practical tips.


Data visualization best practices, practical how-tos, tutorials in multiple software platforms, and guest experts. Designed with busy number-crunchers in mind.

Learn More


#f-post-el-35{display:none !important}

#f-post-el-35{display:none !important}

Learn More