1 Comment

  • Such a nice post! Տurely a need to-read and
    an eye-opener! Ιt definitely helped me thanks.

  • Leave a Reply

    Your email address will not be published. Required fields are marked *

    Formative and Summative Evaluations

    Updated on: Jun 26th, 2012
    Data Visualization
    , ,
    Collage of a bar chart, people, arrow, computer and calendar.

    Last month I described some of the differences between researchers and evaluators (you can read the post here). David Henderson noticed that my ideas could also explain the differences between formative and summative evaluators. I was intrigued by his comment, and I asked David to guest-post on this topic.

    I’m pleased to share this week’s post by David Henderson. David’s the founder of Idealistics and tweets about evaluation here. I hope you enjoy learning more about formative and summative evaluation from David.


    Head shot of David Henderson.

    David Henderson

    “Much has been written about the social sector’s love-hate relationship with evaluation. On the one hand, there are those who presume that evaluation will lead the social sector into a data driven age of enlightenment where only proven interventions are funded. On the other hand, there are those who fear the decisive, and potentially incorrect, conclusions of evaluators who some argue are given too much power to determine which organizations thrive and which ones die.

    The reality of evaluators’ roles in the social sector is far less extreme, and the general sectoral confusion over what evaluation is and is not is partly the result of our inability to effectively articulate the difference between formative evaluation and summative evaluation.

    Summative evaluation is where an evaluator (or team of evalutors) seeks to conclusively test the hypothesis that a given intervention has an expected impact on a target population. This type of evaluation has been popularized recently by the work of Esther Duflo and Innovations for Poverty Action through their use of randomized control trials in evaluating international aid efforts.

    Formative evaluation is where an evaluator works collaboratively with an organization to evaluate outcomes and try to use program data to improve the effectiveness of an organization’s interventions. This is the kind of evaluation performed by internal evaluators and by most evaluation consultants, including myself.

    The standard of proof used in formative evaluation is significantly lower than in summative evaluation. Summative evaluation is concerned with isolating causal effects, usually through an experimental design where a treatment group is compared to a control group to identify an average treatment effect on the treated (ATT).

    Some organizations’ evaluation anxiety stems from the inaccurate assumption that all evaluation is summative, and therefore potentially punitive. However, as Isaac Castillo, Senior Research Scientist at Child Trends recently said at a conference, evaluation “is an activity that produces useful content for everyone, and it should be undertaken for the purpose of program/service improvement.”

    Isaac is right that the real promise of evaluation is to help organizations improve program outcomes. While this does require a certain level of statistical sophistication, formative evaluation does not have the same confirmatory burdens of summative evaluation, nor does it incur the considerable costs that come with true experimental design.

    As evaluators, we would do well to educate those we work with about the difference between evaluative approaches. Doing so might help to mitigate the wrong-headed assumption that an evaluator’s role is to assign a letter grade to social interventions.”

    — David Henderson

    More about David Henderson
    David Henderson joined the Family Independence Initiative in 2014 after running a data analytics and technology company for over eight years that built outcomes management solutions for social sector organizations. David’s professional focus is on improving the way social sector organizations utilize information to make high impact decisions. After more than a decade working in and with human service organizations, David is especially interested in exploring the effectiveness of investing directly in low-income communities. David holds a Masters of Science in Public Policy and Management from Carnegie Mellon University and a BA in politics from Pomona College.

    1 Comment

  • Such a nice post! Տurely a need to-read and
    an eye-opener! Ιt definitely helped me thanks.

  • Leave a Reply

    Your email address will not be published. Required fields are marked *

    You Might Like

    Our complimentary mini course for beginners to dataviz. Takes 45 minutes to complete.


    Redesigning a Thesis Chapter

    Farihah Malik had the opportunity to work with a public health agency, which she was really excited about. Until she had to present the research to a group of policy makers… Condensing two full chapters—73 pages of Farihah’s thesis—into a short report for the policy making group seemed like an impossible task.

    More »

    Inside our flagship dataviz course, you’ll learn software-agnostic skills that can (and should!) be applied to every software program. You’ll customize graphs for your audience, go beyond bar charts, and use accessible colors and text.



    Not another fluffy newsletter. Get actionable tips, videos and strategies from Ann in your inbox.