Leave a Reply

Your email address will not be published.

How Much “Evidence” Does It Take to Convince an Evaluator?

Updated on: Apr 9th, 2012
Data Visualization
Collage of clipboard, clock, people, calendar and calculator.

I share an office with an evaluator from Brazil and an evaluator from France. I feel it is my duty to educate them on all things American, like Ramen noodles, Slurpees and American television.

I recently introduced my teammates to Saturday Night Live and a brilliant Scared Straight parody.

Being researchy people, you can imagine where our conversation went next… We discussed and debated the Scared Straight program, our “good intentions are not enough” mantra for social programs, and its evaluation history.

By now, about a million studies have shown that juvenile awareness programs like Scared Straight actually have negative results for youth. There was even a meta-analysis of 9 studies back in 2002 that found that “not only does it fail to deter crime, but it actually leads to more offending behavior.”

While I’m rarely convinced by a single peer-reviewed journal article, I’m usually convinced when several studies point to the same conclusion. And a meta-analysis is always enough to convince me that something is true.
One of my teammates, however, was not convinced. He’s a huge fan of meta-analysis and understands the strengths and weaknesses of these calculations.

So I’m wondering… When you’re a skilled social scientist, and when you have a first-hand understanding of all the things that can and do go wrong in research and evaluation studies, do you ever believe the result of a study?

What does it take to convince an evaluator that something is actually “true?”

More about Ann K. Emery
Ann K. Emery is a sought-after speaker who is determined to get your data out of spreadsheets and into stakeholders’ hands. Each year, she leads more than 100 workshops, webinars, and keynotes for thousands of people around the globe. Her design consultancy also overhauls graphs, publications, and slideshows with the goal of making technical information easier to understand for non-technical audiences.

Leave a Reply

Your email address will not be published.

You Might Like

Our complimentary mini course for beginners to dataviz. Takes 45 minutes to complete.

Enroll

Change Takes Time: How to Practice Patience in Report Redesign Processes

Guest author Abby Henderson started to have conversations with her colleagues about how they could change their reporting. She started by suggesting they add more data visualizations and fewer tables. When she met resistance to this idea, she started to produce two versions of the reports. One version included the tables they were accustomed to, and the second version included more elements of data visualization. Through providing both options, Abby was able to slowly garner traction and buy-in on including data visualizations.

More »

Inside our flagship dataviz course, you’ll learn software-agnostic skills that can (and should!) be applied to every software program. You’ll customize graphs for your audience, go beyond bar charts, and use accessible colors and text.

Enroll

Subscribe

Not another fluffy newsletter. Get actionable tips, videos and strategies from Ann in your inbox.