We connected through our mutual speaking coach, Heather Sager’s Speak Up to Level Up class. We’ve since followed each other’s careers and were excited to talk together. 

Listen to the Podcast 

Watch the Conversation

3 Tips for Visualizing Social Change Data 

In the podcast, we discussed three tips for coalitions, foundations, and nonprofits that are visualizing social change data. 

Involve Others in the Data Process Early and Often 

First, get the staff, partners, and community members involved in the data sense-making process early and often.  

My favorite technique for involving others is the data placemat process, which I learned from my previous supervisor, Veena Pankaj. 

It goes like this: 

1. First, we compile preliminary findings into a few handouts, or data placemats. Lots of ugly graphs. Lots of unformatted graphs. 

2. Second, we share those data placemats with stakeholders during a data interpretation meeting. We ask the attendees whether they were surprised by any graphs, what additional information they need, etc. We get them to talk about the graphs in their own words. 

3. Finally, we go back and write the final report (or design the final slideshow, or the final infographic) with the stakeholders’ interpretations of the graphs included. 

You can read more about data placemats in this article that Veena and I wrote for the American Evaluation Association. 

Share Aggregated or Disaggregated Data as Appropriate 

Next, figure out whether audiences need aggregated or disaggregated data. 

Let’s pretend that a nonprofit is running a GRE test prep program for high schoolers. As part of the program, the students take lots of practice tests to see when they’re ready to go take the actual GRE test.  

The students in the program need disaggregated data. They need to see their own individual data to determine how they’re doing and if they’re ready individually.  

A lot of times, the staff who are running the programs also care most about disaggregated data. That means they want to see data specific to each student so that they can individualize their instruction.  

There are also some aggregated summary statistics that might be helpful for those staff. For example, the staff might need to see averages.  

Current funders, prospective funders, and other collaborators will also benefit from aggregated data like averages. For example, they might want to see people in this year’s class compared to last year’s class, or this location’s class compared to a different location’s class. 

Problems can arise when there’s a mismatch.  

For example, if you only show the students the aggregated data, it feels too distant. Finding out the group’s average scores is helpful… but not as helpful as knowing your own scores. Or, if you only show the funders the disaggregated data, they’ll miss the big-picture patterns. 

Remember that Data Visualization Isn’t Supposed to be Hard 

You can use everyday software, like Excel. Just tweak the default settings to make the graphs easier to understand. 

You don’t have to learn coding or programming, unless you want to. 

You don’t need to go to school for graphic design, unless you want to. 

Stay in Touch with Ann Price 

LinkedIn: https://www.linkedin.com/in/awpriceces/ 

Twitter: https://twitter.com/annwprice  

Podcast: https://communitypossibilities.buzzsprout.com/ 

Website: https://www.communityevaluationsolutions.com/ 

The post 3 Tips for Visualizing Social Change Data appeared first on Depict Data Studio.

—

Thank you to Ann Emery, Depict Data Studio, and her Simple Spreadsheets class for inviting us to talk to them about the use of statistics in nonprofit program evaluation! If there’s one thing that we never want to miss, it’s an opportunity to tell people their job is easier than they think!

This is why we created ImpactStory Coaching – because it’s actually within reach of small- and medium-sized nonprofits to be clear, confident, and convincing when talking about their impact!

It was also just nice to talk to a group of smart, creative, data-minded professionals who also (yes – that’s right – also) feel overwhelmed by the prospect of having to use statistics.

We used to think our feelings of statistical overwhelm in the nonprofit space was because we were wimps (even though we have literally taken a combined SIX YEARS of graduate-level statistics*).

But then we realized that much of the time, statistics just don’t have much of a role in nonprofit work. And here’s why!

Why Nonprofits Shouldn’t Use Statistics

When working in the nonprofit world sample size is usually very small (i.e., the number of clients served in any given year is usually 40 to 400 people).

Even if you have more people than that (e.g., a school district’s worth of students), it’s still unlikely you need statistics, unless you are trying to answer a scientific-type question (and what scientific-type questions nonprofits with a lot of data might ask is for another blog post on another day).

Simply having a statistically significant group of survey respondents for such a small number of people means you have to get surveys from A LOT of people: 37 of 40 and 196 of 400! This is really hard to do (although we do have tricks for making sure you get surveys from almost 100% of your clients)!

Statistical significance is often mistaken to mean a big difference, but what it really means is a not random difference (e.g., if you looked at a different group of people, you’d find that difference again… it’s a reliable difference). When you have a small number of people, it also means you must have a GREAT BIG DIFFERENCE to get statistical significance.

The math just works out that if you look at 1 million people, just about any finding is statistically significant (e.g., a tiny difference in a big group is almost always not random), but when you’re looking at 100 people, you must have a really big difference to get statistical significance. In the science world, if you have a small group of people and do not find statistical significance, one thing you can do is test a much bigger group!

A Nonprofit’s Mission is to Serve as Many People as Possible to Address an Identified Need

In order to use statistics to identify the impact of a program, you usually need a comparison group (e.g., a random group of people who do not get the program) to which you compare your clients. Ideally, both groups are selected randomly: the people getting the program, and the people not getting the program. (We all know that’s not happening!)

Realistically, the comparison groups available to a nonprofit are either people the nonprofit randomly refused serve – or – a very nonrandom group of people who didn’t want the services the nonprofit was offering. Scientifically, neither of these are good options for comparison groups.

And we’ve never met a nonprofit so flush that they had money to track people they don’t serve. Even if a nonprofit had money to spare, spending this way would not be aligned with its mission to serve as many people as possible.

Nonprofits are Not Set Up to Follow People for a Long Time after Service Provision (e.g., 6 Months to 40 Years)

Most nonprofits provide a service for a specific amount of time, people graduate from that service, and then they go on with their lives (hopefully with more strategies to reach their goals). Staying in touch with people over time is very time consuming and very, very expensive – especially if you want to stay in touch with at least 80% of people (which is a minimum in the scientific world).

If you’re a smaller nonprofit that means you have to track 32 to 320 people over time (to follow up on those 40 to 400 clients you served). Frankly, this is both impossible and still too small to analyze with statistics (see point #1 about statistics with small groups of people).

Using scientific methods to test hypotheses (which are what statistics test) are what scientists do; delivering programming and tracking client progress is what nonprofit practitioners do. 

We have used this example before:  Scientists discover and test medicines to make sure they work. Doctors deliver what’s been shown to work and make sure the people they treat get better.

Two. Separate. Jobs. 

You – our nonprofit friends – are doctors.

Nonprofits Focus on Working with Individual People and Complexity Not Populations and Averages (Which is the Realm of Science)

It’s a cognitive error to assume that statistics (which typically focus on averages) applies to individuals.

For example, the average number of car crashes a person gets into in a lifetime is four (this is scientific knowledge derived from statistics) – but we all know people who get in many more crashes and people who never get in a crash (this is the reality of being an individual in the complexity of life).

And, if you – as an individual – get into four crashes that does not mean that you are now immune to getting in crashes (this is the cognitive error of applying statistics to individuals)!

So, drive safe and buckle up!

Also, if you are concerned about diversity and equity then you need to have more people from marginalized groups from whom you gather data so you can really hear what they have to say. Period.

You do not want to just have a representative number (e.g., a number equal to the proportion in the larger population) because their voices get washed out in the average.

When doing nonprofit work: Each. Individual. Voice. Matters.

What Can Nonprofits Do?

Nonprofits Should Think of Themselves as Conducting Qualitative Analysis with Numbers and Stories

Qualitative analysis basically means you are looking for patterns and changes in patterns in both your numbers data (what people report on surveys) and your stories data (what people tell you in words).

You’re examining how the data look – the shape, the themes, the patterns that emerge, and when the patterns change.

Your Data Team is the litmus test for determining which things are important and meaningful and which things are not.  Data Teams are for answering questions in real life; experimental design and statistics are for answering scientific questions. (Ask us more about Data Teams!  We love to talk about them!)

You HAVE TO GRAPH Your Data to See How it Looks

If you do use math at all it’s probably only to calculate the percent difference, the percent change, and maybe a risk ratio. This means you count how many people say something and how many people didn’t say that.

Graph all the answers in both groups. Then break the groups up differently to better understand the patterns of responses (e.g., males and females, comparisons based on race or income or zip code or classroom or age. . . you get the picture). If you don’t graph your data, you’re sunk. 

You simply have to graph it to see what it’s doing. Mostly bar charts (to compare groups) and line graphs (to look at stuff over time).

Here are some examples:

• Count how many people said “Strongly Agree” and “Agree” compared to “Disagree” and “Strongly Disagree.” What’s the percent difference between the two groups?
• Decide how different these counts are: meaningfully different or slightly different? The best way to make the most valid assessments of how meaningful the differences are is to use a Data Team. (We love a good Data Team!)
• Based on what you know about the people you serve, as well as changes in the community and at your organization, what do you think might be the reasons for those differences? (These are follow-up questions your Data Team can ask during their data review meetings.)
• Think about how you might divide the groups into different groups, or subgroups, to explore deeper questions (e.g., males and females, wealthy and financially struggling, graduated and not graduated, etc.). If you are looking at disparities, what’s the risk ratio of one group having a poor outcome compared to the other?
• If you have data over time (i.e., surveys from the same people at different times) – you may want to look at percent change happened over time?

What Software Should Nonprofits Use?

99% of the time. . . Excel.

If you have data with lots of complex relationships (e.g., data from parents and children, over time, in different programs) you probably want to be using a database like Apricot.

Then you can create and run reports that graph your data with the touch of a button!  And you can still can create downloads of the data behind those reports and create your own graphs in Excel.

If you have many hundreds or thousands of people you are serving, then it’s just easier to clean and sort that data in a statistical package like SAS, SPSS, or R. 

In these cases, we choose to hire someone who’s very good at these programs (like a graduate student taking a stats class) and pay them like $30-$40/hour to clean the data, maybe do some descriptive statistics and show us some averages. Then we have them download the clean dataset (a delimited CSV) and we pull that into Excel for graphing!

Tableau is great for being able to create dashboards you can manipulate and post on the web. But you actually have to know what graphs you want before creating them.

So, we’ll create the graphs we want (in Excel!), and then hire someone to transform the data and recreate those graphs in Tableau so nonprofit leadership can manipulate them or post them on the web.

We hope all this is some weight off your shoulders!  Sign up here to stay connected with us and follow us on all the social media! We have lots more to share!

Connect with MaryFrances Porter & Alison Nagel

Partnerships for Strategic Impact: https://impactstorycoaching.com/

Maryfrances Porter, Ph.D.  – LinkedIn: @maryfrances-porter-psi/

Alison Nagel, Ph.D. – LinkedIn: @alison-nagel-41493a125

The post Why Nonprofits Shouldn’t Use Statistics appeared first on Depict Data Studio.

But sometimes we don’t want to summarize, we want to highlight the variety in scores and remind readers that the chart is actually made up of individual people, not just the mean or median. Long live the messy data, the dispersion, the distribution, the spread!

I could tell you a few descriptive statistics: min = 26%, max = 100%, Q1 = 64%, Q3 = 83%, median = 74%, mean = 73%, standard deviation = 15%. Or, I could show you the spread in this unit-chart-turned-histogram.

Unit charts are not your new go-to chart. They do not replace bar charts. They are not appropriate for all datasets. They’re best for those few moments when you choose to emphasize individual units of data. A unit could be 1 person, or 10 people, or 1 school, and so on. Units can be represented in circles or squares or triangles. Units can be stacked on top of each other to form a histogram, or they can be plotted along a line.

The dataviz challenge: Re-create the chart in in Excel, R, or some other free software program. Then, tweet a screenshot to @annkemery.

Bonus: Make a unit chart for your own data. Or, do you emphasize individual differences with other chart types? Share your ideas with the community!

The prize for playing: A professional development opportunity and bragging rights. I’ll post the how-to guide in a couple weeks.

Want to learn more? I’m presenting about charting techniques at the American Evaluation Association’s annual conference on Thursday, October 17, 2013 at 11am in Washington, DC. Hope to see you there!

The post Dataviz Challenge #6: Can You Make a Unit Chart? appeared first on Depict Data Studio.

Could I ever create one in Excel?!

Yes! Luckily I’d learned about the Values in Reverse Order feature from Stephanie Evergreen. With Joe’s inspiration and Stephanie’s strategy, I started making these beauties for myself in Excel.

I wanted to share the chart secrets with all of you, so last month, I challenged readers to re-create a diverging stacked bar chart like this one:

It looks like I’m not the only one who loves diverging stacked bar charts. Congratulations to the 12 contestants! In order of submission, they are:

• David Napoli
• Anjie Raber
• David Bonachea
• Sheila Robinson
• Amanda Drescher
• Kristin Minichello
• @luno1972
• Hornyik Jozsef (in d3! with code!)
• Stephanie Evergreen
• Angelina Lopez
• Kevin Gilds
• Praveen Gowda

Most contestants seized the opportunity to use their own datasets and made adjustments as needed. For example, Sheila’s dataset fit a traditional stacked bar chart better than a diverging stacked bar chart, and Anjie needed to display cut-off scores.

So how do you make these diverging stacked bar charts, anyways?! There are at least two strategies: Either a) create two separate charts, a strategy demonstrated in previous posts like this one, or b) use floating bars, a strategy demonstrated in previous posts like this one. Stephanie Evergreen blogged about strategy B a few weeks ago and her explanation is pretty awesome, so I’m going to focus on strategy A today.

Here’s a slideshow about the two-charts-in-one strategy. Enjoy!

Bonus: Download the Materials

Share Your Feedback

Nearly all of the contestants requested friendly feedback on their graphs. In most cases, contestants were trying these charts for the first time and thinking about whether or not these charts could be adapted for their datasets. What do you think?

The post How to Make a Diverging Stacked Bar Chart in Excel appeared first on Depict Data Studio.

So when can you use a stacked bar chart?

• Stacked bar charts are for part-to-whole relationships. Use them when you want readers to see both a) one portion of the bar and b) compare that piece to the entire bar.
• Stacked bar charts can be used for tallies or percentages. A tally is the number of actual people, dollars, etc. For example, a nonprofit could display their funding sources in a stacked bar chart – $100K from a foundation, $200K from a government grant, and so on. The reader can see the size of each grant as well as how the grants stack up as a whole.
• Stacked bar charts can be used for nominal, ordinal, or diverging data. An example of nominal data is the racial/ethnic categories of your survey respondents. Ordinal data has a natural order – from best to worst, most to least, something to nothing – like my example. Diverging data is a subtype of ordinal data – when the categories are polar opposites and there’s a clear middle ground or neutral zone in between two ends.

And when can you use a diverging stacked bar chart? Diverging stacked bar charts are just for comparing several sets of ordinal data at once. They work best when you’ve got an even number of categories (like the 4 survey choices). Then, you can easily line up the midpoints along an invisible y-axis.

The dataviz challenge: Re-create the “after” version in Excel, R, or some other free software program. When you’re finished, email me or tweet a screenshot to @annkemery.

Bonus! 1) Adapt this chart for own data. Think outside the box! 2) There are at least two different ways to create diverging stacked bar charts in Excel. Can you find more than one solution? (And these charts are so awesome that you’ll even see one solution on Stephanie Evergreen’s blog next week!) 3) Don’t forget to use custom colors!

The prize for playing: Beer or coffee, my treat, the next time you’re in DC; a professional development opportunity; and bragging rights.

I’ll post the how-to guide in 3 weeks, on September 6, 2013. Happy charting!

The post Dataviz Challenge: Can You Make a Diverging Stacked Bar Chart? appeared first on Depict Data Studio.

Congratulations to the 6 contestants! Click on the contestant’s name to see their chart.

• James Coyle
• Sara Vaca
• Elisa Avila
• Angie Ficek
• Amanda Drescher
• Bryan Stephany

Most of them even applied this chart type to their own datasets. Sara, Elisa, and Angie ended up using different types of bar charts altogether. Check ’em out!

Now it’s time to post the how-to guide.

Step 1: Study the chart that you’re trying to reproduce in Excel.

We’re trying to re-create a small multiples bar chart like the one shown below. We’re comparing how many small, medium, and large nonprofits reported using each evaluation technique.

Small multiples bar chart

Step 2: And the secret to making a small multiples bar chart in Excel…

…is that we’re going to make six separate clustered bar charts. When we copy and paste the charts from Excel into PowerPoint or Word, they’ll look like a single cohesive chart.

Six small charts come together to look like one big chart.

Step 3: Type the data into Excel.

Here’s one of several ways to align your data.

Step 4: Create the first bar chart.

We’re going to create six bar charts. I started with Internal Tracking Forms.

A default bar chart showing the percentage of small, medium, and large nonprofits that use internal tracking forms.

You know the drill: Add data labels inside the end of your bars. Delete the legend, title, tick marks, grid lines, and horizontal axis label. (Later, we’ll insert new text boxes to label everything.) Adjust the axis so it goes from 0 to 100% (rather than 0 to 70%). Change the bar color. Use gray text to de-emphasize less important information like the axis labels. Reduce the gap width from 150% to something closer to 30% or 50%.

Hot tip: Keep the borders. We’ll delete the borders at the very end. The borders help us keep all the charts and text boxes aligned and even.

After a few clicks, we’ve improved the general look and feel of our bar chart.

Beginner Excel users: If you need extra instruction, check out how to make a basic bar chart and my Excel for Evaluation chart tutorials.

Step 5: Copy the first chart.

Rather than re-create the wheel when making the second, third, fourth, fifth, and sixth bar charts, let’s save some time by simply copying the first chart.

Just use good ol’ fashioned copying and pasting to create a second bar chart.

Step 6: Populate the second chart with the second chart’s data.

The first chart is for Internal Tracking Forms and the second chart can be for Interviews. Use the “select data” feature to put the Interview percentages into the chart.

To reduce cluttering, delete the second chart’s axis labels and use the business card trick to make sure each chart’s plot area is the same width and height.

Step 7: Make the third, fourth, fifth, and six bar charts.

Do some more copying and pasting to create the third, fourth, fifth, and sixth bar charts.

Step 8: Add text boxes to label everything and delete the borders.

Insert text boxes. Once everything is aligned, delete the borders.

Save time by copying and pasting text boxes, too. No need to create every single one from scratch!

Step 10: Paste the charts into PowerPoint or Word.

Since we’ve got 6 charts and 14 text boxes, copying and pasting into PowerPoint or Word can be a pain.

Hot tip: Carefully select all 6 charts and 14 text boxes. Right-click and “group” all the items together. Then, you can copy and paste into PowerPoint or Word with a single click!

Grouping the items together makes inserting your small multiples bar chart into Word or PowerPoint a breeze.

Bonus

Click below to download my Excel file.

The post How to Make a Small Multiples Bar Chart in Excel appeared first on Depict Data Studio.