Under the guidance of the inestimable Ann Emery’s “Report Redesign” course, I had a go at making the contents page of a previous evaluation report meaningful.

Here are the steps I followed.

For anonymity, I have renamed the evaluand as (gender org) and the funder as (funder).

The Original

Here’s what the initial Table of Contents looked like:

Draft 1

I cut my multi-page contents page from three heading levels to only Heading 1s.

Draft 2

I changed some headings to be more meaningful.

For example,

• from ‘Summary Report’ to ‘Highlights’
• from ‘Methodology’ to ‘The Outcome Harvesting Approach’; and
• from ‘Contributions that influenced the outcomes’ to ‘The role of (gender org), gender specialists and (funder)’.

Draft 3

I grouped the headings, named the groups, and set it up in landscape.

I also enlarged ‘Contents,’ in response to one of Ann’s ongoing exhortations to “double the size of the headings from what you currently use.”

I moved from Word into PowerPoint.

Draft 4

I created a section divider in my brand colours and added icons.

Ann proposes use of such dividers for short reports, with a different colour for each section – in long reports you’d use a whole page for each section.

I used her ’20-minute page cover’ method by layering a cylinder shape in my brand colour, somewhat transparent, over a Word Cloud, and ‘Contents’ on top.

I added an icon to each section and recoloured the sections to colours I will use for the divider page and headings of each section, taken from my brand colours.

Draft 5

I re-coloured the section divider and put in page numbers.

As ‘gender’ in the Word Cloud overwhelmed the heading, I used the ‘textures’ option in ‘format colours’ to create a grey textured layer over the Word Cloud, and changed the colour of ‘Contents.’

To me this shift from a pro-forma contents page to this version invites the reader to find what they’re looking for in the report.

Going forward, I would plan this out before writing the report, to help organise my own thinking about the contents and how to communicate it.

Thanks to Ann.

The post From Formulaic to Meaningful: Constructing a Useful “Table of Contents” Page for an Evaluation Report appeared first on Depict Data Studio.

My first chance came in the form of a front-end evaluation project for a children’s museum planning a new exhibition on dinosaurs.

Measuring What Kids Already Know about Dinosaurs

The museum wanted to understand what children and families already knew about dinosaurs – including whether they knew what other types of animals and plants existed at the same time.

I designed a fun card-sort activity, where parent-child pairs were asked to work together to sort 19 cards with images of different plants and animals into two piles:

• one pile for those they thought lived at the same time as dinosaurs, and
• one pile for those they thought didn’t live with dinosaurs.

Here’s a sample of a few of the cards we gave to families:

Draft 1

For my first stab at a graph showing the results, I applied several of the best practices I learned about in Great Graphs:

• I sorted my data from largest to smallest.
• I applied color meaningfully – using the client’s brand orange to show the animals that did exist at the time of dinosaurs and gray to show those that didn’t.
• I eliminated the unnecessary visual clutter from the Excel default graph and made some simple modifications (for example, increasing the width of the bars and the text font size).
• I even added annotations highlighting interesting findings.

Here’s what my first version looked like:

Draft 2

I thought I was off to a pretty good start, but I wasn’t sure if my graph was clearly explaining that some of the answers were correct and some were incorrect, so I decided to bring my graph to Office Hours with Ann to see what else I could do.

Ann offered me three simple ideas that took this graph from good to great.

1. Group the bars to better show which responses were correct or incorrect.

Rather than order all the bars from largest to smallest, Ann suggested that I group all the correct answers together (ordered from largest to smallest) and similarly group all the incorrect answers together.

2. Add space between the groups to create a visual distinction.

Although the same effect could be achieved by creating two separate graphs, Ann showed me how to add a gap between two sets of bars in a single graph by simply inserting one (or more) blank rows in the source table. (Note from Ann: Learn more about adding blank rows in this tutorial, and view another example of intentional gaps here.)

To make the difference between the two groups even more obvious, we also added subtitles to indicate correct and incorrect responses.

3. Add icons for visual interest and whimsy.

This graph is for a children’s museum project about dinosaurs. This is the type of graph that is just calling for a touch a playfulness.

We found an adorable dinosaur icon in the free icons that are included with all Microsoft Office products.

We added an orange dinosaur icon to highlight the correct answers and a grey one with a slash through it to highlight the incorrect answers.

Here’s the final version of the graph that I included in my report:

A big improvement made in three simple steps and less than 30 minutes.

There’s a reason the course is called Great Graphs.

Connect with Maia Werner-Avidon

On LinkedIn: https://www.linkedin.com/in/maia-werner-avidon/

Learn more about Maia’s work at www.mwainsights.com.

The post 3 Simple Steps that Took My Graph from Good to Great appeared first on Depict Data Studio.

I did.

I taught myself how to write in third-person language.

I called myself “The researcher…” instead of plain ol’ “I…”

I replaced my everyday words with “smart” synonyms. I literally paged through my GRE study guide. I tried to use as many GRE vocab words as possible.

Then, I started working in the real world.

My bosses rolled their eyes.

Another one from an academic background, they sighed. We’ll have to re-train her from scratch.

I panicked. But if I wasn’t supposed to sound like a textbook, what was I supposed to sound like???

A human!

It took me years to grasp that simple concept. I’m a person. It’s okay to sound like a person.

Nowadays, I make a living by teaching humans how to sound like humans again.

Before/After Makeovers for Common Methodology Sentences

Here are some before/after examples in case you’re still on the textbook-back-to-human journey like I was.

Please please please use these transformations in your technical reports.

I’m not so worried about peer-reviewed journal articles — that’s another battle for another day. Today, I’m focusing on your non-journal writing scenarios.

Who Designed the Survey?

• Before: A survey instrument was designed by the ABC Research Company working under the supervision of the DEF Foundation staff, and key department heads of GHI Agencies.
• After: The ABC Research Company, DEF Foundation, and GHI Agency teamed up to collect data.

Who Responded to the Survey?

• Before: A series of survey instruments were developed to administer among students in the ABC programs.
• After: We designed surveys to collect information from students in the ABC programs.

How Many Responses?

• Before: A total of 14 programs participated in the survey.
• After: Fourteen programs participated in the survey. (Remove redundancies like “a total of.”)

Or…

• Before: A total of 144 programs participated in the survey.
• After: We collected surveys from 144 programs. (Because writing out numbers at the beginning of a sentence is the worst.)

When Did You Collect Data?

• Before: Initial surveys were launched on March 7, 2018 with fieldwork continuing to accommodate the schedules of participating institutions. Data collection was cut off on April 25 to begin data processing. A total of 789 surveys were attempted, with a total of 654 surveys completed sufficiently to include in the final tabulated results. A total of 123 individuals entered their contact information for a drawing.
• After: We collected surveys in Spring 2018. We tried to collect data from 789 people, and 654 people participated, for a response rate of 83 percent—one of the highest response rates we’ve ever had on a survey.

Referencing the Just-in-Case Tables

• Before: We are providing detailed data tables with this report that shows the responses by institution.
• After: Want to view responses by institution? View the appendix on page 31.

Demographics on Respondents

• Before: Overall, undergraduate students comprise 65% of total responses and graduate students comprise the remaining 35%.
• After: Two out of three responses (65%) were from undergraduate students. The rest were from graduate students. (Getting rid of the word “comprise.)

Describing the Survey’s Topics

• Before: One way the important resources and individuals specifically helped at least two-thirds of students were giving them a good sense for the kinds of careers they could pursue with a degree.
• After: We asked students which resources were most useful. Two out of three students said that others had given them a sense of career options that they could pursue with their degree.

Summarizing the Survey’s Findings

• Before: The largest mean share of the total cost are paid for by the student or their family, who account for 50% of the total cost. Student loans are used to cover a mean of 20% of the total cost, and scholarships or other financial aid pay for 30%.
• After: For the typical student, 50% of their costs are covered by the student and their family, 30% are covered by scholarships or financial aid, and 20% are covered by student loans. (Getting rid of awkward language like “mean share” and “account for.”)

Objectively Scoring the Before/After Translations

In my gut, I know the translations are easier to read.

Let’s objectively test them.

Before: 12.1 Grade Level

The human-trying-to-sound-like-a-textbook wrote at a 12.1 grade level.

Okay, that’s not the worst I’ve seen.

The highest I’ve seen is a 36 (from a team of Ph.D. psychologists).

Can you beat a 36??? Let me know if you find any contenders. I’d love to (try to) read it.

(This screenshot is from Readable.com, which used to be a free reading level checker. It looks like they require payments nowadays, but there are plenty of free- and low-cost tools. Like good ol’ Microsoft Word! Comment below if you’ve got a favorite.)

A­fter: 9.3 Grade Level

I personally aim for grade level 6 to 8—throughout my blog posts, books, and even contracts.

I didn’t quite reach my goal. But a 9.3 isn’t horrible, either. The Readable site gives this an “A!”

Higher is not better. Lower is better.

You are a human who’s writing for humans. You are not a textbook. You are not a textbook. You are not a textbook. You are not a textbook.

How to Lower the Reading Grade Level

Try one (or more!) of these techniques:

• Shorten the sentences. An easy fix is to look at your longest sentences. Replace your commas with periods (i.e., break one long sentence into two shorter sentences).
• Shorten the paragraphs. Press the “enter” key lots and lots and lots.
• Use first-person language. Adjust the sentence structure. Change “A survey was administered…” to “The agency administered a survey” or “We administered a survey.”
• Find synonyms. This is the hardest one for me. What’s an accurate, understandable translation of calculations like standard deviation or confidence interval??? I used to pack those terms into the report’s body and hope for the best. What happened? Lots of Dusty Shelf Reports! Nowadays, I follow the 30-3-1 Approach to Reporting. I keep the methods section in the report’s body as short as possible, and I tell readers to check the appendix for more info. I don’t care if the appendix is packed with jargon. Only the technical readers are going to look there anyway, and they’ll understand the jargon.

Your Turn

Upload one of your own paragraphs into your favorite reading level checker.

How did you score?

And more importantly, how can you adjust the language to lower the reading level??

The post How to Write about Research Methods Like a Human (and Not a Textbook) appeared first on Depict Data Studio.

We’ve all been on the receiving end of a long, boring, text-heavy report. It can be challenging to sit down and read, rarely translates insights into action, and is (for lack of a better word) boring. Well… I used to be an author of reports like that.

The Everything Report

At the start of my career, I thought the most important thing a report could convey was… well… everything. My thinking went something like this: “All of the background information, data, methods, and recommendations needed to be extensively covered.  The more technical and academic the language, the better. If I can impress people with my language and expertise, they’ll be more likely to follow the recommendations I suggest.” However, that line of thinking functions under the assumption that people will read to the end of the report to get to those recommendations.

What I came to realize early on was that people, in fact, did not want to read through a long and technical report to get to the recommendations buried at the end. On top of that, spending so much time writing extensive reports was taking time away from strategic and creative thinking about what the data means. I had so little time left for that type of thinking that my recommendations were often vague, unhelpful, or lacking in creative thinking.

Introduction to Data Visualization

Cut to a session I attended hosted by the Arizona Evaluation Network and led by Deven Wisner and Nicole Huggett on data visualization. My brain lit up with curiosity and joy at the idea that data could be conveyed simply, succinctly, and visually. That session got me started on a new journey and led to me asking new questions.

Instead of asking, “How much information can I fit in this report? How technical can I make my language?” I was now asking questions like, “What is the purpose of this report? How do I hope these findings are used? How can I communicate that clearly, succinctly, and visually?”

The next thing I knew, I was enrolled in Report Redesign through Depict Data Studio and spending much of my free time thinking about data communication. More broadly, I was (and continue to be) interested in how we bridge the gap between technical expertise and lived experience, and how we communicate across that gap regardless of building a bridge.

Starting a New Conversation

I started to have conversations with my colleagues about how we could change our reporting. I started by suggesting we add more data visualizations and fewer tables.

When I met resistance to this idea, I started to produce two versions of our reports. One version included the tables we were accustomed to, and the second version included more elements of data visualization. Through providing both options, I was able to slowly garner traction and buy-in on including data visualizations.

The next proposal I made was to include infographics, one-pagers, or shorter summaries with our reports for individuals who may only be interested in the data from a high level. Again, I took on responsibility to demonstrate what I was envisioning and how I thought it could work. This meant taking on extra workload to create the products our leadership was accustomed to, as well as products I wanted us to explore. I brought up data visualization in meetings, attended webinars, and tried to increase the data literacy on my own team.

I was hopeful but hesitant during this timeframe. After all, change takes time, and change takes even more time when you’re suddenly grappling with a pandemic and an upending of our work lives as we previously knew them to be. In this new normal, I was suddenly surrounded by data visualization. The conversation shifted in the world around us, and therefore the conversation shifted in our office as well. There was no longer a question about the utility of data visualization in reports, as we were seeing firsthand how impactful a good visualization can be to convey a message.

The conversations about change continued, as I slowly began to notice how things were shifting in our offices. Suddenly, others on my team were hoping to collaborate on infographics. I was being asked to take on more projects, but each new project was a step in the right direction. We were no longer producing reports without visualizations, and we were starting to explore including one-pagers with our annual reports as a standard across projects.

Around this time, I ended up in a new position at a new company that uses data visualization as a bedrock of our work and didn’t get to see firsthand how the process in my previous office continued to change. However, I found out from colleagues that they now include an infographic or one-pager as a standard with all annual reports, are exploring ways to add more visual elements to quarterly reports, and are continuing to increase the internal skill around data visualization.

Lessons Learned

Here are three.

Patience

The most important lesson I learned in this process was one of patience. From the moment I made my first data viz, I wanted to change everything immediately. That was neither realistic nor feasible, and looking back I wish I had aimed for a slow burn to change our processes.

Self-Reliance

By deciding this was the hill I wanted to climb, I also had to be prepared to climb alone. Sometimes this included late nights in the office making two versions of the same report. Sometimes (often) this included frustration with my own lack of knowledge about data viz best practices. I had to trust that this process was worth the extra work. However, looking back, I wish I had had better boundaries around taking on extra workload to accomplish this change. Finding other champions on my team earlier on may have helped with that as well, as data visualizations are nearly always improved by collaborative brainstorming.

Mistakes

Here’s the thing: anytime you learn something new, you’ll make mistakes. I made plenty, both in my actual visualizations and in my attempts to create internal change in my organization. The goal is not to make zero mistakes, but to use your mistakes as a jumping off point for new learnings. Accepting that mistakes are part of any change process can provide grace from the start about the challenges you’ll encounter.

The post Change Takes Time: How to Practice Patience in Report Redesign Processes appeared first on Depict Data Studio.

She’s sharing examples of slider plots and step-by-step instructions for making them in Excel. Thanks for sharing, Lauren! –Ann

—

Hey everyone! Lauren Fox here from the Research & Evaluation Division (RED) of The University of Arkansas for Medical Sciences. Our group focuses on translating research into practice in fields like early childhood education, child nutrition, abuse prevention, and HIV education.

Much of my job involves working with faculty members and project leads to develop evaluation questions that lead to actionable data.

One of the biggest puzzles we face is how to translate those results (visually and verbally) so that everyone from expert audiences to laypeople can understand our findings and benefit from them.

The data viz world is full of options for visualizing basic data such as change over time, pre/post differences, and percentages/frequencies for a single point in time.

Sometimes however, your data (or your audience) demands a little more.

Case-in-point: When displaying margin of error is important.

The Challenge: Displaying Margin of Error Data

Back before the pandemic, one of our faculty asked for some help in visualizing her data for a conference on childhood nutrition.

I dressed it up the best I could, but it still fell far short of the best practices for data visualization. I figured there had to be a better way to display data with margins of error (a.k.a., the “95% confidence interval”), and set out to find it.

Spoiler alert: I didn’t find anything. So, a little bit at a time, over the course of several months, I built it myself. I call them “slider plots.”

Full disclosure: I didn’t know until after I developed these, but Stephanie Evergreen posted a rough sketch version of this idea using auto-calculated standard error bars back in 2017. Her title used “confidence intervals,” instead of “margin of error” so I missed it in my initial search.

While that means the basic idea behind my “slider plots” isn’t completely new, I’m still excited to build on her work and share this as a small step forward in chart design!

The Old Way

The “old way” involves using column charts with error bars.

The New Way: Slider Plots

Slider plots can be vertical or horizontal. Here’s an example of a vertical slider plot that shows policy ratings from four different neighborhoods.

Here’s a second example of a vertical slider plot that shows teacher ratings in four different schools.

Here’s what a horizontal slider plot of those policy ratings would look like.

And finally, here’s the horizontal version of the teacher ratings.

Download the Excel File with Step-by-Step Instructions

The process to create slider plots follows many of the same steps as creating dot plots and adds a few more to create and customize your margin of error bars.

Start to finish (from creating a data table, to building your dot plot, through creating and customizing your error bars), there are 15 steps, plus a few optional sub-steps.

I’d like to list them all here, but this post would definitely get a TLDR citation from the blog police (Too Long, Didn’t Read).

While many of the steps are similar, vertical slider plots are easier to build so I recommend you start with those first.

The horizontal version may be harder to build, but it has the same readability advantages of classic dot plot we all know and love.

As a bonus, you can download a free Excel file with step-by-step instructions and screenshots, as well as an end-product template you can use to make the process much faster.

Winning Hearts & Minds with Slider Plots

While slider plots do take some time to set up, the payoff for your effort is helping to expand the reach of data viz.

Many in the evaluation community have begun to adopt better data visualization practices to help communicate their work over the last few years, but there are still many spaces (workplaces, conferences, etc.) where we find resistance.

Some of that is fear of judgement; that we won’t be taken seriously as scientists by our colleagues if we present data in non-traditional ways.

If there’s one thing I’ve learned from being an evaluator in the early education space, it’s that if you want to change people’s minds (and then their behavior), you have to meet them where they are.

I’m under no illusions this chart type will suddenly convert all the data viz detractors or revolutionize the field.

However, the changes are small enough and familiar enough that they might be a bridge to expert audiences; a way they can slowly grow more comfortable with the idea that presenting data differently doesn’t make you less scientific.

Know Your Audience

As cool as it is to do something new, it’s important that I leave you with this reminder:

Most of the time, margins of error will not be important enough to visualize unless you’re dealing with an expert audience.

It will most likely confuse or distract less-advanced audiences from the point you’re trying to make.

However, you can try adding a little more explanation in the graph subtitle to bridge the gap (see my slider plots above for examples) if it’s critical for your lay audience to see the margins of error as well.

Connect with Lauren

LinkedIn: https://www.linkedin.com/in/lbfox/

The post How to Visualize Margin of Error Data in Excel with “Slider Plots” appeared first on Depict Data Studio.

Here’s my definition of data storytelling, in case you missed the previous blog post. 

Great!  

Let’s remove the guesswork from our graphs. 

The next step is to figure out which message we’ll highlight. 

We can’t visualization everything—that dilutes the power of our graph. 

What Makes a Useful Data Story? 5 Questions to Ask

Here are five thought-starter questions to help you uncover useful nuggets in your data.  

1. What’s Going Well? What’s Not Going Well? 

2. Did We Reach Our Goals? Why or Why Not? 

3. What’s Surprising? What Unfolded as Expected? 

4. Which Information Needs to be Shared with Others? Who Needs to See This Information? 

5. What Increased Over Time? Decreased? Stayed the Same? 

If you analyze data for a living, then I bet you’re already asking yourself these questions. You were probably trained to approach data this way in grad school. Or, it’s become intuitive after years of on-the-job practice. 

Dabblers in data, this one’s for you.  

Newcomers to data, this one’s for you.  

What’s Going Well? And What’s Not?

Everyone loves a success story.  

Look through your dataset.  

Find the good news and highlight that finding with dark colors and takeaway text.  

I often start with positive findings so that my audience can celebrate a small victory right away. 

But, facilitating an honest conversation with data visualization requires balance.  

After presenting positive news, I present the less-than-positive news.  

For example, the next graph in my report might use a darker color to draw attention to something that isn’t going well. 

Did We Reach Our Goals? Why or Why Not? 

I consult to dozens of grantmakers and grantees each year—Federal, state, and local government agencies, foundations, and nonprofit organizations.  

In the grantmaking world, it’s common for funders to ask their grantees to explain whether they are meeting their targets.  

For example, one goal of a parenting program for teenage mothers and fathers might be to avoid repeat pregnancies. The health centers and high schools that are running the program might have to report whether there was, in fact, a decrease compared to a control group.  

Graphing these goals is an obvious choice. 

What’s Surprising? What Unfolded as Expected? 

Take off your data nerd hat. 

Put on your human hat.  

Step outside the math for a bit.  

Trust your gut instinct.  

I look for numbers that are surprising and unexpected.  

What’s surprising to you, personally?  

Surprising new facts make for interesting reports.  

Nobody wants to read the same old stories over and over and over.

Which Information Needs to be Shared with Others? Who? 

This thought-starter question keeps your data actionable.  

Examine your numbers.  

Who, in particular, needs to see these numbers?  

Think about all the different people who are involved in your project.  

Are there certain takeaway findings that your boss should probably know about? Or the boss’ boss? Or someone outside the organization?  

Who might act differently or make a different decision based on this new information?

What Increased Over Time? Decreased? Stayed the Same? 

Most projects have numbers available at multiple points in time.  

Examine how your numbers are changing over time, if at all.  

Sometimes a number will increase over time. Other times, a number will decrease over time.  

And other times, you might not notice any difference whatsoever. Flat lines can be useful, too! 

Data Storytelling Example: Highlighting a Flat Line in a Workforce Development Project 

I changed around the details, but this example is based loosely on a past project.  

Let’s pretend that you’re leading a career coaching program for adults who recently immigrated to the country. I consulted on a project like this a couple years ago.  

The purpose of the career coaching program was to get those adults into higher-paying jobs.  

A few times a year, the career counselors collected data on the participants. For example, they asked the participants how much they were being paid. The career counselors might even verify their wages by looking at pay slops or tax forms.  

The person responsible for compiling all this data should see whether wages are improving, declining, or staying steady.  

Imagine that you uncovered that wages for most program participants were staying steady—despite hundreds of thousands of tax dollars being poured into this program. 

That flat line has to be shared and talked about! Something needs to be adjusted ASAP. 

Your things-stayed-steady-over-time graph might look like this. 

We applied several data storytelling techniques. I bet you recognized them right away: 

1. We’ve got color contrast (all 30 participants’ individual lines are grayed out, and the average is highlighted in a darker brand color). 

2. We’ve got a takeaway title (“Wages Did Not Increase”). 

3. We’ve got numeric labels on a handful of key data points (the $18.27 average wage at the beginning, and the $18.30 average wage at the end). 

4. We’ve got (light) narrative annotations (“Average hourly wages: $18.30”) 

Looking for Useful Stories throughout the Analytical Process 

When do you look for possible data stories? 

Not the day before your project’s due!!!!!!!!!!!!!!!!!! 

Revisit these questions at each stage of your project’s analytical process. 

Look for Useful Stories in the Raw Data 

I start with my spreadsheets of raw data.  

I ask myself, “What’s going well? Did we reach our goals? What increased over time? What’s surprising? Which information needs to be shared with others?”  

I keep a running list of interesting nuggets in a notebook. 

Look for Useful Stories as You’re Compiling Tables for Your Appendices 

Later, I compile my analyses in tables. The tables often go in the appendix of a technical report.  

This means that I write the last pages of my report first.  

As I’m designing the tables, I ask myself those five questions again, and I add to my running list. 

Look for Useful Stories as You’re Designing Your Full Reports or Slideshows 

Next, I write my report (or create my slideshow, or whatever the finished product will be).  

I look through the tabulated data as I’m designing the report: Which numbers deserve to go into the body of the report?  

Look for Useful Stories as You’re Designing Summaries (One-Pagers, Infographics, Briefs, etc.) 

Finally, when my full report/slidedeck is complete, I pull out graphs that are so interesting that they deserve to go in a summary.  

I’m using the word summary loosely here.

A summary could be a one-page handout, an infographic, a shorter brief, etc. 

Yes, this is the place for those stories to shine.

Yes, you should’ve found stories along to way to include in your summaries. Hopefully!!!

This stage gives you one more chance to think carefully about useful gems in your dataset.

Don’t wait until the end of a project to think about the “so what?”  

This should be an ongoing, intentional process.  

When we think deeply about the data, our audiences will benefit from the added clarity. 

Your Turn 

What’s your process for uncovering interesting stories in your data? 

Do you have more thought-starter questions to add to the list? 

The post What Makes a Useful Data Story? 5 Questions to Ask  appeared first on Depict Data Studio.