I recently had the opportunity to be a guest on James Pann’s YouTube show to talk about reports. James is an Associate Professor, program evaluator, and psychologist who specializes in evaluating health, human service, and educational programs.
Watch Our Conversation
Here’s a recap of our conversation.
My Introduction to Dusty Shelf Reports
I recently had my 7th year anniversary of working for myself. I started out as an evaluator and I always loved the data viz side of my job. Some people love doing logic models or lit reviews, me I love poring through spreadsheets trying to figure out the pattern.
Early on in my career, I was throwing all my effort into a report, staying late, working on the weekends, doing whatever it took to do my best work. My boss sat me down and kindly said, “Ann, you’re a great team member and I’m so glad to have you. I’ve noticed you’re working really late on this evaluation report. It matters, sure, but I don’t want you to sacrifice your personal life for a dusty shelf report.”
I’d never heard that term before and I instantly felt crushed, but also thinking, “there’s got to a better way- this can’t be the standard”.
Why Evaluators Should Be Learning About Data Visualization and Reporting
If you’re working in academia or are a student in grad school students, then you absolutely need to write academic reports.
But if you go outside of academia, there are many other workplace settings that have their own standards.
The bar is being raised so much (in a good way) in thinking beyond the report. Can we also have interactive dashboards? Infographics? Amazing things are being done.
The tricky part is that none of us were trained for how you go from really technical reports in academic settings to non-technical reports, non-technical audiences, and quick turnaround times.
Resources Someone Just Starting Out in Evaluation (and Others) Can Use to Get Proficient in Data Visualization
For a student just getting started or if you’re new to the evaluation field, you can take one of my courses (hint, hint, nudge, nudge).
Soar Beyond the Dusty Shelf Report is our free mini-course where you’ll gain practical techniques that you can apply immediately to improve your visuals.
Start with a Self-Assessment of Your Own Reports
For reporting, the best place to start would be to do a self-assessment. Ask yourself: What’ss the impact of your reports (if any)?
There are four red flags to really look for with reports (or slideshows or dashboards, etc.). Look out for these:
- No Response. No response at all after you send it out.
- Promise to Follow-Up Later. You get a response that says, “Thanks, I’ll let you know if I have any questions.”
- “Compliments.” You get a response that says, “Thanks, we can tell that a really technical team worked on this report. It’s very detailed and thorough.”
- Won’t Read It. They ask for another format entirely like a slideshow, one-pager, etc.
Creating a Dataviz Wall of Fame
One thing I did for myself early one that was so helpful was to create a Dataviz Wall of Fame.
I used to look for examples of great graphs, dashboards, infographics, etc., and would print them out and plaster them on the wall above my desk.
Nowadays, you could have a Pinterest board, folder on your computer, or “favorite” them on Twitter.
Start by actively collecting examples that you love. But then, push yourself to the next level by asking, “What’s the rubric for this? Why is it good? Is it the writing, the chart type, the colors?” Once you can identify why it’s great, you can start to replicate those details.
Data Visualization Podcasts to Learn From
There are lot of great podcasts out there, too.
My favorite data visualization podcasts are:
- Data Viz Today with Alli Torban
- Data Stories with Enrico Bertini and Moritz Stefaner
- Data + Love with Zack Bowders
- PolicyViz with Jon Schwabish
The podcasters and their guests represent a variety of educational paths and career backgrounds. I love hearing about all the “correct” ways to get into this field.
Techniques for Engaging Project Stakeholders in the Reporting Process
I co-authored an article on using Data Placemats, which is a technique you can use to engage project stakeholders in the reporting process.
I want you to step away from the idea of waiting until the end of the project to give the completed report/slideshow and hope that it gets used.
Instead, there’s a three-step process you can follow:
- Develop data placemats with preliminary findings. I use everyday software like Word or PowerPoint, and create one graph per finding. For example, if you gave out a 20-question survey, you’d have one graph per survey question. The graphs don’t need much formatting at all. Aim for one-color traditional graphs, not storytelling graphs.
- Meet with stakeholders and have them explain their findings in their own words. You’re a facilitator, not a lecturer. Pose discussion questions and then sit back and listen. Watch as the participants experience lightbulb moments for themselves.
- Write the final report. Use storytelling graphs with intentional dark-light contrast to draw attention to key findings. Include the stakeholders’ comments from that earlier meeting.
By engaging stakeholders in the data interpretation process, we can build buy-in, which makes it more likely that the data gets used to inform decisions.
Easy Fixes for Reports
Here are the most common mistakes–and easy fixes–that I see.
Not Going Beyond the Report
You only have a report. A report can’t meet every audiences’ need. You need one product per audience.
For example, one audience might prefer an infographic, while another audience prefers a dashboard.
Not Structuring Our Reports
Not starting with the “So What?” The most important information is buried in the report. Put the “So What?” on the first page and get your audience in the game right away. Don’t risk losing your audience.
Not following brand guidelines. Use brand fonts and colors, and especially think about using your recipients brand guidelines. It’ll take you about 10 minutes but shows that you’re willing to go the extra mile and it looks more professional.
Not Designing Each Page
Not aiming for 1+ visual per page. I define visuals very broadly. You might use the logos of organizations, or include a timeline to show the numerous steps taken.
Forgetting to lower the reading level. The average American reading level is 6th through 8th grade. Plug your text into a readability level checker and make sure your reading level matches your audience. Aim to be about two levels below your audience’s educational level. For example, if most of your audience members have completed Master’s degrees, then aim for two levels below that (high school).
How to Obtain Feedback on Your Work
James asked how I collect feedback on my own work.
When you blog and YouTube like I do, where your work is front facing, people will tell you!
I remember posting my first YouTube video about a decade ago. The first comment was from an anonymous person who wrote in ALL CAPS, saying they couldn’t hear anything over the clicking of my keyboard and said it was such terrible quality. I didn’t even know people used a microphone! I watched YouTube videos about microphones and lighting to get to my setup now.
I did a webinar a week or two ago and my phone number is on my website (I forward my business phone number to my cell phone). This guy called and said, “I paid for your webinar you did yesterday and I have one problem with it…” I instantly went into flight or fight mode, wondering what he was going to say. He went on to say how most presenters just read off a script but I didn’t, it was one of the best he’d seen. People will just call and will tell you very openly how they feel about your work so you have to have a poker face.
Masterminds
I’m also in a few masterminds with some other business owners where we act as each other’s sounding boards and brain trusts. I’ll throw out ideas to them and they’ll flat out tell me that it’s not a great idea. It’s really helpful to have a peer group that you trust.
Collecting Feedback After Trainings and Courses
When I do trainings, I collect feedback forms or do an internet survey.
In my larger consulting projects, I build it in to have a debrief. They turn into problem solving for when hiccups may happen in the future.
In shorter projects, I look for repeat clients or referrals. If I never hear from them again, that would worry me.
At the end of my courses, student receive a course survey. I also have behind the scene data for course students that let me know how much of a course they’ve completed, how often they’re logging in, if they’re watching all of a video or not and more. I can see which videos people are re-watching or even which sections they’re re-watching which tells me where they’re getting stuck.
Inspiration for Developing Blog Posts
James asked where I get inspiration for blog posts.
I usually write blog posts on questions that workshop or course participants are asking.
For example, if everybody is struggling on how do I make sure my colors are accessible, that’s a great blog post.
I also do a lot of before/after makeovers which help people see all the little steps that can make a big difference. I try to show the behind the scenes thought process so that people can then replicate it in their work.
More recently I’ve been doing interviews to try and get to know the people in data viz better, and then I turn that interview into a blog post.
Finding a topic that you personally care about is key. I don’t think blogging is supposed to feel like work. You could monetize your blog, but it would take a lot to make a full time living directly from blog ads. It’s mostly a hobby and supposed to be fun. I love the idea of thinking about what’s interesting to you and what do you want to explore more.
Learn More
Subscribe to James Pann’s YouTube show.
Your Turn
Comment below: Which reporting tips were new to you? Which tips are you going to put into practice? Have reporting tips of your own to share?
Leave a Reply