Data as a language of influence
In a much-reported fashion, medieval librarian-turned-educator Shaq Ira reportedly composed the hit song “Data Don’t Lie to the Hips” in the fictional Ottoland tavern. But seriously, data may not lie (intentionally), but it definitely lies by what you do with it. This is why I created a session centered around practicing the fundamentals of data literacy: reading, writing, interpreting, and discussing data to effect change. Thank you to the brave souls hired by TYG (There You Go!) to make data-driven decisions during ATD Focus: Measurement and Analytics in L&D.
The language that speaks and influences data
A key takeaway from this session was to think about data in a different light. Think of data as language, not a pile of numbers in a spreadsheet or a graph in a dashboard. A language that can convey a lot about humans. Human actions, decisions, consequences of choices, beliefs, attitudes, etc. It’s a language we need to practice. The first thing you need to know is where to start. Instead of jumping to exciting solutions, start by asking the right questions. In other words:
You need to calm down!
– Dr. Dash Swift of TYG.
Apply critical thinking like Columbo. If the smoke is coming from the dog’s head instead of the cigar, there may be something wrong with the story presented, even if all parts of the story are included in the picture.
AI-generated “Columbo” with smoke coming out of a dog’s head
Data doesn’t lie: What can data tell us about learning?
When designing a “learning experience” (as we used to call it a course) without a plan to measure its effectiveness after the event, data language often focuses on what can be controlled, such as number of completions, time spent on the course, scores on knowledge checks, and sometimes survey data managed immediately after completion. This is like the Columbo painting above. It’s a fantasy. The illusion that all the pieces are covered. Even if the cigar isn’t real and the placement is questionable.
How to tell more effective stories?
Just like learning a foreign language, speaking data takes practice. The first thing to be cut from a “long” course is practice. We might spend a lot of time discussing one scenario that ultimately satisfies everyone, from content experts to legal and human resources personnel. Well, there could be several scenarios. But are these priorities correct? Without measuring what’s important to you at work, it’s difficult to know.
TL;DR: What data we focus on tells us our priorities.
learning is practice
In my session, I wanted participants to put this into practice. I wanted my students to think, reflect, ask questions, and see how others interpreted the same data. The core loop of the exercise is:
work in a team
Because that’s the only way to see how other people interpret the same data. Read the data statement
These statements refer to a fictitious sales pilot. interpret the meaning of a sentence
(literally, in context) discuss context
And the meaning behind that statement. Determine whether a statement is true, false, or unknown
Based on available data. Purchase data evidence items using the evidence store
These items range from facts, descriptive analysis, interactive data charts, and even linear regression results. Indicates the confidence level (0% to 100%) of the answer. This will affect your total score.
In the real world, there is often more than one correct answer. That doesn’t mean it’s right. What matters is how confident we are that we may not be 100% right based on what we know. Finally, enter a brief description of your selection in the free text field.
Optionally, you can also attach proof of your purchases to support your decision.
Once the decision is submitted, the team can move on to the next statement. Each of the above elements generated individual, team, and cross-team data to tell the story.
The first step is to generate and collect data so that “hips don’t lie”. Next, I wanted to share a high-level design of how to use data to make more informed decisions. Each of these examples serves a purpose and tells a specific story so that the facilitator can provide feedback on the spot. We built the application using AI tools such as ChatGPT to create the original PRD document, Windsurf with Claude Sonnet (including Model Context Protocol servers such as Sequential Thinking) to run the project, GitHub to store code, Vercel to deploy, and Supabase to store data.
Team progress overview
the purpose
See your team’s progress in submitting decisions at a glance
Decisions to support and questions to answer:
Which teams have selected members to serve as program managers (PMs)? Are any teams lagging behind in selection? In what order should I visit the team? What is the team’s overall progress on each statement (0% to 100%)? How can I remind the team of this statement for debriefing? (S1-S18 are statement codes, not actual wording.)
Team progress overview lets you see your entire team’s progress at a glance.
This view should be concise and simple. I’ve used it to monitor the progress of multiple teams. The column header shows the exact percentage of teams completed in that round, and hovering over the label will remind you of the actual statement. I used this view to determine the order in which I would visit breakout rooms to see if they needed help.
What story isn’t this data view telling? About the actual decisions? I know the team made a decision, but I don’t know what decision they made.
Line detail view
the purpose
Displays aggregated data about each statement with details.
Decisions to support and questions to answer:
How many teams responded to this statement? What is the average level of trust across teams? What is the level of agreement between teams? Did they choose the same answer?
And once I opened one of these “cards,” I wanted to find out more about the source of the confusion and disagreement.
The card headings will tell you how confident the team was and whether they all agreed on the same choice or whether there were different opinions. These insights can help you act quickly when agreement is high and spend more time on statements with which participants disagreed.
As I mentioned earlier, we often think of data-driven decisions as numbers-based choices. However, relying solely on numbers can completely mask the nuance and depth that can drive conversation and provide actionable feedback. That’s why I added the free text reasons the team needed.
The challenge with qualitative data, especially open text input, is that it takes time and effort to read and analyze. This is where AI can help. Using the ChatGPT API, all comments were analyzed and summarized to help understand common themes and discussions. I didn’t invest any further time into the solution I built, but I still have a backlog item to include open text inference in the scoring itself.
Data Don’t Lie: Evidence Store
the purpose
The evidence store lists a variety of items, from statements to graphs, to help your team make data-driven decisions.
Each item has a price and each team has a limit on the amount of coins they can spend. This is to ensure that not all items are available for purchase. Just like in the real world, limited resources must be prioritized, often making trade-offs. You can’t wait until you have all the data to be 100% confident in your answer. In other cases, you may have to make decisions without even having “enough” data.
Evidence store includes interactive data charts for purchase
As an example, the interactive data chart below can help your team understand the concept of change in a statistical sense. This interactive graph allows users to select different types of datasets to demonstrate size effects before and after scores. Users can see what the probability is: no effect, small effect, large effect, or random.
Interactive graph showing changes in various sizes
Based on our calculations, changes in raw mean scores may not represent changes of the same magnitude as normalized learning changes or effect sizes.
We also built an evidence view to see what evidence the team has purchased and how many times. Where do I start with the data? One last time. I would buy the process and project success criteria used in the pilot. Understanding what’s important can change the effort and time you spend making decisions about your data.
conclusion
Data is more than just numbers in a spreadsheet or graphs in a dashboard. Data like Hips doesn’t lie. It is the footprint of all human activities, actions, decisions, emotions, attitudes, culture, etc. Think of data as the fundamental building block of the language you can use to tell meaningful, life-changing stories. Or, if you prefer a more practical approach, start with the decisions to be made and the questions to be answered, and work backwards towards the insights and information that will make your data actionable.
PS I know there are three different schools of thought about data. Some believe that data is singular, some believe that data is plural, and the rest will say, “Is this really the biggest problem of our time?”
Image credits: All images in the article text were created/provided by the author. The AI-generated image of “Columbo” with smoke coming out of a dog’s head was generated by Zsolt Olah via Midjourney (2024). Originally published on www.linkedin.com
Source link
