ALISON BEARD: I’m Alison Beard.
ADI IGNATIUS: I’m Adi Ignatius, and this is the HBR IdeaCast.
ALISON BEARD: Adi, we often talk about the qualities of good leaders, and yet it’s still quite tricky to predict who’s really ready to rise to the challenge, right?
ADI IGNATIUS: Yeah, that’s true. Whether you’re promoting from within or bringing somebody from the outside, you have to make a bet, right? Is this person going to be a good team leader? Is this person going to innovate? Is this person going to get the best out of people? It’s a roll of the dice.
ALISON BEARD: Yeah. You want to know not just are they going to be effective, but also will they be unifying? Our guest today says that it actually might be something you can test for. David Deming is a professor of economics at Harvard University and Dean of Harvard College, and he’s long studied subjects like leadership, teamwork, and management, and found ways to assess people on them.
ADI IGNATIUS: It’s interesting, so it’s not just a roll of the dice. Are you seeing surprises about what the science says will make a great leader?
ALISON BEARD: There are a few, but I’ll give you one. He found that people who most want to be managers are worse at the job than those who are randomly assigned.
ADI IGNATIUS: Weirdly, that doesn’t surprise me.
ALISON BEARD: Yeah, I guess it didn’t surprise me that much either, but organizations do need to figure out ways to identify leaders who might not be putting their hands up, but will be the best team players, will be the most innovative, and will really improve the businesses and functions that they are involved in. Here’s our conversation.
David, thanks so much for coming on the show.
DAVID DEMING: Thanks for having me, Alison. It’s great to be here.
ALISON BEARD: Yeah, so I think a lot of us intuitively know what makes a great teammate or an effective leader, but explain what you found in your research on a more granular level.
DAVID DEMING: Yeah, so in our lab, we call it the skills lab. This is something I co-founded with my former PhD student, now a professor at University College London, Ben Weidmann. We develop science-backed, rigorous measures of squishy soft skills like teamwork, decision-making, management, leadership. We say, “How do we make the soft skills hard by creating assessments that are based on people’s performance in a real life setting?” To think of an example, we’re interested in what makes somebody good at working in a team. One way you can do that is you can interview them and say, “Oh, I like the cut of this person’s jib. They seem pretty nice. They’re going to be a good teammate.” Or, you can actually ask the question, when I put Alison on a team with somebody else, does she reliably make the team better? That is what we do in our lab. In order to do that, it requires a little bit of experimental wizardry because, if you think about it, if there’s three or four people on a team and they produce something really great and there’s only one outcome, who was most responsible? You don’t really know. You can’t disentangle the separate contributions of the teammates.
What we do is we say, “Well, we’re going to take Alison, we’re going to assign her to a team with David and Daniel and Steve, and then a team with Mary and Chris and Nathan,” and so on and so forth. If every time we put Alison on that team, she adds value to the team, meaning she makes the team do better than we would’ve otherwise predicted, then we say, “Alison is a team player.” We find that there really are reliable differences across people. Some people really do make teams better consistently and some don’t.
What we found in our study was that that was not really related to any demographic characteristics like age or education or gender. It also wasn’t related to IQ, but what it was related to is your score on something called the Reading the Mind in the Eyes Test, which is a widely used measure of emotion recognition. Basically, the kinds of people who can read emotions and faces tend to be good on teams.
ALISON BEARD: What exactly does the Reading the Mind’s Eye Test show you have in terms of leadership skills, collaborative skills, et cetera?
DAVID DEMING: Yeah, I would say it’s a surface level measure of, can you identify emotions? It’s maybe a wonky academic way to say it is that it’s necessary but not sufficient for being a good team player. If you cannot correctly identify people’s emotions, if I can’t tell that whatever I’m saying to you is making you angry, I probably won’t be good in a team with you, but I might be able to recognize it and also not be a good team player. People who are sociopaths are pretty good at reading emotions, but they’re not good at working in teams because they’re totally self-absorbed. It’s not enough. The test is very simple. It just shows you little slits of people’s eyes and you have to correctly identify the emotion they’re expressing. This test was actually originally developed by an autism researcher named Simon Baron-Cohen, because it was a reliable predictor of high functioning autism in people. Then they found that actually the test in its normal range predicts a lot of things, like whether you understand social situations, whether you can read people and so on.
ALISON BEARD: I will tell you that in preparing for this interview, I got very interested in this test. So I took it this morning. I scored a 31. I should say that it’s a 31 out of 36, not out of a hundred.
DAVID DEMING: Yes. That’s excellent.
ALISON BEARD: What does that mean about how I would behave in a team?
DAVID DEMING: If you think about what it means to be good in a team, it means that you have to be able to understand where other people are coming from in a psychological sense, but also understand what they’re good at and what you’re good at and how to fit in. If you and I were going to work on a team together, we would arrive at some division of labor, like maybe if we were writing a memo, maybe I do the data analysis because I’m an economist and maybe you do the writing or something like that. In order for us to figure out what that good division of labor is, we have to understand each other and we have to understand ourselves. A good team player is someone who understands how to fit in. I might play one role on one team and a different role on another team depending on the skills of my teammates. There’s some notion of adaptability and flexibility that comes with it. I think someone who can read others is somebody who’s just situationally and socially aware, it doesn’t just barrel down the path regardless of what’s happening around them. I think the test at a very high level is just a proxy for that kind of social awareness that makes you figure out how to contribute.
ALISON BEARD: When you did your studies on managers, did you find that they needed the same attributes to be seen as good team leaders?
DAVID DEMING: Yes and no. We did find that that test works, but we created another test to measure a managerial skill, which is really much more about the ability to assign people to roles in their best use. The intuition for this is like, if you’re a manager, what do you actually do? Well, part of what you do is work with people and have difficult conversations and so on. But a big part of your job is, let’s say you’ve got a bunch of workers and then maybe some other resources, and you have to figure out how to deploy it to what uses in a way that maximizes the potential output. A good manager has the skill of noticing what people are good at, what they’re not good at, and then figuring out how to fit the puzzle of pieces together. Economists call this allocative efficiency. It’s like how to allocate resources correctly. We designed a test that basically tests your ability to assign hypothetical workers to hypothetical jobs in a way that maximizes the total output. This test is a very good predictor of the ability to manage teams both in the lab and in the real world.
ALISON BEARD: You studied all of these team exercises, managerial exercises in lab settings. Do you think that the findings would hold in real world settings where people already have established relationships and reputations?
DAVID DEMING: In one of our studies, we attempted to answer exactly this. This study we called, “How Do You Identify a Good Manager?” We first set up a lab experiment where we randomly assign people to these roles similar to the team player setting, except one person has to play the role of manager and the others have to play the role of individual contributors. The manager has basically the job of submitting the assignment and coordinating production. Then we did it two ways, first of all. A funny side quest here. We asked the question, “how good are the managers who are randomly assigned?” Then we compared them to another condition where we asked, what if we ask people how much they want to be a manager on a one to 10 scale? Then instead of randomly assigning, we pick the people who most want to be managers. Then we found, to my surprise and amusement, that the managers who wanted to be managers were worse than randomly assigned. Just because you want to be a manager doesn’t mean you’re going to be good at it. Then we developed this experimental method, again, repeated random assignment of people to these roles to try to figure out who was a good manager. Then we said, okay, we’ve got this test. We call it a test of decision making skill. Then we say, now we’re going to see if this test predicts being a good manager in the real world.
We partnered with a large South American retail chain. It’s like a store that’s like groceries and other products, like a Target or something. These stores have managers in charge of them and the company, which is a large company, rotates the managers across the stores periodically just as part of their normal business practice. People might be in the downtown store for a couple of years and then go to a store in the suburbs for a couple years or whatever. We gave our test to several hundred managers in this firm. Then we asked, okay, if you scored high, we’re going to say maybe you’re a good manager, and you scored low on our test, maybe you’re a bad manager. Then we see when you rotate so that you go from having a bad manager to a good manager, or a good manager to a bad manager, how does that change outcomes in the store? What we found was that managers who score higher on our test, their stores have higher profits when they rotate to that store. It actually does seem that scoring high in this test makes the company sell more products, be more profitable, have higher margins and so on.
We found a couple of reasons for that. One of them is the managers who have good managerial skill, according to our measure, reduce the incidence of stockouts in firms. Stockouts are when a very popular product runs out, people can’t buy it anymore. That’s obviously costly for the company’s bottom line. The high scoring managers had fewer stockouts, so they were able to do inventory control and management much better.
ALISON BEARD: Yeah.
DAVID DEMING: The other thing is we measured throughput at cash registers directly. Meaning, basically, how many customers are you serving? We found was that the high scoring managers had higher throughput and looking under the hood even more, we found that the reason was that they were able to triage certain situations that reduce throughput. One example is if you have somebody at the cash register and they’re clipping a bunch of coupons and they’re standing there and they’re arguing with the cashier and it’s holding everything up, they would come out and triage that situation more effectively and keep the lines moving. The point is, our test, which is a pretty short test, was actually able to measure in the real world something that directly relates to the profitability and success of a store.
ALISON BEARD: The idea is to simulate either a teamwork or a managerial situation in the lab, figure out what distinguishes the highest performers on that task from the average performers, and then use that metric to evaluate people in the real world. So what would your recommendations be to organizations, to team managers to better identify first the people that they think will be great team players, really additive to group projects?
DAVID DEMING: Taking assessment of people’s skills seriously is value that many companies are leaving on the table. In a lot of places we have this technology for assessing people’s teamwork skills that we call an interview, where you and I sit down and you ask me some questions and you say, “Tell me about a time when… ” We have these interview protocols, but they’re by and large not very good and they tend to index on things that are much more about fit between people or chemistry between people rather than some general skill of working well on a team or being a manager. We do that because that’s all we have available, but you can’t actually take performance measurement seriously even of squishy soft skill things like teamwork and leadership and management. Our tests and our approach gives you a roadmap to doing that.
I would say the general lesson is, whatever is the thing you want someone to do, design an assessment that measures that thing as closely as possible. An interview is a conversation. Management or teamwork is partly a conversation, but it’s actually much more than that. You’re measuring a very small slice of what somebody actually does. It’s not really surprising that you don’t get what you expect to get when you do an interview.
Another example of that in the real world is that we’ve tried for years as a society to figure out how to predict what makes someone a good teacher. The Gates Foundation invested millions of dollars a decade ago or so into these Measurements of Effective Teaching project. They ask, “is it content mastery? Is it your SAT score? Is it your grades? Is it the college you went to? Is it which professional development program you went?” It was none of those things. None of those things were predicting who was a good teacher. What did predict it was just giving a short sample lesson. Teaching for one year is a very strong predictor of teaching quality in future years, because it’s like, you’re doing the thing. It’s a performance-based measurement, which is just much more accurate. I would say put people in situations where you ask them to perform and then see how they do, and that’s going to be a much better predictor of their success than whatever off the shelf interview or test that you give them.
ALISON BEARD: Once you feel that you’ve identified your great team players, what does that mean for how you assign roles and responsibilities? Because not everyone is going to excel at that, but they will have other great things.
DAVID DEMING: Who are the people who are actually valuable contributors to your organization but are not good team players? These are people who are pretty inflexible, who do one thing well, but aren’t that interested in… not just getting along, but also adapting to other people. You give those people things that are more modular, that don’t create interdependencies with others. The real value of teamwork skills is that it allows you to be a part of a flexible, fast moving group. In a lot of my research, I talk about the different returns to these skills in the labor market. One thing we found over and over again is that what I would call social skill, the ability to work well with others, is complimentary with technical skills or just general capabilities. In other words, if I have great social skills, but I’m actually not good at anything, that doesn’t really buy you much. But if you have a real talent or expertise and you’re able to communicate that to other people and to use that talent in coordination with others, it’s hugely valuable.
ALISON BEARD: When you talk about the social skills that specifically correlate with being a great team player, how does that differ from what we know already about emotional intelligence?
DAVID DEMING: Yeah. Well, so it’s related to emotional intelligence. I think that a lot of the tests of emotional intelligence are situational vignettes and things where there isn’t an obvious right answer. One of the virtues of this test that we use is you just know whether… You said you got a 31 out of 36. We have an alternative of just asking people, “are you a people person?” These five factor personality tests or the Myers-Briggs or things where there’s not a right or wrong answer, it’s just you’re self-identifying. One of the things we found in our management study I mentioned, we asked these people this question, “are you good at reading people?” There was no relationship at all between whether people said they were good at reading people and whether they actually were on this test. In fact, the managers in the self-selected condition I mentioned, the ones who self-nominated, yeah, I want to be a manager, they tended to be overconfident in their social skills, meaning they would say, “yes, I’m a people person,” but score low on the test, much worse than 31 out of 36. There’s a type, they tend to be more male. There’s a type of person who is gregarious and extroverted and socially dominant, but isn’t actually a good team player. These are adjacent concepts. I think probably we tend to promote and interview and index on that more than the actual ability to work well with others.
ALISON BEARD: Okay. Moving on to the skills that make a great manager, we talked about the social skills and the resource allocation. How can organizations do a better job of identifying strong leadership candidates who fit that profile?
DAVID DEMING: In our paper, we did something to try to answer this, by essentially simulating different decision rules. Suppose I promoted managers based on their social skills, suppose I promoted them based on their performance on the job. We call this the Peter Principle. Peter Principle, the idea is you keep getting promoted to your level of incompetence. If we just said whoever the best individual contributors are will make them managers. That actually is better than randomly picking people and it’s better than self-nomination. It has some information in it, but it’s not as good as picking on management skill directly.
My lesson from this is actually this is a distinct skill. Some people are just good at things, but if you design a test or an assessment, performance-based assessment that is specifically focused on the thing you’re trying to capture, you can do better than just picking a smart person. I think basically organizations should invest in developing these kinds of assessments. Again, it doesn’t have to be one test that everyone takes. It’s actually, well, “can I create a simulated environment that is like the thing I actually want a person to do and then test people on that?” That actually if you can create a reliable way to predict who’s going to be a good performer in your organization, that’s worth a lot of money, so you should probably invest in it.
ALISON BEARD: Can you also train for the kind of skills that you’re talking about?
DAVID DEMING: Great question. It’s not something I know the answer to scientifically because we haven’t gotten there yet, but I have an instinct, which is, I think you can. I think social skills like any other skills can be trained and developed, but also like anything else, people have different levels of it at baseline. It’s easier for some people than others, but you can learn. I know my own case, I think being a father and having two teenage daughters in my case built my social skills quite a bit. I’m much more socially aware than I was when I was 20 years younger.
ALISON BEARD: So what you’re describing, do you see that as a radical change from the way that team leaders and organizations currently approach how they staff projects and how they promote people?
DAVID DEMING: I don’t know if it’s a radical change. Many high performing businesses tend to have scientific ways of approaching some aspects of what they do. Toyota Lean, Six Sigma, these management techniques that really make it into a science, like how do we increase throughput on a production line or how do we optimize sales? There’s lots of sophisticated work that goes into that. Then I think we go to the softer side, the more people oriented side and say, “Ah, we don’t know.” I think the lesson is, companies have a lot of these tools in their toolkit already. They just only apply them to some domains. I think what I’m saying is that same approach can actually work in the softer, more people oriented domain. It just requires a little bit of thoughtfulness about how you measure progress and success. You can’t measure somebody’s social skills with a pen and paper test very well. You have to actually put them in social situations and that’s a little bit harder, but I think it very much pays off.
ALISON BEARD: But those pen and paper tests can give you some clue—
DAVID DEMING: Yes
ALISON BEARD: Online tests, we’re primarily talking about at this point, but then—
DAVID DEMING: Yeah
ALISON BEARD: You should give them situational tests to ensure they have the skills they need.
DAVID DEMING: Yeah. Can I just say, just to be precise, what we did is we said, okay, we’re going to measure people’s performance in a very sophisticated way. Then we’re going to ask, “can we even approximate it with a test?” But the tests, even though they were predictive, they were imperfectly predictive. If you ask me like, “okay, I can hire a person who scored 36 out of 36 on the Reading the Mind in the Eyes Test, or I can hire a person who performed really well on the teams in your lab-based assessment,” I would pick the latter over the former any day because that’s a real high stakes measurement of somebody’s ability to perform in a team.
ALISON BEARD: What advice would you give for leaders and organizations that want to implement more of the simulation exercises that you’re talking about? How do you do that at scale in ways that actually serve as a real world test?
DAVID DEMING: First thing I would say is give me a call. Second thing I would say is actually technology, AI is going to make this much cheaper and more possible. We already do this in professions where the cost of failure is extremely high. If you are a pilot, they don’t stick you in a plane and say, keep crashing until you figure out how to do it right. You build a flight simulator. It’s very expensive and takes a lot of time, but you do that because the cost of failure is terrible.
Similarly, like medical students, when you learn how to perform surgery, we have surgery simulations, same thing. What that tells me is this actually is the best way to learn, but it’s been too costly in the past to do it for everybody. I think technologies like AI, the ability to build realistic world simulations of certain processes, that cost has gone way down just in the past few years. I think you’re going to see a proliferation of this kind of learning in different settings. Imagine a simulation where you learn how to have a difficult conversation with your boss, like negotiating for a raise. You could simulate that, you could try it 20 times before you do the real thing. Who wouldn’t sign up for that?
ALISON BEARD: You mentioned you’re finding that the people who volunteer to be managers aren’t the ones who are best at it. Does that mean that we should be wary of the people who put their hands up first?
DAVID DEMING: There’s a selection bias in it. I don’t think it means the people who… In our study, we didn’t find that they were worse than the people who said, “I don’t want to be managers.” I would say that it’s just not really related to it all. Just because somebody raises their hand and they want to be a leader does not necessarily mean they’ll be good at it. It means they think they’ll be good at it. Those are just very different things. It tends to load on certain characteristics, being overconfident, being male, things like that. There’s things we can do to look out for that. This is an example of where an actual assessment eliminates traditional biases in hiring.
ALISON BEARD: What if you see that the people on your team who would be good team players and good managers actually prefer to stay in individual roles? Should you push them out of their comfort zone?
DAVID DEMING: I think it depends on the person. I know that in my case, I’m now in a leadership management role, and that’s not something I ever wanted, but circumstances pushed me into it and I found that I enjoy it more than I thought and I’m better at it more than I thought. I think there’s some of that, but also if there’s people who just live in fear of having to manage people and they’re awesome individual contributors, maybe don’t rock the boat, maybe find somebody else. There’s people who don’t want to do it and there’s people who really don’t want to do it. Part of the maybe people skill of this even softer side is figuring out which of those two categories somebody is in. I will say one other thing too, which is, even our measures, and we put it out in the real world and it is a sophisticated measure of performance, it’s still incomplete. Because there’s a thing about the ability to, let’s say, gain trust over long-term relationships that we’re not capturing, but it’s extremely important. Amy Edmondson talks about psychological safety in teams and that’s something that you can measure in any one instance of a team, but what really matters is, if I’m following this person day after day and they are managing me, do I trust them? Do I think they’ll have my back in a difficult situation? Do I believe them when they communicate? Are they authentic? We all know those things are important and our measure doesn’t really capture that, but I think if anything, it’s understating the importance of these skills. Maybe the next test will find a way to measure long-term social skills or long-term trust too.
ALISON BEARD: Let’s turn to another study that you did on what it takes to effectively manage AI agents.
DAVID DEMING: Yeah.
ALISON BEARD: Explain to listeners what you found there.
DAVID DEMING: Yeah. We did our management experiments again. We basically replicated that initial study and we developed, again, this repeated random assignment method I mentioned to figure out who’s a good manager. We repeatedly randomly assigned people to manage different groups and asking who was good at it. We had these estimates of who are our good managers, who are our bad managers. Then we asked the question, if you’re good at managing a team of people, how likely is it that you’ll also be good at managing a team of AI agents? Then we took the same managers where we had this ground truth, if you like, of how good of a manager you are with people. Then we gave those managers teams of agents to work with on the same task they were using with people called the hidden profile task. It’s widely used. The task is you have to elicit information. Everybody on the team knows something you need to solve a problem, but not everyone knows everything. You have to, through dialogue, elicit the information, figure out who knows what, share information and solve the problem.
That’s something you can do with agents, you can do with people. We found that the correlation between management skills with people and management skills with agents was 0.81. That’s very high. That’s like the correlation between height and blocks per game in the NBA or the correlation between math scores and reading scores on a state test or something. It’s very highly correlated. What it’s saying is that this is a core skill that really does transfer across the two contexts. It’s because it’s, what is it? It’s like, okay, what questions should I ask? How should I put people in conversation with each other in order to surface information that I need to know to make a decision? There’s no reason why you couldn’t do that with AI agents in the same way you do with people.
ALISON BEARD: It’s still counterintuitive to think that you would need the same social skills—
DAVID DEMING: Yeah.
ALISON BEARD: —to work on a team with artificial intelligence—
DAVID DEMING: Yeah.
ALISON BEARD: —or lead that team than it would be to lead humans. You would think it would be less important.
DAVID DEMING: Yeah, I agree. I think probably it’s because of the test we’re using, we’re measuring short run management skills in both cases. Again, as I mentioned, there’s this longer run relational aspect to the job where it’s like we see each other day in, day out and you have to trust me and we go through difficult situations together. We’re not really capturing that on either side of it.
My gut is, it’s hard to think about actually what it means to manage agents in that context because it is much more human. But that getting people to trust you is a very important part of management, but there’s another part which is just like who’s doing what and how do I keep the trains running on time? I think we’re really measuring that part of it and less the trust building aspects of it.
ALISON BEARD: If you had to point to specific behaviors that make people both effective at managing or working with humans and AI agents, it would be around asking good questions, being curious, connecting dots.
DAVID DEMING: Yeah. Also, not talking too much. One of the things that we noticed, we looked at the logs, the chat logs. These are online conversations, both with people and with agents, so you can actually analyze the transcripts, and the good managers didn’t talk more. They actually talked a bit less or a very similar amount, but they asked the right questions and they put their teammates in conversation with each other too.
Part of it is, in my own experience, trying to run a meeting, if you’re in charge and you talk first, you bury the information. People don’t always surface. You have to be thoughtful about when you interject and what you say. You can see the good managers doing it. They’re saying, “Hey, tell me a little more about that. Hey, how does that relate to what you know?” You’re just setting up the conversation in a way that gets you what you need. I think that’s a common skill with people and with our robot overloads.
ALISON BEARD: What do you think this means bigger picture for human-AI collaboration in organizations in terms of how senior leaders should think about staffing teams, ensuring that they’re good human team players and good human AI managers?
DAVID DEMING: Well, I think it means that we’re all managers now because as AI, especially agentic AI, becomes a bigger part of the workplace, you have to think about how you’re going to deploy those computational resources to aid your own work product. That’s a management task. It’s like I’ve got some things available to me, these super intelligent agents that never sleep, but also don’t always have a ton of common sense and can’t read my mind. They don’t know exactly what I want. I have to communicate it to them. In much the same way that I can’t be a good manager if I don’t communicate well to people what I want.
That’s also true with the context that you give at a chatbot or an AI agent. I think the skill of management, asking good questions or aligning resources in an organization, that’s going to be incredibly important in the future. I guess that’s good news for business schools or good news for people who teach management. I think it does mean that we probably have to educate differently and emphasize some of those skills more earlier.
ALISON BEARD: What’s next for the research you want to do at the skills lab?
DAVID DEMING: Well, I’m very much hoping that we can do more real-world validation of our studies. What I most want is for people to take this idea seriously and for more people to be doing it. Part of the shadow agenda here is I think a world where we actually value those skills, surface them, talk about them more, is a better world, that this actually is a thing that’s not just economically valuable, but also makes you a better person.
ALISON BEARD: Well, it’s always nice to see when lab research can have real world implications and impact. David, thanks so much for talking with me today.
DAVID DEMING: It was my pleasure, Alison. Thanks so much for having me.
ALISON BEARD: That was David Deming, professor of economics at Harvard University and Dean of Harvard College.
This Thursday, IdeaCast launches our special five-week HBR Innovation Summit series featuring the leaders driving innovation in a time of rapid technological change. First up: my conversation with Jack Dorsey, Block Head, Chairman, and cofounder of Block, on Building a Company Around Intelligence.
Next week on Tuesday, Adi will interview Aarron Atkinson-Toal on the Science of Why We Buy.
If you found this episode helpful, please share it with a colleague and be sure to subscribe and rate IdeaCast in Apple Podcast, Spotify, or wherever you listen. If you want to help leaders move the world forward, consider subscribing to Harvard Business Review. You’ll get access to the HBR mobile app, the weekly exclusive insider newsletter, and unlimited access to HBR online. Just head to hbr.org/subscribe.
Thanks to the team, senior producers Mary Dooe and Rachel Swaby and senior production editor Kristin Murphy Romano. Thanks to you for listening to the HBR IdeaCast. We’ll be back with that bonus episode on Thursday. I’m Alison Beard.
