Does an instrument assess 'looking good' or actual behaviour?

There is a design issue that all credible assessment tools must resolve. Let me first provide a little background for this discussion.

In the 1960s, three instruments assessed the five conflict modes: Blake and Mouton (1964), Lawrence and Lorsch (1967), and Hall (1969). So why did Ken Thomas and I develop a fourth instrument - the TKI - to measure conflict-handling behaviour?

Already by 1970, Ken and I were acutely aware of the potential social desirability response bias in all self-report assessments – the tendency for people to respond to test items in order to look good to themselves or others (whether this bias is conscious or unconscious) versus accurately disclosing their actual behaviour or interests. For example, we suspected that most people would prefer to see themselves as high on collaborating and low on avoiding, regardless of their actual conflict-handling behaviour. And perfectly in line with our initial expectations, when Ken and I collected research data on the three existing instruments, we found very high correlations between the social desirability of the five modes – measured in a few different ways – and people’s actual mode scores on each self-report assessment.

On average, more than 90% of the variance in mode scores was explained by social desirability (88%, 92%, and 96% for the Blake and Mouton, Lawrence and Lorsch, and Hall instruments, respectively). Consequently, for all practical purposes, these instruments were not measuring a person’s conflict-handling modes, since people’s scores were largely explained (accounted for) by social desirability.

To make a long story short, Ken and I repeatedly tested the social desirability of many conflict items for our new instrument and then paired items (for example, a collaborating item was paired with an avoiding item) that were both equal in social desirability. Since a person could not choose any of the TKI’s 30 A/B forced-choice items merely to look good – the pairs had been matched on social desirability – respondents now had to disclose candidly how they actually behaved in conflict situations. Indeed, when Ken and I collected research data on the TKI instrument, just as we had done for the other three instruments, the variance of mode scores on the TKI instrument that could be explained by social desirability dropped to 17%, as compared to over 90% for the other three instruments. That was a huge improvement!

Over the past 40 years, quite a few people have told me that it’s a bit demanding to respond to the TKI’s A/B format – even though it takes less than 10 minutes to complete all 30 forced-choice items. But these comments don’t surprise me. I believe that once the social desirability response bias has been removed from any instrument, it requires more diligence for respondents to disclose their actual behaviour or preferences, rather than to simply pick items to look good. But the intent of every self-report instrument is to measure what it claims to measure, not some other concept – such as social desirability.

Several additional conflict assessments have been published since the 1970s. In virtually all cases, the developers of those new instruments use something other than a forced-choice format (such as a five- or seven-point Likert scale for each conflict item), because they want to make their assessment easier for respondents. And yet, these developers seem to be unaware of what effect different scaling methods have on the social desirability response bias.

In the interest of ease, and ease alone, these newer assessments of the five conflict modes have thus fallen into the same trap as those three original assessments developed back in the 1960s. What goes around comes around! But ease of responding – by implicitly giving people the opportunity to look good to themselves or others – is no substitute for a valid assessment. In fact, I believe that the lasting and growing success of the TKI assessment over the past 40 years is precisely due to the fact that it minimizes the social desirability response bias and thus provides its users with an accurate assessment of their relative propensity to use the five conflict-handling modes.

For the original publication that compared the four conflict instruments on social desirability, see www.kilmanndiagnostics.com/mode.html

Ralph H. Kilmann, PhD, is CEO and Senior Consultant at Kilmann Diagnostics in Newport Coast, California. Formerly, he was the George H. Love Professor of Organization and Management at the Katz School of Business, University of Pittsburgh – which was his professional home for 30 years. He earned both his BS degree and MS degree in industrial administration from Carnegie Mellon University (1970) and a PhD degree in management from the University of California, Los Angeles (1972). Kilmann is co-author of the TKI assessment and has published more than twenty books and one hundred articles on conflict management, problem management, organizational design, change management, and quantum organizations.