Earlier this year, two sociologists
provoked a probing debate about the integrity and value of interviewing as a
research method. Colin Jerolmack and
Shamus Khan, respectively of New York University and Columbia University &
Berlin School of Creative Leadership (Steinbeis University), launched their
broadside in the May 2014 issue of Sociological Methods & Research. While this methodological deep dive and
accompanying critical discussion were made in the context of a specific
discipline, the challenges mounted have relevance for research in all fields where
interviews are widely used. For
management and business studies, in particular, in which interviews of leaders
are often central to substantiating and defending accounts and insights of
organizational and industry practice, such questioning has potentially
far-reaching implications.
Though Jerolmack and Khan’s
argument is dense and sophisticated, one of their major concerns is captured in
what they call the ‘Attitudinal Fallacy,’ which claims that correlations of
reported attitudes with situated behavior are never high enough to presume
equivalence. ‘Attitudes are poor predictors of action’, they contend, and since
interviews are shaped by and convey subjects’ attitudes, they constitute an
inherently unreliable approach to researching the reality of these individuals’
actions.
Respondents to the original piece
closely engaged this issue of ‘ABC’, or Attitude-Behavior Consistency. For instance, Karen Cerulo, a researcher of
cognition and culture at Rutgers, acknowledged the longstanding discrepancy or
word and action while nevertheless urging that critics don’t overstate the
problem. Sometimes what people think or say
does inform what they do, she notes, and so we should still value, if with more
circumspection, the content of interviews.
Moreover, even if words and actions are sometimes specifically
inconsistent, individual level data can sometimes illustrate more general
patterns of action worth highlighting.
To their credit, Jerolmack and Khan
don’t reject interviews entirely but instead call for greater methodological
pluralism in research. Rather than
relying entirely or disproportionately on interviews, in other words, they urge
increased deployment of ethnographies, deep or participant observation,
qualitative data gathering, little or big data, and holistic analysis. In research on business and leadership,
especially, questioning the integrity of interviews and calling for multiple
research methods compounds the challenge of gathering data and insights about
individuals and organizations.
This is not only an issue for researchers
of business and management research and analysis but for readers and
practitioners. Ask yourself: How many recent
books or articles on leadership or effective business practice have you read
that rely on interviews, typically of successful leaders about their beliefs
and experiences, to advance a distinctive insight or approach? Content drawn from interviews or other
first-person sourcing of business leaders’ viewpoints is especially common,
indeed is a major selling point, in popular business writing and journalism.
Connecting Individual Attitudes with Organizational Behaviors
The words of leaders are typically taken by readers
and listeners as reliable expressions of their behaviors – past, present, or
future. The credibility of those words,
and those speaking or writing them, is crucial, of course, and turns back to
the fundamental challenge of ABC. While
we may debate whether Marissa Mayer’s online strategy for Yahoo! is the right
one, for example, we accept that her words capture what she and the company are
actually doing.
Yet the matters of credibility and consistency are
further tested by the break between individual claims or characterizations and
the often widely disparate actions of organizations. Even as leaders speak, the connection to collective
behavior – as summary, explanation, cause, plan – remains opaque amidst so many
possible causes. As much as we want to
ground our understanding of corporate and collective reality in personal attitudes,
explaining corporate or organizational behavior demands wider attention to
persons, situations and the actual interactions between them.
Research and
Access in Business and Organizations
Jerolmack and Khan identify another
shortcoming, the ‘Accounting Fallacy’, to emphasize the fallibility of individual
accounts and self-reporting and how they cannot accurately stand in for
analyses of actual behavior. To generate
wider explanations of the situations and interactions in organizational life requires
multiple research approaches. Yet in
most corporate settings, these are impossible for outside researchers and
analysts to pursue. Confidentiality,
distraction, and the priorities of time and other resource allocation typically
limit or preclude more robust access to people and situations.
The usual response to this lack of
access is to judge leaders’ words by specific indicators of their
organizations’ performance. Rather than methodological
pluralism in research about corporate actions, particular outcomes – like
products, services, and financial, innovation or creative results – become the
basis for assessing those words. This
makes certain sense. After all, we
invest in some firms, say Alibaba, because of the value they can deliver or
promise to customers and in the marketplace. Yet even in a world of inevitably incomplete
information, attributing the successful or failed results of complex
organizational behavior to individual words is a poor substitute for ascertaining
more robust and varied understanding of the actual collective situations
producing the results.
Interviews as Public and Professional Performances
Another source of uncertainty in
trying to connect the words of leaders to their organizations’ actions is the recognition
that many interviews, whether public or ostensibly not, are performances. Both the carefully crafted pronouncements of
leaders and their closely managed professional personae affect whatever claims
researchers and others may be able to make about them. Even when the words may not correspond to
actual behaviors of the organization, present or past, they may nevertheless be
presented in the interests of the company, its brand management, and public
relations.
Ultimately, an ethical aspect
emerges here: do we trust what leaders say?
And even if we recognize that some words may be exaggerated (recall
Steve Jobs’ ‘reality distortion field’), do we justify that hyperbole in terms
of other ends, like increasing shareholder or market value or hyping and
delivering new products? While the trust
issue exists in all interviews, where we can question the correlation of
self-reports of behavior, the test of consistency between words and actions
becomes all the more complicated when leaders may be trying to achieve multiple
ends and speak to multiple audiences in the same interview.
Authorities, Elites, and Celebrities
If the interview responses of
leaders are shaped by a range of organizational interests, the interactions
with researchers and others that prompt these responses can be strongly
impacted by the status or reputation of the leader himself or herself. We choose to interview or otherwise solicit
the perspectives of leaders in the first place because of their experience or
position, expertise or accomplishment. Yet
that very appeal to authority can limit the thoroughness of the interview and
consequently the integrity of the verbal data emerging from it.
At a practical and human level, interacting
with very senior or elite leaders involves unequal power relations that can be difficult
to navigate. Consider the prospect of
interviewing Richard Branson or Sheryl Sandberg or Martin Sorrell. Establishing the kind of access, trust and
rapport so important for substantive exchanges and then producing meaningful
data is particularly challenging with the involvement of subjects like these who
possess great seniority, reputation, and even celebrity.
Caveat Lector
Taken together, these
considerations suggest that interviews with leaders, since they typically concern
collective or corporate behaviors as well as those leaders’ reports of their
own individual actions, would benefit from more multidimensional analysis and
layered understanding of their accounts.
While this call resonates with Jerolmack and Khan’s general imperative
to expand the range and integration of methods employed in researching social or
corporate action, it takes on special significance in a field of research where
leaders’ words are accorded such value and prominence. For external researchers of organizations,
gaining meaningful access and developing robust contextual understanding of the
special situations (and constraints) of organizational life are ongoing
challenges. For readers of business and
management research as well as more popular analyses reliant upon the words of
leaders, the methodological debate should prompt us to question, constructively
but more diligently, the accuracy of the claims leaders make about their own
actions as well as those of their organizations.
