A researcher can easily fall into the trap of losing the ability to look at his own work critically. He spends a lot of time looking at data and trying to figure out what’s going on, and eventually comes up a story that explains what he’s seeing. When he’s feeling pedantic (which is often) he calls this story a theory. He then tries to see if this story holds up — in other words he tests the theory — by gathering more data and looking at it in different ways.
An intellectually honest researcher tries hard to prove his own story wrong by looking for data that do not support the story he’s come up with, and by seeing if another story explains things better. This part of the work is critically important, as it helps to ensure that the researcher doesn’t look silly later on when someone else comes up with the ‘right’ story. It also strengthens and refines the theory itself. My colleague Clay Christensen, in his description of theory building, talks about the important role played by anomalies, or observations that don’t fit the current story and so spur the quest for a better one. In short, a conscientious researcher tries hard to find other explanations and anomalies.
You’ve figured out by now that I’m not talking about faceless researchers for abstract reasons. I’m selfishly talking about me and my work, and asking for help in the search for other explanations and anomalies related to a story I’ve helped come up with.
Erik Brynjolfsson, Michael Sorell, Feng Zhu, and I developed a story about the relationship between IT and competition. Over the past couple years we’ve built the story and tried to test it in as many different ways as we could. We couldn’t do all the tests we wanted because of data limitations, but this is almost always the case with empirical research — the data are never perfect or complete.
So we did the best we could. In a later post I’ll describe what I think is the best piece of story-testing that we did, one that yielded results I still find fascinating and powerful. It’s a test inspired by baseball and a very smart researcher in a totally unrelated field. I hope you find the story as interesting as I do — stay tuned.
We eventually came up with a pretty bold story; bold because it makes some strong claims about how the world of business is changing, what’s behind these changes (spoiler alert: it’s IT), and why. I’ll inelegantly summarize the story in a few bullet points:
- In the middle of the 1990s, there appeared two major additions to the ‘toolkit’ of information technologies available to companies: The Web (which introduced the business world to the Internet), and commercial enterprise information systems like ERP. Both of these were novel, and were much more than incremental improvements; they were quantum leaps forward in corporate IT.
- These technology innovations increase the impact of managerial innovations. They let a company take good ideas — improved business processes, sets of workflows, plans about who should get to make which decisions, etc. — and propagate them widely and with very high fidelity throughout the company or value chain. Prior to the mid 90s these types of good ideas often had only local and temporary impact. Thanks to the new technologies, from the mid 1990s on these ideas had broader, deeper, and longer-lasting impact.
- Good ideas are competitively valuable.
- Good ideas with broad and deep impact are even more competitively valuable. They increase the gap between winners (the companies with good ideas) and losers (the rest) in an industry.
- So after the new technologies appeared and increased the power of good ideas, the gaps between winners and losers in technology-consuming industries started to get bigger.
- These gaps were biggest in industries that spend the most on IT. IT spending, in other words, is a substitute (or ‘proxy’ ) for the volume of technology-enabled good ideas in an industry.
- Because people within companies keep having good ideas, the winners and losers in any particular industry don’t stay stable over time. IT, in other words, doesn’t just increase the gaps between winners and losers. It also makes it less likely that the winners at any point in time will stay on top. This churn or turbulence is greater in industries that spend more on IT.
- This new, nastier competition does not depend on continued IT innovation. It only depends on continued managerial innovation. If all the technology vendors were to close up shop tomorrow competition in all industries would not eventually revert to where it was prior to the mid-1990s. The current IT toolkit lets companies propagate business ideas faster, more broadly, and with higher fidelity. That’s all that’s necessary to increase the pace of competition, and to keep it high. Of course, the tech vendors are not about to shut themselves down and we’ll see a lot more innovation from them; this will only serve to further increase competitive nastiness. But technology innovation is the icing on the cake of managerial innovation.
Erik and I present a full version of the above story in the current (July/August 2008) issue of Harvard Business Review, and show the different kinds of data we gathered to assemble and test the story. We also discuss other possible explanations for the patterns we observe and talk about how we tried to test these alternative stories.
After finishing all this work I have a problem: I believe my own story. Like my colleagues, I’ve tried hard to listen to objections, constructive criticism, and other possible explanations for the patterns we’ve documented. And I don’t buy them. We’ve tested some of the alternative explanations and found them lacking, and the others just don’t seem that plausible to me.
Faith in one’s own work and results is a fine thing, but blind faith is a bad thing. And I’m worried that I’ve been so close to this work for so long, and so excited by our findings and conclusions, that I might be missing something.
So I’d love your help. Please read our HBR article (it’s freely available online for at least a month) and, if you’re so inclined, the academic paper that presents the work more formally. Take a look at our story and the data that support it. And see if you can think of any other plausible explanation for the patterns we observe and document.
If you can, please let me know about it. Leave a comment or, if you prefer, contact me directly. I’m sincerely interested in hearing other stories. Of course, if you think our story is spot on and needs no amendments or elaborations, that would be great to hear, too. But I’m most keen to hear what else you think could be going on. If IT isn’t changing competition in the way we describe, what is?