My MIT Center for Digital Business colleague Erik Brynjolfsson and I published an article in the Wall Street Journal on April 25. It described some recent research we’ve been working on, in collaboration with McKinsey, on the broad question what makes a company ‘good at IT?’
The piece that ran in the Journal ended up being very short, and included no graphics. So I wanted to post a bit more information about this research here, and also include a few graphs illustrating what I think is an important and interesting element of our work (if you want the most complete description available, a paper written for academic audiences is available here).
Here’s a bit more on the research design and results:
We combined data from three sources. The first was a survey we developed (and conducted by phone) of the organizational and IT management practices at major American companies. We limited ourselves to publicly-traded companies so that we could make use of our second data source: their annual reports. Many previous studies simply asked companies how they were doing; we wanted ‘harder’ data on their financial performance. Our third data source, developed by Lorin Hitt at Wharton and Sonny Tambe at New York University, was a composite measure of how much IT each company had.
We got at least partial data from 330 companies, across all industries, and analyzed it using a standard economic framework called a production function. This relates a company’s inputs (capital, labor, IT, and so on) to outputs like revenue and value-added. We quantified the organizational and IT management practices from our survey and analyzed how they affected the production function.
And what did we find? In line with prior research, standard inputs like capital, labor and IT were strongly associated with output. However, some firms vastly outperformed others, even when controlling for all the standard inputs. We found that several of the management practices could explain these differences in performance. The strongest relationship, one that held up across all our analyses, was between higher output and greater emphasis on data-driven decision making. The companies that reported they had the data they needed and actually used it to make decisions (instead of relying more on intuition and expertise), were the ones with the highest productivity and profitability.
This relationship held across the wide range of industries included in our study, and was quite strong. Companies that were one standard deviation higher in being data-driven had 4% higher productivity and 6% higher profits than the average in our sample, all else being equal.
But isn’t everyone data-driven these days? Haven’t modern tools for analytics and business intelligence transformed how most companies make decisions and sense their environments? These tools are widely available, after all, and people have been touting their benefits for a while now. But our data show clearly that not all companies are ready, willing, or able to become data-driven. As the graph shows, most companies rated themselves somewhere between 3 and 4 on our aggregate 5-point scale in this area, and many others put themselves below average.
We observed the same situation with other practices related to better performance, such as having clear technology governance, decentralized decision-making, and more outward-looking data-gathering practices. Our analyses indicate that these are associated with significantly better performance, but they’re far from universally adopted.
To underscore that last point about how much variation there is in these practices, here’s a set of histograms showing how our respondents ranked themselves in being data driven and externally focused, and having consistent business practices and good IT governance. Each of these is an aggregate measure constructed from a set of questions where respondents gave numerical answers, with 1 being low (e.g. “we’re not at all data driven”) and 5 high (“we’re extremely data driven”). Some of the IT governance questions used only a 3-point scale, so its histogram has 4 as a maximum.
I find these graphs pretty remarkable. They show that best practices are far from universal, even once they’re universally recognized as being ‘best.’
So here’s a very simple logic chain: Good exploitation of technology leads to good business performance. Good exploitation of technology depends on a set of best practices. These practices are not widely adopted. Therefore not all firms will be able to exploit technology well. And therefore firms will have different levels of performance.
We believe that the data support this set of cause-and-effect statements. Do you? Do our data and arguments agree with your experience and observations? And do you think that in the future technology is going to make companies more similar, or more dissimilar? I strongly believe that the trends we observe are going to continue and accelerate, and that tech is going to make companies more dissimilar over time. Leave a comment, please, and tell us if you agree.