Input by Many, Decisions by ????

I recently posted about what I call the the Great Decoupling of information flows and decision rights within organizations: the fact that it’s now so cheap to process, store, and transmit data that there’s no need to be stingy with information by sending it only to decision makers. I wrote:

"Most of what I’ve seen recently strongly indicates that the sudden near-disappearance of information costs is bringing up a fascinating and consequential set of questions for organization designers and corporate leaders. They now have the freedom to place decision rights where they wish without being hampered by information costs. What are the long-term consequences of this great decoupling?"

In that post, however, I think I actually understated the extent to which information technology is opening up new possibilities about decision making and governance within organizations. In "The Great Decoupling" I focused only on disappearing information costs, but there’s also another very interesting development: the appearance of technologies for collecting and distilling distributed knowledge in novel ways. These include:

An internal blogosphere. I spent time recently at one of the world’s largest technology companies, talking with the people who were responsible for deploying enterprise 2.0 tools, including employee blogs. On the page that listed the most recent blog posts I saw a title something like "Why Our Recently Announced Strategy is Misguided." I was a little surprised by this, and asked the team if this level of feistiness was rare, and if the people who wrote such posts found themselves in hot water. They assured me that the answer to both questions was no. 

It struck me that I was looking at an excellent tool for gathering informed feedback on topics of interest. And there seemed to be a lot of cross-talk among blogs; employees were leaving comments for each other, posting in response to previous posts, etc. I try not to be a wide-eyed technoptimist, or to say "everything’s different now!" each time I see a new technology, but I do think this internal blogosphere was something new under the sun. I don’t see how a company, especially a large and geographically dispersed one, could hold an ongoing, public, open-to-all conversation about important topics without E2.0 tools.

Wikis. In the default configuration for mediawiki software, each page has an accompanying ‘discussion‘ page where contributors have background conversations, hash out any differences, and together decide what should go on the main page. This arrangement facilitates not only collection of information, but also convergence of opinion. The mass of high quality Wikipedia articles on controversial topics like global warming attests to the efficacy of wiki technology and the dual-page structure.

Prediction markets, which are essentially stock markets where traded securities are tied not to the future profits of a company, but instead to other future events such as the results of an election, whether or not a competitor will ship a product on time, or next quarter’s total sales. Securities in prediction markets have prices (just like shares in a ‘normal’ stock market do), and people use the market to trade with each other by buying and selling these securities.  Because traders have differing beliefs about what the securities were worth, and because events occur over time that alter these beliefs, the prices of securities vary over time in prediction markets. 

In his seminal 1945 article "The Use of Knowledge in Society" the economist Friedrich Hayek highlighted how important markets and prices were, and it’s worthwhile to quote from the article at some length:

"The… problem of a rational economic order is… that the knowledge… of which we must make use never exists in concentrated or integrated form but solely as the dispersed bits of incomplete and frequently contradictory knowledge which all the separate individuals possess. The economic problem of society is… a problem of the utilization of knowledge which is not given to anyone in its totality.

We must look at the price system as such a mechanism for communicating information if we want to understand its real function… The most significant fact about this system is… how little the individual participants need to know in order to be able to take the right action… It is more than a metaphor to describe the price system as a kind of machinery for registering change, or a system of telecommunications which enables individual producers to watch merely the movement of a few pointers, as an engineer might watch the hands of a few dials, in order to adjust their activities to changes of which they may never know more than is reflected in the price movement…

The marvel is that in a case like that of a scarcity of one raw material, without an order being issued, without more than perhaps a handful of people knowing the cause, tens of thousands of people whose identity could not be ascertained by months of investigation, are made to use the material or its products more sparingly; i.e., they move in the right direction…

I have deliberately used the word "marvel" to shock the reader out of the complacency with which we often take the working of this mechanism for granted. I am convinced that if it were the result of deliberate human design, and if the people guided by the price changes understood that their decisions have significance far beyond their immediate aim, this mechanism would have been acclaimed as one of the greatest triumphs of the human mind."

In his book The Wisdom of Crowds, James Surowiecki echoes these points, applies them to companies, and wonders why more managers are not prediction market enthusiasts: "Corporate strategy is all about collecting information from many different sources, evaluating the probabilities of potential outcomes, and making decisions in the face of an uncertain future.  These are tasks for which [prediction] markets are tailor-made.  Yet companies have remained, for the most part, indifferent to this source of potentially excellent information, and have been surprisingly unwilling to improve their decision making by tapping into the collective wisdom of their employees."

All of these technologies help accomplish a critical task: aggregating relevant knowledge in ways that were not previously possible. And this is where things get really interesting because as I wrote earlier, the Golden Rule for decision making is that decision rights should be aligned with relevant knowledge. 

Well, if E2.0 tools let crowds come together and aggregate their relevant knowledge, why shouldn’t companies take the next logical step and also use the tools to let the crowds themselves make decisions related to this knowledge? Why shouldn’t companies, for example, just automatically convert their prediction market’s consensus opinion for next quarter’s sales into purchasing and replenishment orders to factories? Why shouldn’t they let all employees use a wiki to collectively design their next generation of products? In short, if crowds really are wise, shouldn’t crowds make important decisions?

Friedrich Engels didn’t think this was possible. In an 1872 letter to Theodor Cuno, Engels disparaged the anti-authoritarian views of a rival school of socialists: 

"How these people propose to run a factory, work a railway or steer a ship without having in the last resort one deciding will, without a unified direction, they do not indeed tell us."

In a 2006 seminar to my academic area at HBS, MIT’s Tom Malone showed a video that gave a technology-based answer to Engels’s skepticism. It showed the attendees at a conference trying to landing a plane by committee, using nothing but paddles that were red on one side, green on the other. Technology in the conference room continuously watched the paddles and moved an on-screen device according to the will of the majority. In his book Out of Control, Kevin Kelly described what happened when the on-screen device became a plane headed for a distant runway:

"Loren Carpenter launches an airplane flight simulator on the screen. His instructions are terse: "You guys on the left are controlling roll; you on the right, pitch…" The plane is airborne. The pilot is…5,000 novices. For once the auditorium is completely silent. Everyone studies the navigation instruments as the scene outside the windshield sinks in. The plane is headed for a landing in a pink valley among pink hills. The runway looks very tiny…

Nobody decided whether to turn left or right, or even to turn at all. Nobody was in charge. But as if of one mind, the plane banks and turns wide… The mob decides in unison, without lateral communication, like a flock of birds taking off, to pull up once more. On the way up the plane rolls a bit. And then rolls a bit more. At some magical moment, the same strong thought simultaneously infects five thousand minds: "I wonder if we can do a 360?"

Without speaking a word, the collective keeps tilting the plane. There’s no undoing it. As the horizon spins dizzily, 5,000 amateur pilots roll a jet on their first solo flight. It was actually quite graceful. They give themselves a standing ovation."

The video provided a vivid demonstration of what Malone meant when he wrote in The Future of Work  that "With new… technologies… it is now becoming economically feasible –  for the first time in history –  to give huge numbers of workers the information they need to make more choices for themselves."

But the video also showed something else, something that I at first overlooked because I was so impressed at the sight of 5000 novices flying a plane: the crowd actually fails, twice, to land the plane properly.  As Kelly writes (in a passage I largely omitted from the extract above):

"But group mind seems to be a liability in the decisive moments of touchdown, where there is no room for averages. As the 5,000 conference participants begin to take down their plane for landing, the hush in the hall is ended by abrupt shouts and urgent commands. The auditorium becomes a gigantic cockpit in crisis. "Green, green, green!" one faction shouts. "More red!" a moment later from the crowd. "Red, red! REEEEED!" The plane is pitching to the left in a sickening way. It is obvious that it will miss the landing strip and arrive wing first… [T]he flight simulator entails long delays in feedback from lever to effect, from the moment you tap the aileron to the moment it banks. The latent signals confuse the group mind. It is caught in oscillations of overcompensation. The plane is lurching wildly. Yet the mob somehow aborts the landing and pulls the plane up sensibly. They turn the plane around to try again…  It tries landing again. Again it approaches cockeyed. The mob decides in unison, without lateral communication, like a flock of birds taking off, to pull up once more."

This example shows, I think, the danger of becoming too enamored of crowd wisdom. We now have novel and powerful technologies for aggregating formerly dispersed knowledge and tapping into the wisdom of crowds. I agree with Surowiecki that these tools are currently underexplored within companies, and much of my teaching and writing at present aims to show business leaders new opportunities and encourage experimentation. 

But I don’t think I’ve ever said "turn over as many decision rights as possible to the crowd assembled by Enterprise 2.0 technologies." This would be extraordinarily rash advice. There are many good reasons to assign important decision rights clearly and cleanly to individuals within companies, and we should proceed very carefully indeed before we move away from this practice. 

These individuals, however, should in turn think carefully before turning their backs on technology-enabled aggregated knowledge and crowd wisdom as inputs to their decision making processes. Smart business decisions are competitive differentiators. Why pass up a good opportunity to get smarter when making them?