I am a huge fan of the TV series “Mad Men,” which has aired for two seasons on AMC. The show revolves around the employees of Sterling Cooper, a fictional Madison Avenue ad agency in the early 1960s. It’s written and filmed with the intelligence and attention to detail that we’ve come to expect from the best television since “The Sopranos” showed us just how good the small screen can be. The third season of “Mad Men” will start in August; my withdrawal symptoms are becoming acute, but I think I’ll be able to make it that long.

The show’s main character is Don Draper, who is in many ways not a nice person. He’s selfish, deceitful, unpleasant to coworkers, and serially unfaithful to his wife. He is, however, extraordinarily good at his job, which is to think up compelling ad campaigns for the agency’s clients (he also looks damn good in a suit). Many episodes center on Draper’s efforts to come up with the slogan that will differentiate a company’s offerings and cause them to fly off the shelves. He has a clear idea of his own skills; as he said during a boardroom battle, “I sell products, not advertising.”

And does he?  Well, the show is ambiguous on this point. He’s clearly creative, insightful about how consumers and markets work, and ridiculously effective in pitch meetings. The show’s first season, for example, opens with him figuring out a whole new way to advertise cigarettes as health claims were being disallowed (”Lucky Strike: It’s toasted.”) and closes with a meeting in which he convinces Kodak to stop referring to its new slide projector as a wheel (”It’s not called a wheel. It’s called a carousel.”).

But “Mad Men” spends very little time on whether or not these ad campaigns worked –  whether they led to greater sales. I’m not sure if this is deliberate or not, but it is pretty typical, especially for the era. It’s historically been quite difficult to assess the effectiveness of a given campaign, or even of total spending on the Don Drapers and Sterling Coopers of the world. As the pioneering 19th century retailer John Wanamaker said, “Half the money I spend on advertising is wasted; the trouble is I don’t know which half.”

When there’s this much uncertainty it’s a common strategy to put all one’s trust (and dollars) with a business oracle like Draper. But is this still the right strategy when times change and it’s possible to see through the fog of business better? Or is it at least possible to combine insights from business oracles with other sources and methods, thereby making improved business decisions?

I got to thinking about these questions during the conferences I attended at MIT last week, which were all about IT’s impact on the business world. The presentations I heard and data I saw indicated to me that the era of Don Draper –  of wholehearted and unquestioning trust in business oracles — might well be coming to an end

To explain why this is, let me first present a strawman of the oracle-based mode of making important business decisions, then describe some alternatives, or at least complements, to it. It won’t surprise most readers to see that these alternatives and complements have a strong information technology component.

Business decisions that spring from the work of oracles like Draper have a few common attributes. They tend to be:

• Opaque. Draper couldn’t explain his creative process if he tried. He just ‘knows’ what will work and what won’t. Watching him, I was reminded of Cayce Pollard, the heroine of William Gibson’s novel Pattern Recognition. Pollard is a savant about brands, and one of her services is an expensive but brutally simple evaluation of a proposed logo. She looks at it, once, and says yes or no. Her clients are not allowed to ask any follow-up questions because she wouldn’t have answers for them. She can’t explain how she knows whether or not it’s good; she just does.
• Not amendable. Decisions by Draper, Pollard, and other oracles tend to be take-it-or-leave-it propositions, not subject to refinement.
• Not disconfirmable.  It’s very hard to know if Draper’s decisions are good ones. How would Kodak know if it’s really a better idea to call its slide projector a carousel rather than a wheel?
• Not revisited. Because they’re not amendable and not disconfirmable, there’s little point in going back and reviewing an oracle’s decisions.
• Universal. Draper gives a single right answer, one that leads to mass media campaigns. There is one market out there, and he knows the one best way to best reach it.

In addition, old-school business oracles like Don Draper share a few characteristics themselves. They are:

• Individualistic. Draper works largely alone. He has a small staff of writers and graphic artists, but they work largely to flesh out his ideas and carry out his orders.
• Accepting of few inputs. Draper takes ideas from his staff, but not from anyone else. He brushes off input from account executive Pete Campbell (who is admittedly a tool and a back-stabber) and throws away a research report on smoking written by a Freudian psychologist (OK, that might not be such a bad idea…).
• Charismatic. People at Sterling Cooper know that if they can just get current or prospective clients into a meeting with Draper, he likely will close the deal. He can project the ‘reality distortion field’ ascribed to real-world folk like Steve Jobs and Shai Agassi (wish I had it).
• Intolerant of competition, second-guessing, and unofficial channels. Good ideas come from Don and Don alone. He tries to fire Campbell for pitching an idea to a client.
• Credentialed. Draper is the head of creative for Sterling Cooper, so he must be good.  In the business world, common credentials include degrees (especially MBAs) from good schools, impressive job titles, time spent at leading companies, accumulated years of experience, and reputation among insiders. Credentials are externally visible signals of quality, of oracle-dom.

The above lists of characteristics are focused on a single fictional character in the advertising industry, but in my experience they’re fairly common across business oracles and their decisions in many real-world settings as well. When I reflect on how I’ve seen strategy, marketing, planning, and product design decisions made at large organizations, I see a lot of the stuff listed above.

To be sure, I also see business oracles gathering lots of data, commissioning studies, and sometimes even running experiments. But I often get the sense that the point of all this activity is to confirm the soundness of the oracle’s initial idea, rather than to test it (a state of affairs captured elegantly by this New Yorker cartoon). Several people at last week’s workshop on business experimentation observed that it takes months for many companies to set up even a simple experiment today, and opined that this is because of the great care taken to ensure the outcome. I found myself nodding my head in agreement as I read in last Sunday’s New York Times magazine the following passage written by Matthew Crawford, who dropped out of organizational life to open a motorcycle repair shop:

“[C]ertain perversities became apparent as I settled into the job. It sometimes required me to reason backward, from desired conclusion to suitable premise. The organization had taken certain positions, and there were some facts it was more fond of than others… Further, my boss seemed intent on retraining me according to [the] cognitive style… of the corporate world… This style demanded that I project an image of rationality but not indulge too much in actual reasoning.”

The main problem with relying on old-school oracles to make important business decisions, though, isn’t their backward reasoning or sometimes false rationality. It’s the fact that they might be wrong. When you follow the advice of a Don Draper you’re making a big bet on a black box –  you’re committing substantial resources (like the money and effort required for a national ad campaign) based on a decision process that you don’t understand very well and can’t understand better. You’re just trusting in the oracle’s wisdom, and if he turns out to be not that wise you’re in trouble.

Is there an alternative? Can businesses lessen their dependence on the world’s Don Drapers, even in nebulous, touchy-feely, hard-to-quantify areas like advertising and brand-building?

I think they can. To show how, I’ll stick with the topic of advertising and show how a few things I’ve seen recently indicate to me that there is an alternative to relying so heavily on business oracles. I am by no means an expert on advertising, digital or otherwise; I’m concentrating on this discipline because it’s been historically very rich in oracles like Draper, and because I’ve recently been exposed to some advertising and marketing technologies I find fascinating.

The first of these relates to the alchemy of coming up with a good idea. Kamal Malek and Noubar Afeyan, two MIT-trained engineers, realized that new products, packaging, brands, ad campaigns, etc. are actually combinations of a relatively small number of elements. The packaging for a new line of eco-friendly copier paper, for example, is a combination of colors, logos, other design elements, descriptive words, and a few other important features. A branding oracle would come up with one combination of these, or direct his team to put a few options together so he could pick the ‘best.’ A more scientifically inclined oracle might put a few options in front of a few focus groups to get some consumer input. But all of these approaches ignore the vast majority of possible combinations of attributes; they don’t ever consider most of the ways that colors, logos, words, and so on can be combined.

Malek and Afeyanm started a company called Affinnova because they realized that in the era of the Web a very different approach was possible. Affinnova uses some pretty high-powered math (including genetic algorithms and conjoint analysis) to figure out which combination of attributes appears ‘best’ to target customers. Staples actually did want to figure out packaging for its eco-friendly papers, so according to a Business Week article:

Affinnova set up a panel of 750 consumers across the country who, over the course of a week, participated in a 20-minute study of Staples’ paper line. Each was shown a screen of three possible packaging designs and asked to select their favorite. The software analyzed their choices in real time, and presented three new designs. “In total, we put 22,000 choices in front of consumers for the Staples test,” says Steve Lamoureux, Affinnova chief marketing officer. By looking at selections over multiple generations and across the whole panel, the software identified preference patterns—a tendency toward a certain color or font or wording—and ultimately identified the top concepts.

As a result of the Affinnova study, Staples made several changes. For instance, the company ditched the special green packaging of its recycled papers, instead incorporating its eco-offerings into its regular line, which is packaged in red (basic), blue (midrange), or gold (premium). A green band across the top of the new packaging indicates the percentage of recycled content, and a large triangular recycling icon just below the paper type reiterates its environmental credentials.

The only thing opaque about this process is the details of the algorithms used, which very few of us would understand anyway. It’s based on constant amendment, competition, and revisiting  instead of being one shot take-it-or-leave-it. It’s also not universal; Affinnova’s software can identify whether the target population breaks down into discrete segments, each with its own preferences, or whether there really is only one best answer. This process also depends on tons of inputs instead of only a few, and is blind to the charisma and credentials of any participant. Affinnova presented case studies of its effectiveness, but I’m not aware of any systematic research to investigate whether its methods do in fact yield better results in the marketplace than do oracles.

I did learn, though, about research that both presents an alternative to one-size-fits-all marketing and tests its effectiveness. It’s well established that people have differing cognitive styles; some are visual while others are verbal, some are analytical while others are holistic, etc. On the Web it’s not difficult at all to serve up different versions of the same ad tailored to each style, but it is hard to quickly figure out what someone’s cognitive style is. People don’t just show up and announce it when they visit a site, and they’re likely not willing to take a short quiz to find out.

So MIT’s John Hauser, Glen Urban, Guilherme Liberali, and Michael Braun fell back on some old math to help them serve up appropriately customized ads. The Gittins index was formulated by the statistician J. C. Gittens in 1979 in response to a nasty problem that had been around since at least World War II: how can you get the most money over time out of a slot machine with two arms, each of which has unknown payout odds? The Gittins index helps with this (do not ask me how), and can also be used to arrive at a quick and good guess about cognitive styles based on a person’s initial clicks around a site (think of each click as a pull on one arm of a multi-armed slot machine).

Hauser, Urban, and their colleagues souped up the Gittins index for the Web, and used it in combination with a bunch of other hardcore techniques to serve up ads with the same content but differentiated look and feel to prospective customers of BT’s broadband Internet service at an experimental site. In other words, they ‘morphed’ the website as they learned more about the visitor’s cognitive style. They found that sales increased by almost 20% during the trial period, which would translate into approximately $80 million in additional sales if implemented at full scale.

As I said, I’m not an advertising expert, but isn’t this kind of sales jump remarkable? Here again, we see a departure from one-shot and one-size-fits-all approaches to important decisions, and a move toward experimentation, iteration, and contingent answers. We also see that the MIT team, in the tradition of all good scientists, reached back deep into relevant prior work in a few disciplines (cognitive psychology, statistics, operations research, etc.) instead of just relying on their personal storehouse of knowledge. They also substituted quantitative rigor for qualitative intuition whenever possible, and they tested all their hypotheses and conclusions instead of patting themselves on the back.

At the workshop on business experimentation I heard many examples of similar approaches to making important business decisions. Not all of them involved collecting data via the Web. Jim Manzi, for example, related how Applied Predictive Technologies uses all kinds of corporate data to design and evaluate its experiments; the example he gave concerned changing convenience store layouts. All the presentations, though, stressed decision-making processes that were very far from those employed at Sterling Cooper. And while some of the presenters were charismatic and all had excellent credentials, they sounded not at all like Don Draper.

Other examples I’ve come across recently show me that even if a company wants to rely heavily on old-school oracles to make important decisions, it’s becoming much easier to assess over time whether or not they were good decisions. In the realm of digital marketing and advertising a rich toolkit now exists to measure the effectiveness of different campaigns, channels, adword buys, and so on. Hubspot and Clickfox are two companies that specialize in this work.

With this toolkit in place, the key skill becomes not so much coming up with ideas that might work as walking away from ideas that aren’t working. Michael Sikorsky, founder of the crowdsourced business idea incubator Cambrian House, helped me realize how important this is. He described how Cambrian House set up a series of tests for every idea, then immediately stopped working on a given idea once it failed even if Sikorsky or other senior people were personally enamored of it. This kind of discipline is rare –  most of us hold on to our brainchildren far too tightly, and far too long.

Mounds of data, clever algorithms, and the ability to run rapid-yet-rigorous experiments combine to lessen our dependence on the Don Drapers of the business world, and to treat their work not as the final and complete answer but rather as part of a broader, rigorous process. I used the word diminishment in the title of this post, not disappearance. I’m not saying that creative and insightful people are no longer valuable, or that the flash of inspiration can (or should) be automated. I’m just saying that such flashes should, whenever possible, be subject to all feasible useful scrutiny, and that technology is vastly increasing both the quantity and quality of available scrutiny.

I’m also not saying that there are no true business oracles out there; the track records of Steve Jobs and Warren Buffet speak for themselves. But the number of people who think they’re oracles is much larger than the number who actually are. Mind you, I’m not calling the rest con men or charlatans, just folk who are overconfident in their predictive abilities. Overconfidence is a well-established cognitive bias, and is reinforced by our tendency to remember what we got right and forget our mistakes and errors.

Malcolm Gladwell wrote a fascinating New Yorker article a while back on criminal profilers –  the guys who look around a murder scene then tell the police to look for (for example) a white male in his thirties who lives alone, enjoys outdoor activities, is personally neat, works in a low-level white collar job, and has difficulty talking to women. Gladwell calls this kind of prediction the “Hedunit” (from “He done it…”), but after looking carefully at profilers’ track records concludes that:

“[The profiler] did not really understand the mind of the [criminal]. He seems to have understood only that, if you make a great number of predictions, the ones that were wrong will soon be forgotten, and the ones that turn out to be true will make you famous. The Hedunit is not a triumph of forensic analysis. It’s a party trick.”

Making good business decisions is not as important as catching murderers, but they’re both too important to be addressed with party tricks. I’d like to see the conception of a business oracle undergo a substantial change. The old-school business oracle was someone like Don Draper who said to a company “I know what will work.” The new version, I hope, is someone who says to a company “I know how to figure out what’s working.”

What do you think –  am I being too hard on Don Draper-style business oracles?  Is my fondness for technology, analytical rigor, and experimentation blinding me to something important about making business decisions? Or do you share my belief that old-school business oracles need to be consulted with a bit less blind reverence these days?  And are you looking forward to the third season of “Mad Men” as much as I am?  Leave a comment, please, and let us know.

Even though I’m heading down the river to a sweet gig at MIT, my affiliation with Harvard will continue. I’ve been named a fellow at the University’s Berkman Center for the Internet and Society, starting in July. This is a real honor, and I’m very grateful.

The Berkman Center, which started at the Law School, is a legendary gathering place for sharp people who are interested in the the Internet’s impact. I look down the list of current and past fellows, see people like Jimmy Wales, Karim Lakhani, David Weinberger, and Doc Searls, and know that I’ll learn a lot and have great sounding boards for ideas. Since Charles Nesson is on the Berkman faculty I might also get in some good poker games…

I’d like to thank all the Berkman faculty and staff, particularly Urs Glasser and Colin Maclay, for giving me this opportunity. I am confident that my association with the Center will give me plenty to think (and blog) about.

I’ve spent the past three days drinking from the geek (remember, that’s a term of praise) firehose at MIT. Monday was a Center for Digital Business workshop on IT-fueled business experimentation organized by Michael Schrage and Erik Brynjolfsson (with a tiny bit of help from me). I’ll blog more about this event later; I’m still wrapping my mind around what I learned.

Tuesday was the CDB’s annual sponsors meeting; I gave the lunchtime keynote on Enterprise 2.0, but had a lot more fun listening to presentations on the wide variety of research conducted at the Center.

I tweeted about both these events, as did McKinsey’s Michael Chui; our hashtags are #mitee for the experimentation workshop, and #mitcdb for the meeting yesterday.

Today is MIT’s annual CIO symposium, and I just listened to a panel discussion among Erik B., Tom Malone (founder of the Center for Collective Intelligence), and Jeanne Ross (director of the Center for Information Systems Research) on “The Future of IT.” The panel was moderated by Gary Beach of CIO Magazine, who asked more than one question about whether cloud computing was that next big thing.

The cloud seems to be on everyone’s mind these days; the symposium has a panel dedicated to it this afternoon. Most folk, myself included, think that whether or not cloud computing deserves the title of Next Big Thing it’s a pretty big deal, and will have a large impact on IT producers, IT departments, and CIOs in the years to come.

But what about CEOs and other senior line executives? What, if anything, does the cloud mean for them? A question asked by Gary this morning highlighted one view of the cloud’s impact on top management. He asked attendees to raise their hands if their company had a CEO. Of course, most hands went up. He then told them that the ‘E’ he was referring to was ‘Electricity.’ We laughed and lowered our hands.

Gary’s point was that stuff that becomes a commodity –  that becomes undifferentiated, universally available, and cheaper and cheaper over time –  ceases to deserve attention from the very top of a business. Electricity is vital to business, and at one time let to a massive restructuring of manufacturing industries, but we no longer have Chief Electricity Officers. Why not? Because the time when it ‘changed the game’ of manufacturing is long gone, and it’s now just a commodity that comes out of the wall.

Many cloud computing advocates consciously or unconsciously use the imagery of electricity to describe the shift now taking place. Companies are ‘plugging into the cloud’ and computing is becoming a ‘utility’ delivered from a ‘grid’ of service providers. The conclusion is clear: that IT resources like processing power, storage, development environments, memory, and applications (even integrated ones) will soon come out of the wall reliably and in the required amounts, just like electricity does now. And the consumers of these resources will need to give no more thought to them than I do to the current that cooks my toast in the morning.

This imagery is simple, intuitive, and seductive. It’s also, I believe, deeply unhelpful and counterproductive. The problem with it is that it will lead to conversations like the following:

“Hey boss, this powerful IT stuff now comes out of the wall, just like electricity does. Should we get some of it? It’ll be more reliable and cost less than what we’re doing now.”

“Yes, make it so.”

“OK, what kinds and how much?”

“Huh? There aren’t ‘kinds’ of electricity, and I thought the whole point was that it would come out of the wall in whatever quantity we required. So just plug us in and stop bothering me.”

My main problem with the IT-as-electricity metaphor is that it provides a lovely justification for business leaders to spend less of their time and attention on IT-related issues. I hope all readers are by now thoroughly sick of hearing me say that I believe this to be a very, very bad idea, and a great strategy for getting throttled by your IT-savvy competition at some point down the road.

But cloud computing is definitely moving us toward having something come out of the wall. If electricity’s not the right metaphor for it, what is? As I was sitting listening to the panel this morning I challenged myself to do better. Here’s what I came up with:

Cloud computing is moving us toward a world in which some kinds of business change come out of the wall. Not all types of business change will come out of this wall, but some will. A few of these will be easy, most will be at least somewhat difficult. Some will be profound, some trivial. The different types of cloud-enabled change will affect various constituencies, both internal and external.

The reason I like this imagery better is that it will (I hope) lead to conversations more like:

“Hey boss, thanks to cloud computing some types of business change are now coming out of the wall, just like electricity does.  I’m going to go ahead and plug us into the cloud, OK?”

“OVER MY DEAD BODY!!!”

“Why not?  Don’t you want to change?”

“Of course I do, but I want a BIG say in defining the changes this organization signs up for, and I also want to lead them. Now, what exactly is coming out of the wall, and what’s it likely to do to us?”

“Well, that’s a longer conversation.”

“Well, let’s start having it.”

The imagery we use to talk about big trends like cloud computing is going to shape perceptions, so we’d better be careful with it. I am deeply uncomfortable with imagery that portrays IT as anything like a commodity or utility, because busy businesspeople in most industries don’t give a moment’s thought to commodities and utilities — to stuff that comes out of a wall.

They all care, though, about innovation and improvement and refinement and disruption –  about the many flavors of business change. I think we technology enthusiasts would be well-served if we started talking in those terms, instead of using language that will relegate our offerings to the utility closet.

What do you think?  Do you like the IT-as-electricity imagery?  If so, why?  And are there any problems with my IT-as-change imagery?  Leave a comment, please, and let us know.

In the interests of naked self-promotion I wanted to show off the cover to my book Enterprise 2.0: New Collaborative Tools for Your Organization’s Toughest Challenges, which will be published this fall by Harvard Business School Press.

The book covers and expands on many topics discussed in this blog. It’s about how the Web’s novel emergent social software platforms can benefit companies, and how smart organizations and their leaders are making effective use of them.

I’ll post more information about and excerpts from the book as the publication date draws nearer. I’m also working to ensure that it’s available on the Kindle, and perhaps on other ebook readers as well. For now you can read a bit about it on the Amazon pre-order page. And while you’re there, why not just go ahead and order copies for yourself and all your strong, weak, and potential ties? 

If you have any questions about the book, please feel free to ask them here via a comment.