The case on Google’s internal corporate prediction market that I wrote with Peter Coles and Karim Lakhani is now available for wide distribution (a teaching note for this case is also available to faculty). The case’s introduction explains what prediction markets are, and why they might be interesting to business leaders:
"Prediction markets were very much like stock markets. They contained securities, each of which had a price. People used the market to trade with one another by buying and selling these securities. Because traders had differing beliefs about what the securities were worth, and because events occurred over time that altered these beliefs, the prices of securities varied over time.
In a stock market like the New York Stock Exchange the securities being traded were shares in companies, the prices of which reflected beliefs about the value of the companies. In a prediction market, in contrast, the securities being traded were related to future events such as an American presidential election. In this case, the market could be designed so that each security was linked to a candidate, and its price was the same as the estimated probability that the candidate would win, according to the market’s traders.
Prediction markets on the Internet had proved to be remarkably accurate at predicting the results of political elections and other events, and the Googlers had wanted to see if they could also be productively used within companies to forecast events of interest such as the launch date of a product or whether a competitor would take a specific action. The experiences of the previous seven quarters had shown that Google Prediction Markets (GPM) were in fact quite good at predicting such events. Googlers put none of their own money at risk when they traded within GPM; instead, they bought and sold securities within GPM using “Goobles,” an artificial currency."
I’m going to teach this case on Tuesday in my MBA course, and am really looking forward to it. It’s one of my favorite classes of the semester, and will be made even better by the fact that Bo Cowgill, the Googler who initiated prediction markets within the company, will come to Boston to share his insights with my class (and also with Tom Malone‘s at MIT).
Cowgill has written a paper with Justin Wolfers and Eric Zitzewitz analyzing data from Google’s markets, and Wolfers and Zitzewitz also wrote a more general overview of prediction markets. The Wikipedia article on the topic is another good resource. Prediction markets on the Web include the Iowa Electronic Markets, InTrade, NewsFutures, and the Hollywood Stock Exchange.
Our case concentrates on two issues: how to encourage more trades and more liquidity within a corporate prediction market like Google’s, and how business leaders can and should use the information provided by the market.
After writing the case, teaching it a few times, and spending some time understanding the mechanics and utility of prediction markets, I share the puzzlement articulated by James Surowiecki in his book The Wisdom of Crowds:
". . . the most mystifying thing about [prediction] markets is how little interest corporate America has shown in them. Corporate strategy is all about collecting information from many different sources, evaluating the probabilities of potential outcomes, and making decisions in the face of an uncertain future. These are tasks for which [prediction] markets are tailor-made. Yet companies have remained, for the most part, indifferent to this source of potentially excellent information, and have been surprisingly unwilling to improve their decision making by tapping into the collective wisdom of their employees."
Why is this? It’s not because the technology is hard to acquire: Inkling Markets, Xpree, and Consensus Point, among others, will happily provide a company with Web-based prediction market software. So what is the real stumbling block? Is it that companies don’t really want the most accurate information about future events to come out and be widely known?
Leave a comment and let us know what you think, or what your experience has been. I’ll post more on this topic after our class on Tuesday.
{ 16 comments… read them below or add one }
I have come to conclusion that availability and quality of management tools greatly exceed ability of corporate managers to use them productively. Many corporate executives, I have worked with, are much more comfortable with conceptual reasoning and analysis of historic data, than use quantitative methods for predicting future outcome of their decisions. Perhaps it has something to do with a fear of accountability, but they seem to treat forecasting as a matter of “art” rather than “science”.
Hi Professor – I have hit many roadblocks at my company, which actually you and I spoke about a few months ago. Here’s an overview of what I have encountered, in no particular order…
1) The myriad usage scenarios within an enterprise actually makes it harder to get the investment necessary to try prediction markets. Why? There is reluctance to take the risk yourself when there are benefits for the enterprise as a whole. In other words, if it has that much potential, someone higher up the food chain should fund it. This isn’t merely a funding issue, however. It’s also a measurement issue. Very few people are given a target metric for anything that doesn’t fit neatly into the balance sheet, with managing quarterly expenses at (or near) the top of the list. Naturally this leads to the opinion that if it’s not being measured, it’s not a priority. We acknowledge this needs to change, but gaining agreement on what those new metrics should be is a big hurdle unto itself.
My point here is that, at least initially, it likely would be easier to find a sponsor if the business case was more targeted, which is what I’m trying to do now.
2) Despite all the press regarding how accurate prediction markets are, there is a great deal of skepticism regarding their accuracy for our company, or where specifically they would be most accurate. If we pilot the technology in the wrong area, or otherwise don’t set up the markets properly, the whole thing could get the hook quickly.
3) The ROI is not immediate, nor is it necessarily quantifiable as part of the initial business case. We can cite how terrible most companies are at forecasting or decision-making in various areas, but we really need our own ‘forecasted vs. actuals’ historical data as a starting point. Gathering that data from various business units is quite a chore to say the least. We also need to run traditional methodologies and prediction markets concurrently to see how the results match up.
4) Despite the ROI extending beyond forecast accuracy to areas such as social networking, knowledge management and information flow, these benefits are less tangible. I believe this will change over time, but most of the companies that have even deployed social software still are not doing real social network analysis…they’re just trying to increase collaboration and team productivity. Their ability to gain insight from the underlying data is far down the road, when they’ll wake up one morning and realize that Enterprise 2.0 and Business Intelligence have converged (with ROI more readily apparent).
5) Social software adoption is already an issue, so there is concern regarding the participation level necessary to make the markets as accurate as possible. Aside from the usual inhibitors to the adoption of new tools, the concepts of online reputation, digital identity, virtual currency and overall openness are novel to most employees. There is reluctance to contribute in any form, via any medium, without maintaining anonymity.
6) Assuming the CIO and/or IT Dept makes all funding decisions for internal technology deployments, prediction markets would have to show greater business value than the approved/funded projects currently in plan. So this gets back to the ROI issue (and points to the potential scenario of using prediction markets as part of that same IT funding/planning process).
7) Traditional methodologies for forecasting and decision-making are well within the comfort zone of most executives. Even when they are willing to try other methods, those experiments are very controlled and merely supplement the formal methodology. It is believed that instinct plays factor as well, but not just anyone’s instinct (that’s crazy talk).
Prediction market results which conflict with or undermine project status reporting, for example, would potentially give managers and executives less decision-making power and less credibility. If managers administered the market themselves, and made the necessary adjustments early enough, I think this concern would be mitigated. Today, however, self-preservation is a big factor, often at the expense of the larger organization…and credibility could go out the window anyway.
Also, there is limited quantitative data on prediction market accuracy for longer term events. So any businesses which try to produce mid-term to long-term forecasts would take a higher risk, have to compensate for market liquidity issues, etc. Still a worthwhile exercise, however, in my opinion.
9) If the enterprise happens to be an E2.0 software vendor (“I have this ‘friend’…”) then there is the build vs. buy vs. partner issue, strategic fit, line-of-business ownership, go-to-market strategy, etc.
Despite these and other obstacles (like that pesky day job), I’m continuing to promote prediction markets all over the corporation, and have built up quite a bit of interest. I’m not stopping until I find an executive sponsor or receive a rational explanation as to why we will not pursue the topic further.
Frankly, I don’t think a rational explanation exists. Then again, corporate decisions aren’t always rational, which brings us back full circle.
Happy to discuss further if you’d like…
BP
(Go Sox!)
I think the wisdom of crowds concept is great, and I think prediction markets make sense for external events, but I wonder about conflicts of interest on internal event betting. If you’re betting against a project being on time (or betting for it to come in at a later date), don’t you have an incentive to “slow roll” any requests that come your way from that project? Or worse if you’re actually a member of that project?
And think of the political games that could get played if someone wanted to make another team look like it was failing by betting heavily against their metrics? Panicked execs saying “Well, the market says your project is going to be a train wreck…”
Professional sports has dealt with this problem for a very long time. It’s obviously a big no-no to have players betting on – or especially against – their own team.
Not that I have any deep rooted concerns on this topic, but I thought I might play devil’s advocate.
“Is it that companies donÂ’t really want the most accurate information about future events to come out and be widely known?”
Why I think companies may be right to be cool towards predictive markets:
1st: An information market at a company is probably not efficient, (I know that the article claims that the Google market is “reasonable efficient”, but so was Thailand’s currency market before George Soros) and at best, the predictive market at a company is the weak form of market efficiency where the current price is reflective of the historical series of events. Way too much room for manipulation in a market like this.
2nd: Time value of money: The instances when having an accurate prediction is worthwhile is when the stock value is at “$1″ not at “$80″. Sure, I can very accurately predict when a movie ends when the credits roll, but I am a poor predictor when I am watching the trailers. The problem is that businesses need accurate predictions during the trailers, not during the credits.
3: Relationship of the outcome significance to the prediction market performance:
It is one thing to predict the if a new product will meet sales goals in yr 1, it is quite another to predict the success of breakthrough products like the ipod. Why my concern? I look to physics. The biggest shakeup in physics over the last 300 years was the theory of relativity. Problem, when Einstein developed the theory, scientists were almost universal in predicting the theory to be wrong/incorrect as compared to Newtonian physics. Well, they were wrong. Granted, this point doesn’t dismiss predictive markets, but it does make me wonder how effective they really will be?
Having mention the above, if predictive markets do work, I certainly can foresee a time when companies, as a work performance measure, provide their employees with equal amounts of Goobles and have their employee’s bet away. The most successful will be evaluated and cultured for positions in strategic thinking roles within the firm. Interesting
Oh, by the way, both Sox sux. Go Cubs!
As noted in a recent article in Inside Knowledge Magazine, the number of companies that have implemented an internal prediction market is modest (lower bound was in the 30s as of 2006, although growing rapidly). Feedback from companies utilizing prediction markets has identified critical deployment considerations, including trader knowledge, question/security design, and trader participation however the limited trials and low adoption rate of prediction markets indicates that resistance to their use pre-dates these deployment considerations.
Tom Davenport, a Professor as Babson College, has identified traditional hierarchical organizations as a major obstacle to the use of prediction markets (as well as trader participation once deployed). At BitInsight (www.bitinsight.com), a consulting firm utilizing prediction markets as part of a decision support service, our experience would concur with this organizational resistance. The resistance ranges from disbelief in the process, concerns about data leakage, concerns about the impact on current business processes, and concerns that the process will “undermine management” (to use the words of Microsoft in their presentation on their prediction market, PredictionPoint at the recent Conference on Corporate Applications of Prediction/Information Markets).
By definition prediction markets are highly visible and if not carefully managed can be controversial, note the furor over DARPA’s terrorism futures market. This visibility adds to the personal risk from failure and precludes “skunk work” executions in most cases. The (ideally for many questions) cross organizational nature of the trading pool also adds organizational obstacles to market introduction.
Prediction markets are the data mining tool to access the unstructured data stored in the enterpriseÂ’s distributed human capital (broadly defined). It provides another source of data to be considered and leveraged; it does not, per se, make or even recommend any decisions to the corporation. Defining the tool as such may make it more palatable to senior management whose buy-in and support are absolutely critical to successful trial and on-going incorporation of this technique within the enterpriseÂ’s business processes.
Several corporate executives, I have worked with, are much more comfortable with conceptual reasoning and analysis of historic data, than use quantitative methods for predicting future result of their decisions
Catching up on my blog reading…
Based on my conversations, there are a number of obstacles at present re: prediction markets in corps:
- most execs I talk with aren’t familiar with prediction markets. Whereas everybody’s heard of social networks, blogs, and wikis (even though a lot of people still aren’t quite sure what a wiki is).
- prediction markets are a more difficult concept to understand (vs social networks, blogs, wikis). Thus significant education is required for execs and end-users. More education = bigger time suck thus lower ROI & opportunity cost.
- the prediction market biz process is more complex and more integrated into existing decisions/processes, than is the process required to implement/support social networks/blogs/wikis. What outcome will be forecasted; what end-users will participate; what’s the timing/schedule/due date; who will report the results and to what audiences; how will results be integrated into existing biz processes; etc. Again = more work, greater time sink, greater risk, lower ROI.
- there aren’t yet enough corporate examples of successful prediction market use. Google and Microsoft have shared their experiences publicly; but those two are not seen by most companies as a realistic example for their own organizations.
We’re definitely pre-chasm on prediction markets for corp use (whereas social networks are post chasm — much wider adoption).
And here’s the fundamental problem with prediction markets, IMO: the democratic, collective, participative nature of prediction markets comes into direct conflict with the hierarchical nature and existing communications patterns of most organizations.
Organizations are about the allocation of decision-making rights. Power is based upon who/what group has the authority/ability to make a decision. (Yes decisions can be subverted…but the official decision does matter in a very real way, even when it’s not implemented effectively.)
Also, leadership (formal or informal) in any organization is very much about communication and dialectics: what information is provided to whom, what context and meaning is assigned to that information by the speaker, which points are emphasized and which are de-emphasized, etc.
Well crafted communications are key to productive dialogue and decision-making. Poorly crafted communications can be a major obstacle and even derail issues permanently. There’s an enormous effort in organizations put into crafting effective communications. Yes, some of this is BS/spin….but some of it is legitimate communications work to communicate effectively between different constituencies/audiences.
Prediction markets pose a direct challenge to both of these (decision authority and well crafted communications).
Prediction markets second-guess the official management opinion/decision. And by engaging a wide group of people, the pred market sends a meta-message that everybody’s opinion is just as good as anybody else’s. It also implies that any answer that differs from the collective wisdom is inherently ‘wrong’ in some way.
Also, like all-hands meetings and employee surveys — prediction markets are a collective group experience. And that is different from the ongoing efforts of managers to balance out many complex issues between their organization and others, and tailoring their communications for those various contexts.
For example, there can be many reasons why a manager’s “official” forecast/prediction is modulated and differs from a group collective opinion. But those those nuances of social dynamics aren’t accounted for, with a prediction market.
That is PM’s strength — the direct unmodulated answer — but within an orgnanization/social dynamic, that’s also a problem.
Prediction markets aren’t quiet. They aren’t subtle. They are a blunt instrument — they produce a straightforward, simple answer to a straightforward question — with a big public splat.
It’s not about the specific answer per se. It’s the bigger social context: of collectively and publicly involving a wide group of people in an issue that previously involved a select group only; the expectation that employees now have, for engagement and responsiveness and dialogue, after being asked to give their opinion.
Look at the angst that employee surveys and all-hands meetings can cause for management — and that’s for mechanisms that have been in use for decades and for which there is a lot of knowledge about how to use them effectively. And managers still stress over them.
Anything that stirs up collective action among employees is worrisome to managers. So it’s no wonder that prediction markets, being a brand new democratic/collective mechanism, would cause a lot of anxiety and resistance among managers.
Prediction markets are not quiet, are not subtle but a blunt instrument.They produce a straightforward, simple answer to a straightforward question—with a big public splat.
Companies that are looking to use prediction markets in their business need to build a business case for doing so.In order to build that case, they need to determine what information can a market efficiently provide, and how can that improve their strategy and day-to-day operations?I like to address three main question areas that prediction markets can solve, and how you can start developing a business case based on the results of the improved forecasting.
The first, and perhaps most powerful, is project management.
Every project has various key milestones, and those on the critical path are even more visible to and closely watched by managers and executives.Most forecasting to date is largely self-reported by team leaders.They are largely meant to be as honest as possible in reporting project status.In reality, this is highly dependent on the company’s culture, and in far too many businesses, project status is a commonly-known lie.(Unfortunately, this also means that it can be a politically dangerous kind of market to implement, since the project managers that tell these lies usually don’t want to be exposed!)
The business case for a prediction market in project management should be calculated based on the effects of slips in a project’s schedule.For projects or products where sales are front-loaded (such as films, major software releases, ground-breaking new drugs, and the like) a projects slip into the next fiscal quarter or next fiscal year could cause a serious impact in potential earnings.If that information was available weeks or months earlier, how would that affect sales, reputation in the marketplace, and other factors important in your industry?How much does it cost to plan for a major product roll-out in January, only to have to cancel it all and do it all over again in April?These are the questions that can build a business case for a project management prediction market.
The second type of market quantifies your industry.
In many industries, this means defining the cost of goods needing to be purchased, the quantity of goods or services to be sold, or the price potential of the goods or services to be sold.In some industries, this could mean quantifying the user growth of a given product.(Such as Google’s prediction markets, How many users of [Google product] will we have at the end of Q3?] This is the classic case written about by HP’s research center, when they asked their traders to predict sales figures for various products for the next quarter.It turned out they were measurably better than the forecasting system they had been using.
Quantifying a business case here is generally more computational, and therefore more straightforward.What are your costs associated with inventory of products that you can’t sell because you thought there was more demand?How much less revenue do you earn because you have to discount products to move them out the door?Alternately, what are your costs associated with paying extra to ramp up capacity because you didn’t realise the demand was as big as it was?How many lost sales did you experience because of empty shelves?These numbers are concrete and can hit your company’s bottom line.They can sometimes be the easiest way to prove your business case.
The third type of prediction market I want to discuss quantifies risk.
I think the problem we have with prediction markets is that they are more like the options market than the stock market. I think the problem with getting the needed liquidity is found with the fact that it’s a probability game which generally means the house wins. I do think that the social prediction markets has given way to new opportunities through niche opportunities. People feel they can invest in something they understand or have above average changes of knowing the outcome where the average person likely feels they have no idea what the stock market will do on any given day.
Just writing about Spigit, which has a nice prediction market capability to harvest employee intelligence and forecast likely successes for enterprise initiatives. Check it out: http://www.spigit.com/products/predmarket.html (Just a cool product, I'm not affiliated.)
That Sounds interesting, I agree with you.Please keep at your good work, I would come back often.*
Ways to make money
The industry is now based on prediction marketing.
Not knowing anything more about prediction markets than what I read here, I can say investing in prediction markets sound a whole heck of a lot like ordinary sports betting/gambling. It sounds like a person may be just as safe (maybe more) betting on professional sports spreads than these “prediction markets,” because it would seem a heck of a lot easier for a company to manipulate when and how it would release a product–a company, I would think, has a great deal of control over the release of its products, for example, whereas a boxer may be more prepared than his opponant and winning a fight but goes down from a lucky shot. At least in professional sports nowadays, this kind of “fixing” much seems less likely to happen. Random thought, but I really am confused…
Not knowing anything more about prediction markets than what I read here, I can say investing in prediction markets sound a whole heck of a lot like ordinary sports betting/gambling. It sounds like a person may be just as safe (maybe more) betting on professional sports spreads than these “prediction markets,” because it would seem a heck of a lot easier for a company to manipulate when and how it would release a product–a company, I would think, has a great deal of control over the release of its products, for example, whereas a boxer may be more prepared than his opponant and winning a fight but goes down from a lucky shot. At least in professional sports nowadays, this kind of “fixing” much seems less likely to happen. Random thought, but I really am confused…