The Good and Bad Kinds of Crowds

This past week I rolled out a couple Enterprise 2.0-ish experiments in my MBA Class Managing in the Information Age. First, I attempted to use crowd wisdom to outsmart my students. Second, I let them form their own online crowd during a single class.

The previous week I had thrown down a challenge: any students that outpicked me in the men’s NCAA college basketball tournament (aka “March Madness”) would win freedom from cold calls for the rest of the semester. I use an Excel-based random cold call generator in class and my students absolutely hate it, so they had ample incentive to fill out an entry within the ESPN group I set up.

I told my students that there was plenty of help and advice about the tournament available online, as well as many, many freely available brackets completed by different flavors of expert. I also told of them none of this would do them much good, though, ’cause I was such an ardent college hoops fan that I would surely outpick them.

This was a baldfaced lie.  I haven’t watched a basketball game in years, and have no idea who’s any good these days. I’m certainly not better informed than the tournament organizers, who seed the 64 teams based on their expected performance. So for me, a smart strategy is to just pick the team seeded higher in each game, thereby taking advantage of all the intelligence baked into the tournament.

I also think this is a pretty smart strategy no matter how well informed you are. I wonder how many ‘experts’ predict the tournament’s results better each year than the seeds alone do. I bet it’s not many. I bet even fewer experts would have a track record over many years of outpredicting the seeds.

But I also thought it would be possible to do better than just picking the seeds by tapping into crowd wisdom —  seeing, in other words, if a crowd thought that the tournament organizers got it wrong in any cases. And my preferred way to do this is to look at prediction markets.

NewsFutures set up prediction markets for each of the first round games in the tournament. When I checked them shortly before the deadline for submitting a bracket to ESPN, I saw that they were predicting 3 upsets: #10 Maryland over #7 California, #9 Tennessee over #8 Oklahoma St., and #10 USC over #7 Boston College. In all other cases, the collective prediction of the NewsFutures traders was that the higher-seeded team (the one with the lower number) was more than 50% likely to win in the tournament’s first round.

So in all cases except the three listed above, I picked the higher seed. With one exception: #12 Arizona had, according to the market a 48% chance of beating #5 Utah, and I couldn’t pass up the chance to correctly call a big upset like that. So I went with Arizona.

In all rounds after the first I simply picked the higher seed. I could have set up my challenge to students so that we had to pick one round at a time. Setting it up this way would have let me use the markets established for later rounds, but I’m pretty sure my students would have caught on to this strategy before too long, and we would have all converged.  Maybe next year…

I don’t know any of the traders in the NewsFutures markets, and don’t know exactly how to interpret the data they provide on trading volume – the number of contracts held for each game (beyond knowing that more trading is better). I just have a lot of faith in prediction markets‘ ability to forecast real-world events, and wanted to put that faith to the test in a visible way.

None of my students guessed correctly that I had used prediction markets to make my picks. I asked them to describe their strategies; they relied on their knowledge, seeds, odds from Las Vegas, and hometown affiliations (a very bad strategy). None of them said that they’d used the markets as I had. The most intriguing strategy I heard was to use the results of the video game company EA Sports‘ simulated tournament to make picks.

The first round of the tournament is now over, and the markets correctly predicted the upset victories of Maryland and USC. In addition, #12 seed Arizona did win, so the markets did an excellent job of highlighting this possibility. They were wrong about Tennessee over Oklahoma St. There were seven additional first round upsets that the markets did not correctly predict.

Overall, my strategy of relying on the markets left me in significantly better shape —  two victories worth –  than would have been the case if I’d relied only on seeds.  I’m currently tied for fourth place among the 39 brackets submitted as part of my class, and in pretty good shape for the rest of the tournament; no one has more possible points remaining than I do. The bracket using results from the EA sports simulated tournament is tied with me at present, but has fewer possible points remaining.

My second experiment with emergent social software platforms was much simpler: I just told all my students to get Twitter accounts, then allowed them to tweet freely during class on Friday, March 20. I also told them to send one tweet using the hashtag #HBSMIA2009.

Twitter expert and celebrity and Pistachio Consulting founder Laura Fitton (@pistachio) is coming to class later this semester, will encourage students to tweet during class, and will also (I believe) display these their tweets on a screen throughout class. I found myself unwilling to take that last step, but did want to get my students comfortable with the practice of live tweeting, and also wanted to see what it felt like to teach class while that was going on.

So I told them that class on the 20th would be an exception to HBS’s standard ‘screens down’ policy (i.e. no use of digital devices during class), and that they could tweet using whatever device they preferred.

I’ll ask my students what they thought about the experience, but I thought it was miserable. Class discussion limped along at well below its normal levels of engagement, interest, and insight. I thought it was due to my bad class plan, a comparatively weak case, and/or the fact that the 20th was the last day before spring break.

Any or all of these could have been part of the explanation, but I’m quite sure that another part was the tweeting that went on. When I reviewed students’ tweets after class, I found that a lot of them remarked on how difficult it was to pay attention to what was going on in the room and on their screens. And it was very clear that the screens won.

Speaking to an audience that’s tweeting away is now a fact of life at most technology conferences (as clearly evidenced by this year’s South by Southwest). Laura says she likes it, and I’m eager to learn from her why this is and how I can turn live tweeting to my advantage when speaking. So far it feels to me like trying to talk to people who all have TVs in front of them. I realize that live tweeting might be beneficial to some constituencies (like the tweeters’ followers), but it feels to me like a pure negative for speakers. We’re now competing for attention with a very compelling interactive activity.

I know this is the new reality of public speaking, and I know I’ve got to get good at it, but I’m not sure how I’m ever going to come to like it. And I know my classrooms are going to remain unwired. I want my students to concentrate on the discussion taking place in meatspace, not the ones in cyberspace.  I want to be clear: I like twitter a lot and use it a fair bit myself (follow me at @amcafee if you like), but I don’t like it in a classroom when a live discussion is (supposed to be) taking place.

Two questions to wind up this post. First, how could I have made better use of crowd wisdom and other available information to make better March Madness picks? And do you have any tips on how to be a good Twitter-assisted public speaker?  Leave a comment, please, and let us know.