Under the auspices of the Management Lab I’m getting together later this week with a group of very sharp people who think about companies and how to change and improve them. We have a pretty specific agenda: to come up with the set of experiments we want to run on corporate guinea pigs. The MLab’s mission is to provide such guinea pigs by putting us in contact with companies whose leaders have both the curiosity required to allow experiments and the clout required to get them up and running. So our sessions this week should be great fun; I’m looking forward to brainstorming with my colleagues about how to tweak organizations and watch what happens.
I’ll be concentrating on technology (duh) and working to design IT-enabled interventions. I’ll be particularly eager to design them around Enterprise 2.0 tools and approaches, and to address questions like:
- Does a salesforce that uses Twitter outperform one that doesn’t?
- What happens when you start measuring contributions to ESSPs ? Does hit accelerate or kill participation?
- If the goal is better output from the R&D department, should the Enterprise 2.0 environment be limited to the department or open to all ?
- Does a prediction market yield consistently more accurate forecasts of future sales than the forecasting group does?
- Does it help when the CEO starts to blog? How much?
All of these experiments involve comparisons (before vs. after, group A vs. group B, method X vs. method Y, etc.) and have relatively ‘hard’ outcomes of interest like productivity, accuracy, and output. This is not an accident; it’s instead a reflection of my training and interests. I teach in an Operations Management department, and am fundamentally interested in how to build more and better widgets. I think the changes in social structure that accompany the introduction of new technologies are also interesting, but less so. For whatever reason my research interests coincide with the concerns of the stereotypical busy, pragmatic, skeptical line executive: is this stuff going to let us get more widgets out the door, win happy customers, and make money?
Given these interests, what experiments should I advocate? How would you design research to determine how much of a positive difference Enterprise 2.0 can make, or research to understand effective adoption strategies? If you had a pristine greenfield site, what experiment(s) would you conduct? This is our chance to play mad scientist; how should we best take advantage of it? Leave a comment, please, and let’s get the brainstorming started.
In addition to the MLab itself, I want to thank Tim Brown and IDEO for hosting our session and Gary Hamel for organizing it. Gary’s enthusiasm for actually changing the way companies are run is palpable and contagious.
{ 12 comments… read them below or add one }
I’d be interested to follow this and see the results. After a year of integrating 2.0 applications improvements are slowly beginning to occur. http://twitter.com/timeshareguru
Thank you for your post and the opportunity to comment here.
In my opinion, the first thing that springs to my mind after reading this post is that since all the intended experiments involve quantitative outcomes, would it not be more useful to develop a common “unit” or “rating” system to effectively measure the outcomes?
In other words, a system (perhaps akin to the Enterprise 2.0 ratings as proposed earlier) could better measure such outcomes. What I am thinking is that this measurement unit system shouldn’t just incorporate Enterprise 2.0 variables (like authoring, editing etc.) but also physical tangible (output or productivity etc.) and intangible (work done? knowledge contributed?) efforts.
This should be made applicable to both the groups (the control group as well as the experimental group)However, care should be taken to see that the unit also measures the control group properly and is not skewed towards the experimental group.
Perhaps this should be the first step of the research design process (maybe even a mini organizational experiment) and then after obtaining a generally applicable unit (since no measurement unit is 100% accurate) should one proceed.
Thank You.
Q: What happens when you start measuring contributions to ESSPs?
Comment: Measurement is important to demonstrate business value beyond the level of anecdotes, but cannot be effectively applied as performance benchmarks for individuals or teams. We measure things in order to manage them. The challenge is to carefully avoid two unintended consequences of focusing on the numbers instead of the objective. First, we must be careful that we don’t create incentives for “high volume / low quality” participation (like the infamous call center reps that hung up on every third customer immediately to reduce call duration). The unique benefits of ESSPs depend on the voluntary, self-selecting, and internally motivated participation of members within a healthy culture of collaboration. Metrics are important tools to validate improvements to the usability and usefulness of our systems. But if they become performance measures, we risk killing the goose that lays the golden egg. The second problem is that a simple “more is good” yardstick misses the fact that not all types of participation produce equivalent benefits. We should gather data to test hypotheses about ESSPs: does one type of activity displace another or are the two positively correlated? What kinds of behavior or impact on business performance do we expect to see? etc.
Q: If the goal is better output from the R&D department, should the Enterprise 2.0 environment be limited to the department or open to all ?
Comment: R&D seems to thrive with a hybrid model. In general, it’s good to work in the open, and to proactively solicit feedback – especially from people who can provide a business perspective. However, collaboration within small groups out of the “public eye” is also needed, especially in early stages. Good R&D practices eliminate a lot of ideas early, and that process might confuse outsiders, or – worse yet – look like organizational failure. Maybe this doesn’t just apply to R&D – anybody transitioning to a more transparent working style can grow more comfortable by working within smaller, more confidential environments in addition to the “wide open” ones.
Just my 2 cents.
Hi Andrew,
maybe I could be wrong but my experience teaches me that a big part of the benefits comes exactly from the social, political and cultural structure in which enterprise 2.0 technology are introduced. This is especially true for tools positioned at the center of the bullseye where interaction and openness are key. A peer to peer rating system or an enterprise microblogging tool produce very different outcomes inside different business and political contexts.
I completely understand your focus, but I believe that a sort of enterprise 2.0 readiness should be somewhat considered to produce accurate and meaningful experiments
Wow, I’m hesitant to think about this from a true greenfield site because I think it’s unrealistic. And it is certainly unrealistic that you would have a control group and variable group that are both greenfield. One thing you could do would be around CEO blogging. My question is: are the effects (presumably positive) of CEO blogging due to the content that the CEO comes up with or the fact that it is a CEO? One thing to do would have one company’s CEO actually blog and another company’s CEO post blog entries written by someone else…say a mid-level exec that knows enough to make it sound like the CEO. Multiple iterations might be required to account for one particularly good or bad blogger, either in the CEO or faux CEO column. If you wanted to get complicated, you could have the CEO post blog entries under someone else’s name. Make sense?
Wow, I’m hesitant to think about this from a true greenfield site because I think it’s unrealistic. And it is certainly unrealistic that you would have a control group and variable group that are both greenfield. One thing you could do would be around CEO blogging. My question is: are the effects (presumably positive) of CEO blogging due to the content that the CEO comes up with or the fact that it is a CEO? One thing to do would have one company’s CEO actually blog and another company’s CEO post blog entries written by someone else…say a mid-level exec that knows enough to make it sound like the CEO. Multiple iterations might be required to account for one particularly good or bad blogger, either in the CEO or faux CEO column. If you wanted to get complicated, you could have the CEO post blog entries under someone else’s name. Make sense?
“If the goal is better output from the R&D department, should the Enterprise 2.0 environment be limited to the department or open to all ? ”
A you wrote in your original post “we don’t really know” about the effects of ‘walled gardens’. The value of knowing is more efficient use of all of our resources including time in these efforts.
Hi Andrew. I think this posts reflects a bit of my recent experience with “2.0″. Inside companies it always comes down to ROI in the mind of the executive. Which is good.
My fear is that the questions you pose may be too high level. You may find yourself falling into the knowledge management trap of “if everyone searches for information 5x a day, and it takes 5 minutes now, but with our new shiny system its 2 minutes, that 3 minutes saved x 5x a day x # employees x Avg Salary = a Gazillion in savings.”
The only successes I have seen eventually come down to a few key business metrics that are ALREADY a point of concern or care for management. These important drivers just need to be unearthed, dusted off, and then connected to a very targeted use of new technology. For sales I might take a look at a new RFP database with tagging, rating, etc instead of asking about twitter usage.
I think all this is highlighted in the current economic times where novel ideas inside organizations will be under evermore scrutiny. Unless the new approach is tied to what is on management’s mind AND their impact can be proven directly, the struggle will continue.
Take care,
Paul
Not that you didn’t have it before, but you now have my full attention.
Nothing will help advance the adoption of these tools more quickly than to measure their impact on business performance. I’m one of the creators of FlowThink, a collaborative platform that incorporates E2.0 tools into a single work environment for all types of organizations. I’m also ‘on the front lines’ selling it into companies every single day. My two main challenges are:
1. How will a collaborative work environment fit into each specific work environment? This requires a collaborative effort with each individual client and won’t be solved with a brochure. Exploring this question is the fun part of the job.
2. How do these tools impact productivity? Team work? Efficiency? And how are others measuring success? This can be answered with some solid research that produces useful literature.
All the best. I can’t wait to hear more.
Andy, great setup for a couple of questions I’ve been considering for a while in my organization; one functional and one technical.
1. Functional: What are the sustainable, measurable benefits at the operational level of an organization in a purely competitive market of incorporating an Enterprise 2.0 approach to enterprise planning over an Enterprise Architecture (EA) only approach? By Enterprise Architecture only approach, assume a collaborative intelligence only engagement model; whereas the 2.0 approach implies a complementary collective intelligence engagement model. The alignment of planning and operating models are unspecified. The experiment could be parameterized per operating model. For more on the paradigm see the Jeanne Ross book on EA as Strategy.
2. Technical: Contrasting Enterprise 2.0 with a model driven and service oriented technical architecture, which provides the lowest total cost of ownership over a five year period ? By model driven and service oriented technical architecture I presume a functional Enterprise Architecture approach defined in 1 above along with a top down refinement from design-time BPM artifacts to run-time execution whose implementation requires architects as intermediaries throughout from process specification to service implementation.
BTW – I’m an architect who does formal modeling with a background in complexity theory. I’m currently engaged in advising members of my organization in both folksonomy and formal ontology, so I am very much interested in your findings.
Rick
You bring up some excellent points! I’d be interested to know about how much a CEO blog helps.
Andrew — I’d be happy to volunteer HubSpot if you are looking for guinea pigs. We only have 50 people, so we might be too small. Brian.