Should Knowledge Workers have E2.0 Ratings, Part 2

by Andrew McAfee on October 22, 2008

I received a wealth of insightful comments in response to my earlier post on Enterprise 2.0 ratings for knowledge workers. I thought I’d use some of them to continue discussing the topic, starting with the very basic question of whether measuring E2.0 participation is a good idea at all. A few commenters felt that it was not, and that the very act of measurement would pollute or drain enthusiasm from exactly the activities E2.0 enthusiasts are trying to encourage.

From Kevin Gamble:  I have no doubt that some organizations will attempt to quantify participation… As soon as you try to quantify it, however, you will kill the goose… This would be something akin to paying people to contribute to a KM system. You’ll get nothing but garbage.

From Bryan Labutta: As far as a rating system is concerned, I would be hesitant to implement one around Enterprise 2.0 because I would not want to stifle an individual’s willingness to contribute. As I mentioned above, I feel like the biggest benefits to Enterprise 2.0 should come when employees realize the benefits on their own and contribute at will. If a regular contributor sees that their personal feedback rating is very low will that reduce the amount of time they spend editing wikis and writing blog posts?

From Fenton Travers: E2.0 is about embracing the good, and not being afraid of the ‘bad’.  A rating, that your boss is going to look at and beat you up about is pretty pointless management activity.  Please GET IT GANG, E2.0 is not for the bosses it’s for you!

From Lim Boon Chuan: IMHO no, it is hard to quantify tradition work and services, till no there isn’t really a good measure of the current not to mention Enterprise 2.0 which is much more abstract.  Measurement can help gauge the level of effectiveness of an enterprise.  I qualify it by adding “accurate measurement”.  To date I do not see any accurate barometers or quantitative tools to measure Enterprise 2.0 performances.  Inaccurate measurements are worse than accurate measurements as it will bring uncertainty, frustrations and distrust which will work against the organization… It is certainly useful to be able to quantify some aspects of Enterprise 2.0.  But until we do have sufficiently accurate tools, lets not even try it.

From Samuel: But doesn’t KM research show KPI’s hurt knowledge sharing? Would measuring E2.0 contribution do the same? Furthermore there have been some interesting experiments trying to measure social media ROI. But it’s still hard to do this objectively. Will the comparison between my e2.0 contribution and that of my colleague be fair? Can’t we ‘just’ ask for stories and try to quantify them? Ask employees to tell managers how the tools helped them or others become more productive.

I imagine, though, that making E2.0 measurements and ratings visible, might instead do a couple positive things:

* Encourage friendly competition. Lots of people are fiercely proud of their PageRanks, TopCoder ratings, number of Wikipedia edits, etc. and work to keep them high only to preserve bragging rights. Slashdot, in fact, had to replace their numeric karma scores with text labels because people paid too much attention to the scores, treating them "like some sort of video game."
* Make people strive to improve their scores. I know I’ve been inordinately proud of my Technorati ranking, even though it has no direct bearing on my professional success. The desire to maintain it has definitely driven me to keep blogging regularly.

And I don’t really see how measuring an activity and publicizing the results will massivley discourage the activity in question. I imagine that some people will likely be turned off by the measuring and stop contributing, but are these people the majority or the minority of the population at large? Or of members of an online community? Of coworkers within an enterprise? My intuition and (admittedly not vast) experience tell me otherwise. They tell me that the simple act of publicly measuring E2.0 contributions will increase participation, not decrease it. A couple commenters seemed to share this view:

From Atul Rai:  I do quite agree with you that participation in social computing cannot be reduced to one number, but as you said, this could be treated as a set of numbers … some folks would be good at edits, others at authoring … In addition, this is making the entire idea of social computing what it should be … fun!

From (my former student) Alex Bain: I think a scoreboard can be extremely motivational. I remember the Cambrian House founder mentioning in your class that the virtual currency they created for their community lead to a surge in contribution…  I’ve also seen a scoring system work within a company. I know the designers that work at Zurb, and they boil down their contribution to their company’s blog to a single number, and keep track of who’s winning: http://www.zurb.com/article/88/team-motivation-for-us-its-just-a-game [they say this has lead to both more and better work]

Later posts will consider what will happen if an organization moves beyond simply measuring contribution and takes more active steps to encourage it, such as putting direct incentives in place. I wanted to start the discussion, though, by positing that public measurement of individuals’ E2.0 activities will, absent any direct incentives, encourage and increase participation and contributions, rather than decreasing them. Do you agree?  What else do you think?  Leave a comment, please, and let us know.

  • http://byronmiller.typepad.com Ron Miller

    Hi Andrew:
    Love your blog, but I don’t see anything in this second post that would convince me that a rating system is a good idea. Maybe it’s because I’m distinctly uncompetitive, but I agree with the folks who left comments on your earlier posts, Enterprise 2.0 is not about competition, friendly or vicious, it’s about getting people to share information in a spirit of goodwill.

    If you set users against each other, to me, that is completely contrary to the spirit of Web 2.0 knoweledge sharing and collaboration. It may give you chest pounding privileges to say you have a higher rating than another fellow-employee, but it doesn’t necessarily mean that the other employees contributions are any less valuable, only that people have failed to discover him or her.

    I think it’s ultimately a misguided idea and would end up stifling the very qualities you are trying to encourage.

    Ron Miller
    By Ron Miller
    http://byronmiller.typepad.com

  • http://www.scottquick.com Scott Quick

    What seems oddly missing from this discussion is that E2.0 is a means to an end.

    What seems to be emerging here is a system that serves its own purpose, rather than the business and its shareholders.

    Why do you think E2.0 systems are any different that other productivity tools?

    Why not align E2.0 with desired business outcomes – increased sales, greater market share, etc..?

    Heck, I can name more than a few E.20 companies that are financially struggling because they missed that boat.

  • http://ascendia.jp Steven Veltema

    At my company we have a fun oriented company SNS site (not a full fledged E2.0 site and not productivity oriented) based on a Mixi type service, which has participation points. Certain points for posting, commenting, introductions, friending, reading and what not. Each member is the ranked into gold, silver, etc. The site is manly just for fun so rankings aren’t particularly used for anything, and in general ignored. Some employees do like to compete for points and these competitive employees tend to keep the site lively with new contributions. I would say the key this point system, is that it is not used really for anything other than fun (which applies to the whole SNS site in general) so it doesn’t feel like you are being “measured” as a participant. I’m sure that if employees felt that they were being assessed based upon SNS participation, that participation would become stale and dry up rather quickly.

  • http://www.b-r-ent.com Vincent

    I’m sorry andrew but drive socialperformance by the Ego will be OK for a few months but when you pay peanuts you’ve got monkeys.

    So you should be able to give some good stories about what employees could gain with good inputs.

    You have to work with the HR Staff to do that and makes from the socialperformance concept a new way to drive a compagny

  • http://talentedapps.wordpress.com Meg Bear

    I think that a rating system is a good idea. I think it has a downside but the downside (as you suggest) is more about what you use it for. I think one thing to think about is those who don’t ever participate. I talk about that a bit here http://talentedapps.wordpress.com/2008/10/23/a-new-protected-class-the-digital-hermit/

  • http://www.hubspot.com Brian Halligan

    We are working on the measurement tools. The first step is twitter.grader.com. Check it out…

  • http://communityzenmaster.com/blogs/lliu Lawrence Liu (Telligent)

    I agree 110%, Andrew! Simply put, you cannot confidently improve what you cannot measure. You also cannot effectively monetize what you cannot measure. We will soon release 2.0 of our Harvest Reporting Server that provides “social analytics” for social media sites and online communities powered by Telligent Community Server. In fact, Harvest provides a radar chart (amongst many other charts) that’s very similar to the one you’ve envisioned at http://blog.hbs.edu/faculty/amcafee/index.php/faculty_amcafee_v3/should_knowledge_workers_have_enterprise_20_ratings.

    How serious and committed is Telligent about this particular area of social software? Well, we just hired a noted Research Sociologist as our Chief Social Scientist. More details soon!

    I would be happy to arrange a private demo of Harvest 2.0 for you. Please ping me anytime.

    Lawrence Liu
    Director of Platform Strategy
    Telligent

  • http://blog.k1v1n.com Kevin Gamble

    Thanks for the shoutout Andrew.

    It’s a good conversation. I’ve been thinking a lot about workstreaming technologies, and there are a lot of similarities to this discussion. When we can measure and capture everything we have to be very careful that they aren’t misused. E2.0 is ultimately about trust (trusted relationships). If you mess with that dynamic you are treading on some dangerous ground. I worry a lot about how workstreaming technologies could be misused in the wrong hands. Think of timeclocks only exponentionally more evil.

    I agree totally with Lawrence above. You get what you measure. An essential element of the E2.0 enterprise is looking at outcomes differently. Simply put, you should be looking at the value added and do your best to ignore all of the inputs. They don’t really matter.

    Kevin

  • Barbara McGill

    My company was trying hard to get online discussion adoption (rather than email). Early on, someone posted, by name, how often each person logged into the system and wrote something. The response to that posting was huge! People were very offended and felt coerced into using the system. Our experience is that you get a few early adopters who will shine in the metrics. But, unless people “fall” over the tools and can see business value, i.e. show me how this will help me get my job done, just showing metrics only motivates a few who are motivated by being at the top in lots of things.

  • Donna

    How else can an organization gauge employee response to Enterprise 2.0 than through measurement? If the process or product is significant, a nonresponse indicates resistance or apathy, neither of which are beneficial.

  • http://luisalberolasblog.blogspot.com/ Luis Alberola

    I think this is a very important conversation you just launched. How you measure the performance of a particular community through its E2.0 ratings has the potential to become what I call a Peer People Review. And I think a Peer People Review can be used by organization for some of its non hierarchical populations (experts for instance).

    We are working on the subject with several clients, you can follow us at http://www.boostzone.fr/.

  • Bryan Labutta

    Andrew, I appreciate your insight as usual and have given a good amount of thought to this post and how it relates to your initial post on the topic. I think I would be more receptive to an Enterprise 2.0 ranking system that worked on the basis of peer ratings and not based on raw contribution figures. This may be what Luis was mentioning in his comment.

    To me, rating someone based on the number of times they have edited a wiki page, written a blog post, saved a bookmark, or tweeted about their day lacks value. People could fill the Enterprise 2.0 systems with useless information just to achieve a high ranking. However, if users were rated/ranked according to feedback from their coworkers then the people providing true value to the enterprise start to shine through. It’s similar to what Amazon does with their reviewer rankings. One person may have written 100 reviews and another person only 20, but the second person may have a higher ranking because people found those 20 reviews more helpful than the 100 reviews.

    I may have misunderstood your radar chart in your first post, but I think that was the source of my negativity toward rankings. I would rather see people that produce valuable information be rewarded with high rankings from their peers (which, in turn, encourages them to produce more and others to produce better content) than see people ranked highly simply through having a higher volume of contributions than others. That is the type of “friendly competition” that I agree would be a great addition to Enterprise 2.0.

  • http://steveboese.squarespace.com Steve Boese

    Very interesting post and series of comments. I have been working in the past year with three distinct groups in the introduction of 2.0 tools, Grad students, a small faculty group, and an IT organization. I would have to say in my experience with these groups, the introduction of ‘scoring’ schemes probably would not improve the adoption rates (save possibly the students who are more inclined to ‘compete’ with each other). In the groups I have been working with, building a convincing use case, demonstrating the potential benefit, and ‘teaching’ how to use the tools has been the predominant focus.

  • http://atulrai1.blogspot.com Atul Rai

    Hi Andrew, thanks for the interesting post. Yes, i think we need some form of visibility into how e2.0 is shaping up. Moreover, if we look at peer-feedback, rather than a sort of performance appraisal type rating, we are actually looking at a way to build some form of measurement into this, which is in line with the entire idea of e2.0. After all, feedback itself is a form of knowledge sharing … think commenting on a post by your friend on facebook! In this way, i tend to agree with Scott on this.

  • Niko

    Andrew,

    thanks for the interesting blog. I love to read it.

    In case any extrinsic motivation (e.g. salary) is coupled with e2.0 approaches the participation of employees must be monitored and KPIs must be generated per employee. That is a no-brainer. The question is: does it make sense to couple extrinsic motivation to e2.0 participation?

    From my point of view we must start the evaluation with exploring the motivation of people to participate in wikis, discussion boards and other collaborative tools outside of e2.0 on the web. Why am I writing this comment here without any extrinsic motivation? Because I love to share information and discuss with competent people.

    Binding extrinsic motivation to e2.0 participation would motivate the employees who do not understand the culture and the benefits of e2.0. A lot of content would be put on the e2.0 network to show engagement. The amount of irrelevant data (rather than information) would rise.

    Those who support e2.0 without extrinsic information today would be disappointed from the e2.0 system which would become administration and visibility as an end in itself. They would stop feeding valuable information to the system.

    Binding extrinsic motivation to e2.0 participation indirectly would demotivate the good employees.

    Don’t do it!

  • http://www.zurb.com Bryan Zmijewski

    Andrew-
    You happened to mentioned my company (ZURB) as an example, so I thought I would give you some specific feedback from our experience.

    1. We’re a service firm and wanted to bring more visibility to metrics in the business. The blogging points are only one piece- real time billable hours, company newsletter sign-ups, twitter feeds, leads, rss feeds subscribers, comments, etc. are all part of of a daily view of what is happening in the business. Everyone in the business sees this information.

    2. We’ve balanced the “numbers” with real-time feed activity to help reduce “competition” fatigue. It’s about the interaction as much as the totals. Superstars get their fix while other employees can still participate.

    3. While the personal numbers are interesting, we want employees see the collective benefit of our efforts. I implemented a profit sharing plan at the same time, so I think the metrics helped everyone see how to create value.

    4. We’ve had 5 consecutive quarters of record growth (the last two grew significantly with the introduction of the dashboard and profit sharing).

    Word is still out on it’s effectiveness, but early indications give me the sense that we’ve got momentum created from the effort. We’re a small firm, so it’s unclear how this scales to a larger enterprise.

  • trdedektiflik
  • pixbook

    I think this is a very important conversation you just launched.

    Ways to make money

Previous post:

Next post: