Good business plans delivers on results, and to get results you first have to be able to determine what they should be and be able to measure them. Ie seen many business operations that aren quite sure what results they are supposed to be delivering, or have no easy way to measure those results. They end up not really progressing or succeeding in the long run.
With the new territory that is Web 2.0, this comes sharply into view. Organizations that implementing or running Web 2.0 services like blogs, forums, wikis, and other social interaction systems, all need to know how to measure them and what measurements are meaningful. At least wee lucky that in the online world, collecting data and doing business analytics can be more automated.
Many companies already agree that for Web sites (Web 1.0) you need to be able to determine pageviews (PVs), and unique monthly visitors (UVs) as your two key metrics, to determine the success of the site.
But now consider what Web 2.0 is about and think about if those metrics still give meaningful information. If youe an organization like ours where our Community has a wide range of Web 2.0 services, how do compare those metrics between that of a forum and from a blog? Does it even make sense do that when what youe really interested in are things more like: How vibrant or healthy is our community? Who do people interact with? Is our communtiy self-supporting or do we have to do a lot to keep it alive? How much does it cost us to support our community?
My idea on this is that PVs and UVs are too low-level to answer these questions, and we need another level of metrics beyond that which Il call participation metrics. These metrics are used to try to answer the questions, or at least get a sense for what those levels are.
Now, the catch: How do determine participation metrics in a Web 2.0 system when even the ways people participate are very different between blogs, forums and other services?
The key, I think, is to go back to social network theory and the core ideas of collaboration; in particular, the idea of relationships between the members of any social network or community. It fairly easy to quantify a relationship, but it very hard to determine the quality of the relationship.
In this case, I focusing on the quantity of relationships, as well as the population mixes. Taking dW as an example, there are many ways of looking at our population but the one that interests me here is the relationships between a consumer and a producer. Simply said, you can look at four main population segments:
- the developerWork staff (dW)
- our internal ustomersacross the many product and technology groups in IBM (IBM)
- the general membership audience of dW (members)
- the authors, content sources, bloggers, and major contributors/interactors (the experts)
Thus, you can create a matrix of sorts here based on the interaction activity going on a specific area:
|Service Type||Service Use-case||Relationship|
|Blog||General technology blog||Expert-2-member (e2m)|
|Blog||Blog by IBMer on tech/product||IBM-2-member (i2m)|
|Forum||alphaWorks tech forum||Member-2-member (m2m), IBM-2-member (i2m)|
|Podcast||dW produced podcast||e2m, dW-2-IBM (d2i)|
|Forum||Expert roundtable forum||e2e, e2m|
You can go on defining more and more based on every (repeatable) use-case you can think of. More significantly, what this does is coaslesce together all the Community uses that generally contribute to specific relationships. While not entirely accurate, you could generalize that each use case mostly contributes to one or two types of relationships.
Thus, you develop a mapping across your entire landscape of interaction types for participation metrics based on relationships. If you have multiple communities (or dozens like dW has), you could limit the scope of the data to all service uses relevant to a specific community (e.g., IBM Rational ClearCase community) or specific set of communities (e.g., all IBM Rational communities), or you could look across all your communities at once (e.g., all dW). You have essentially, a set of participation metrics that applies to a range of data.
How do you use these metrics? It depends upon the questions you ask:
- Is our community self-supporting?
This is a relatively easier question because it generally looks at your ratio of member-2-member relationships versus all others. On the other hand, if you find your staff responding with all the answers all the time, (e.g., i2m) then the answer might be o
- How vibrant is your community?
First you need to define what vibrancy or health means to your organization. For some organizations that might mean how many people are talking about product X (the # of relationships in community around X). Others might look at how many experts have formed around your community. Yet others may be interested if the self-supporting segments of community are growing.
- How much does it cost us to support our community? What are the cost centers?
These are tougher ones. It requires a second set of data defining how much it costs to deliver each service or community use. But you can map, e.g., e2m is costing us $X across the entire community, and $X1, $X2, for the top activities in e2m.
- How do we learn what is effective in our community?
Here it helps if you have several or many communities to compare against. E.g., within expert2member, you could have a dozen different communities spread across different topics. Examine why the top e2m communities are growing faster than the ones on the bottom. It might be subjective elements (population for topic 1 is just growing much faster) or it might be things which you can address such as use-case features, or the approaches of the experts. It will at least help you to recognize how these are doing and provide a basis of comparison.
Again, this idea is more of a method than actual steps to take for your communities. You can see that the information is subjective to the goals and direction of your organization.