11 Dec 08 The 90/9/1 rule is just one kind of behavioral segmentation
To anybody who has been measuring social networks for long, the “90/10/1 rule,” subject of recent buzz, is nothing new. I don’t just mean online social networks, I mean social networks in the real world, long before computers became a social networking medium. Mark Williams, a community manager, asked the right question in his blog, what is it good for? It is a guideline, Mark says – a way to set reasonable expectations with clients who might imagine that a far larger percentage of visitors will become deeply involved in the community.
Mark is right – that is certainly the primary purpose of the rule, but it is just a start. When you think of it as a way to segment community by a particular kind of behavior, you’ll quickly recognize that there are other behaviors that are worth examining similarly. Call it a “contribution” behavioral segment, since it is is based on how much each visitor contributes to site’s content. There are many other interesting behavioral segmentations:
- Responsiveness – which of the community’s interactive features do visitors use and how often do they use them?
- Retention – how often do visitors come back?
- Churn – what is the turnover rate for visitors?
- Topics – do people participate equally in the community discussions?
One of these days, perhaps we’ll all know what is normal segmentation for various types of communities (e.g., a support community will be quite different from an affinity community).
For deeper insights, compare the different segmentations and look for disconnects. I would be especially concerned to find disconnects between contribution and the first two examples, responsiveness and retention (my “R&R” of engagement). If the major contributions aren’t using interactive features as frequently as they contribute, that might reveal a design or even more fundamental problem with the features. If they aren’t returning to the site at a normal rate, that suggests trouble ahead.
Side note: I suspect that behavioral segmentation is a good way to find communities within communities. One of the challenges of community management is to figure out when a group needs to be split into two or more. Discovering cliques that are naturally following the normal patterns might be candidates to spin out. In other words, I’ll bet behavioral segments are somewhat of a fractal phenomenon. And if nothing else, they give us more ways to generate pretty visualizations, eye candy for that next conference or sales presentation.
Tags: community, Engagement, R&R, Segmentation
26 Nov 08 Engagement R&R
People argue for complex, customizable engagement metrics. Unfortunately, they often fail to specify what kind of engagement they are measuring. Brand engagement? Web site engagement? Social media engagement? These are different, but people unfortunately tend to lump them together, robbing metrics of significance. Almost everybody I see talking about engagement also fails to distinguish positive and negative engagement, which results in even greater vagueness. These problems are sometimes rendered invisible through complexity – an engagement formula that has lots of variables to play with surely it must be better than one that relies on a few.
I approach engagement metrics differently, partly because I constantly remind myself of two things:
- Metrics are only a proxy for the energy that goes into visitors’ behavior. What comes out of the analytics system is not necessarily what went it.
- Simpler is generally better because many metrics correlate very well to each other; greater complexity yields greater ambiguity.
The research I’ve done on community engagement suggests that the most important types of metrics are retention and responsiveness – my R&R of engagment. I look at those metrics by segment, but that’s about as complex as it gets. When I examine formulas with greater complexity, I always find that some of the variables are essentially measuring the same thing (page views and time on site, for example) or they reflect poor thinking about what is really being measured. At the very least, it is worthwhile to measure each variable’s contribution, via principle components analysis or a similar multivariate analytic method. If they don’t contribute significantly, you’re just wasting time and resources on them.
Retention measures how often people come back. Responsiveness measures how much more they do than just passive page viewing. In the context of social media, retention metrics look at how often people return to a site, a category, a forum or even a single discussion thread. Responsiveness refers to behaviors like posting messages or comments, voting in a poll, updating a profile, uploading content and so forth. There is a great temptation to weight those actions. E.g., posting a message is worth five points, but voting in a poll is worth only one. I have generally resisted this, mostly because I haven’t see a significant difference in the results, but also because of the way that I think about what’s being measured. It’s sort of a physics problem.
What we really want to know is how much energy visitors are putting into their participation. For example, writing a one-word reply to a message doesn’t take much energy. Finding a relevant web page and pasting its URL into a posting with some original comments takes a lot more. Sometimes a lot of energy is invested in a very small action. The ambiguity that is most important to recognize is valence – the fact that the energy people invest in social media can be positive or negative.
Valence is especially important in neutral venues (versus fan clubs) and when opinions run high. A simple example: during the last U.S. presidential election, Obama supporters may have been highly engaged in McCain venues and vice versa. Without a measure of valence, all you could say is that those people were engaged in the social media venue and the election, not in specific candidates. In other words, it is fine that we don’t measure valence as long as we remember what that means about the numbers we produce.
The same issue is present in influence measurement (which is closely related – you can’t be influential unless you are engaged). To an Obama supporter, McCain was highly influential, but with a negative valence. It is a well-known fact of marketing that people engage with things that they dislike and disagree with – frequently! We all have a bit of the talk radio host inside.
In short, it is critical to keep in mind that when we measure engagement, the action we measure may not accurately reflect the energy that went into it and that we almost always include visitors who are engaged as cheerleaders and critics. Keeping those two ideas in mind will lead to better metrics and interpretation.
Tags: engagement, influence, pca, valence
Tags: Engagement, responsiveness, retention
