Tag Archives: metrics

Measuring Engagement

1 Reply

There’s a worthwhile discussion on the measurability of engagement at Web Analytics guru Eric Peterson’s blog. Eric sets out to measure the engagement of a few users in his RSS base and, based on the responses of those users, will now have to figure out how to readjust his metrics. Some of the best information is in the comments section, especially where Gartner’s Bill Gassman weighs in on the slipperiness of engagement:

Each organization’s version of engagement will be unique. It will be
derived from a number of root metrics, probably under a dozen. Common
root metrics will be frequency, recency, length of visit, purchases and
lifetime value. Some organizations may include visitor actions, such as
subscribing, providing personal information, writing a comment, or
participating in a blog. Soft metrics, such as attitude, influence and
obsession may be used. Not all root metrics will come from the Web
analytic tool. Many will use metrics from other channels such as call
center actions and physical store visits.

Anyone sitting on the fence waiting for a dashboard of engagement and ROI metrics should check back in a year or so. This isn’t going to be wrapped up in a bow any time soon. Of course, by the time the metrics are solidly drilled, the competitive edge will have long since dissolved into an efficiency game. Hell, when Betty Crocker has an RSS feed–and a nice one at that–this is no longer the territory of early adopters.

Marketing ROI: Return on [Your Concept Here]

5 Replies

I got a trackback this afternoon from Brian Solis, and his post on ROI vs. ROP for social media metrics. ROP? Yet another formulation of RO[blank] intended to sound like a financial metric. Brian’s passion about the need to engage customers and the value of social media is spot on. But I have to groan when I see another new permutation of Return On Anything Short of Actual Revenue.

As I argued passionately during the last frothy marketing trend on "brand", marketers are great at creating, evolving and innovating ideas. But they are also trend-crazy. And when you combine those two predilections with serious business issues, you end up with a profession that continually confuses the hell out of the rest of the business group. Every marketer you talk to defines things a different way. But more important, especially on the topic of ROI, marketing loses all credibility with the board room suite when it twists and bends financial metrics designed to measure value creation into concepts that skirt the issue of accountability for actually creating value. This goes for Pepper’s and Roger’s Return on Customer as well–even though there are many good ideas there about how to value customers. 

ROI means something important that too many marketers seem completely unable to get their arms around. It means demonstrating in clear terms that the programs you’re pumping money into are returning real cash value to the company. Is that hard in marketing? Absolutely. Often it’s next to impossible, or at least the cost of measurement is greater than the cost of the program in the first place. But marketers need to be straight up with those challenges, and not dismiss or dilute the value of ROI by introducing ROFOTM (Return On Flavor of the Month).

Businesses are right to ask about the ROI of social media. If I put a dollar in the stockmarket, I get a ton of data that helps me peg the risk of getting my dollar back with interest. Sooner or later, marketers are going to have to learn how to explain why a shareholder’s dollar is better invested in the company’s marketing programs and not the stockmarket. Marketing will not win any points by responding with a feint and a dodge, and saying "these other metrics are more important", or my personal favorite, "they don’t get it."

Social media is early in it’s evolution, even if it is building on older concepts (remember The Well?). The value creating potential varies tremendously from industry to industry, and from company to company, and it also comes with some significant risks for companies used to commanding and controlling their messages and markets. Those challenges should not be swept under the rug, with social media packaged up, wrapped in a bow, and sold as the new Web 2.0 elixer of life. Those challenges–like how best to measure the potential ROI of social media–should be mapped, debated and addressed openly.

There are a lot of good arguments to proselytize social media today. It’s relatively cheap, it can generate market insights often more expensive to gain in other ways, it can improve customer intimacy, it can dramatically improve visibility on the Web–there’s a lot more, but good measurement is not on the list, yet. Let’s please spend a little time and energy figuring that one out before we package it up and sell the hell out of it.

Measuring Social Media

4 Replies

I want to go a step further in breaking down the dialog over social media metrics, in the wake of the Factiva roundtable. The questions I asked yesterday were what should be measured, why should it be measured, and what will the impact of those measurements be. Let’s start by looking at who wants to measure social media in the first place.

When a social media channel first takes off, the measurements that are important first are those that are relevant to the Content Provider. Take Marketonomy. I want to know what kind of people are reading my blog. I want to know what they find interesting. I want to know how to better target my content to build a better relationship with readers. Currently, those metrics are not very robust. I know how many people subscribe to my RSS feed, but I know next to nothing about them. I know which of my posts get read the most, but with the small ratio of comments to readership, I’m not always certain what drives those posts to the top. The metrics I wish I could get today are demographic. I don’t need to know *who* my readers are personally, but I would like to know have a general profile of their professional background and areas of interest.

Once a social media channel has some traffic, new measurements come into play. If you want to bring on advertisers or sponsors, you need to demonstrate the value of your channel over others. In print, you point to subscription demographics and circulation numbers to justify ad pricing. On the Web, you’ve got traffic. Of course, these metrics are open to gaming. Publications find creative new ways to inflate their circ numbers, just like Web sites find creative ways to inflate traffic. Content providers package pretty numbers to push up ad prices, and ad buyers poke holes in those numbers to push the price down, and the same game will certainly develop with social media.

Just like the current dialog in Social Media circles, traditional media has mounted periodic campaigns to introduce new metrics that justify more ad spending. I once worked for a magazine that tried to measure pass-along rates, and I’ve put the same concept into play with email campaigns using source codes. But these metrics are usually a sign that the provider doesn’t have leverage with buyers, which means they probably don’t have high demand. Just as traffic/circulation has been the bottom-line metric for traditional channels, I would argue that it’s going to remain the primary metric for most social media channels too–the more traffic you have, the more demand there will be to reach your audience, the less granular your metrics will need to be, simply because you’ve got waiting buyers.

But the complicating factor in all this is how a channel grows and maintains a robust audience, because once it reaches critical mass, it has a hungry mouth to feed. You’ve got to find worthwhile topics to cover, people to interview, or at least engaging people that will drive dialog and audience participation–and if they’re really that interesting, other people will want them too. That’s where other players in the value chain for delivering content often come into play, and other metrics matter to them. Whether you’re dealing with an interview prospect’s secretary, or an executive’s PR rep, or even if you’re relying on users creating content, the question is: Why should anyone bother driving content into your channel? The answer is that there must be some value they can derive from it, and one of two metrics will demonstrate that value.

For anyone actually engaging in social media, the important metric is likely Participation. The more people are engaged, the more people I have to talk with, or at least listen to. For anyone interested in getting content in front of your audience, like a company or PR group, the important metric beyond raw traffic is likely Influence–which is the power to impact the opinions, attitudes and behaviors of a target audience. In Social Media, traffic, participation and influence are certainly related, although the relationship doesn’t seem all that clear yet. You can have high traffic and low participation, and still have signficant influence–just as the Wall Street Journal has for generations. You can generate influence with low traffic and high participation–just as my former job at the CMO Council did with programs engaging a small group of leading marketing executives.

I’ll dig into the dynamics of influence more next time. For now, I’ll just repeat the argument that if you have significant traffic, you’re going to have high demand to reach your audience. More granular metrics will help you raise the value of reaching your audience, or compensate for a lower traffic numbers than competitors.

Social Media Metrics

12 Replies

I participated in a great dialog in Palo Alto last night about social media and metrics, hosted by Factiva. It’s one of the more focused gatherings I’ve been to recently on social media—which typically refers to the kind of content that connects people, as exemplified by blogs, forums, wikis, peer networks and other similar channels. It was more like a working session among practitioners to try and define some of the parameters for how social media should be measured–should we be looking at influence, engagement, audience reach, demographics? Jeremiah Owyang has a full write up of the event, including the outcome of our prioritization of metrics.

What was striking to me was that even in a room full of social media-savvy people from all sides of the industry, the discussion was still somewhat chaotic, with a lot of semantic negotiations over the meaning of words like "relevance" and "engagement". This reflects a similar level of disarray in the community at large over how to tackle measurement now that demand for it rising. So I want to step back for a minute and think about the context for social media metrics. Why is there a growing demand for measurement? Who wants to measure what? Why does it matter? And what is the likely impact of measurement on the evolution of social media?

One of the most incisive comments of the evening came from Linda Kozlowski from Flieshman Hillard, who said "suddenly everyone wants to make a pie chart out of social media," and that’s exactly the mile marker we’ve reached in the evolution of the medium. Social media has moved out of the realm of early adopters, where people engaged and propelled the medium from a sense of passion and vision, and has moved toward the mainstream, where audiences are growing and businesses are trying to adapt the medium to promote business objectives. Podcasts, for example, are no longer the exclusive domain of true-believers broadcasting on their favorite passion from their basement, they’re being developed more and more by marketing departments and even networks who see an audience rapidly reaching critical mass. As more money flirts with social media, the tone of development quickly moves from passion and vision to effectiveness and ROI.

So now a lot of businesses want to get into social media, and their reasons are all over the map. Some truly want to engage their audience and see how it impacts their business, ostensibly by improving customer responsiveness and product development. Others want a new channel to connect with their target audience. Others just want to be part of something cool. But in the current business environment of intense focus on business process and metrics, almost everyone is asking how they can measure success with the new medium. The promise of cash flowing into social media development, but gated by pointed questions over metrics of success, has a lot of people scrambling to come up with incisive measurements to populate the CMO’s dashboard–like measuring the degree of influence certain bloggers have with an audience, or measuring the degree of engagement over a key issue. New metrics are a great idea, but I think there’s a deeper discussion that has to happen before this issue really shakes out.

First of all, why is measurement really needed? The most poignant answer is that anyone in the social media business is not going to get any money if they can’t put a value on what’s being purchased. The companies with money to invest want to buy insight, access and influence with a targeted audience. Ironically these same companies accept very tenuous metrics in mediums where they allocate huge budgets. Look at advertising. Across every traditional medium, from television to print, metrics are notoriously tenuous in linking investments to bottom-line returns, and businesses have learned how to negotiate pricing and enhance measurement of impact in creative ways. On the Web, where tighter measurement of traffic is possible, measurements of behavior and action are available–like click throughs and purchases–instead of just impressions. But if you really look at the dynamics of advertising on the web, it seems that tighter metrics are more a feature of less powerful channels, which need to provide more justification to potential buyers to stimulate demand. The heavy hitters–the large networks with massive audiences–don’t need to price their space on delivery of measurable actions, because the demand for impressions is high enough that they don’t have to.

I would argue that the same dynamic is shaping the development of metrics in social media today. Interest in moving into social media is high, but companies don’t have a context yet in which to place social media alongside other costs they already understand, like PR and advertising–even if the effective measurement of those other costs remains somewhat tenuous. So they’re going to ask a lot of questions about how to measure the value of what they’re buying, and they’re not going to be satisfied with the answer that measurements in social media are really no better than measurements in traditional media. Social media companies will race to provide metrics that show impressive results, but that’s not necessarily a good thing, especially given the ways measurement may shape investment and in turn shape the evolution of social mediums.

Last night, the metric that popped out as being the highest priority was “engagement”. If you’re going to have a social medium, it’s useful to know just how social it is. Are people actively commenting, debating, sharing ideas? If this becomes the metric of a successful channel, what does that really mean, and how will that narrow the perception of, and investment in, different types of social media channels? Some topics, and some communities, drive tremendous levels of dialog and engagement. The audience is comfortable engaging online, and passionate about their views. But some topics and communities attract a lot of “lurkers”, people who will hang out and listen, but not step out on a limb and comment, for any number of reasons ranging from fear of a slap-down, to concern over having their comments Googled when they apply for a job. One of the interesting things about SecondLife is that unlike a forum or blog, you can physically see all the lurkers watching from the sidelines. If your goal is to drive brand awareness, these lurkers are tremendously valuable, and just because they’re not engaged in the verbal discussion, doesn’t mean their presence isn’t valuable to a marketer. How should a metric of “engagement” be valued alongside “presence”?

This is already a long post, so I’ll pick up the thread again tomorrow on some other issues. But to wrap this up, I think it’s important for social media gurus to take a breath before racing down the track to deliver new metrics, and start asking some critical questions about why these measurements matter, who they matter to, and what the implications are of adoption. Are the measurements designed to aid in gaining insight into market issues and trends? Or to refine and direct influence over market dialogs and impressions? Are the proposed metrics really an accurate measurement of value, or just a reflection of a particular bias toward one form of social dynamic? And perhaps most importantly, what are the potential unintended consequences of leading investors to value particular measurements of social media interactions? (Think of Nielsen ratings for pricing television advertising, and the stupid spectacle of sweeps week.)

Kudos to Factiva and Jeremiah Owyang for putting together a great dialog, with a great meal in the bargain. This is only the first round of an important conversation.