I participated in a great dialog in Palo Alto last night about social media and metrics, hosted by Factiva. It’s one of the more focused gatherings I’ve been to recently on social media—which typically refers to the kind of content that connects people, as exemplified by blogs, forums, wikis, peer networks and other similar channels. It was more like a working session among practitioners to try and define some of the parameters for how social media should be measured–should we be looking at influence, engagement, audience reach, demographics? Jeremiah Owyang has a full write up of the event, including the outcome of our prioritization of metrics.

What was striking to me was that even in a room full of social media-savvy people from all sides of the industry, the discussion was still somewhat chaotic, with a lot of semantic negotiations over the meaning of words like "relevance" and "engagement". This reflects a similar level of disarray in the community at large over how to tackle measurement now that demand for it rising. So I want to step back for a minute and think about the context for social media metrics. Why is there a growing demand for measurement? Who wants to measure what? Why does it matter? And what is the likely impact of measurement on the evolution of social media?

One of the most incisive comments of the evening came from Linda Kozlowski from Flieshman Hillard, who said "suddenly everyone wants to make a pie chart out of social media," and that’s exactly the mile marker we’ve reached in the evolution of the medium. Social media has moved out of the realm of early adopters, where people engaged and propelled the medium from a sense of passion and vision, and has moved toward the mainstream, where audiences are growing and businesses are trying to adapt the medium to promote business objectives. Podcasts, for example, are no longer the exclusive domain of true-believers broadcasting on their favorite passion from their basement, they’re being developed more and more by marketing departments and even networks who see an audience rapidly reaching critical mass. As more money flirts with social media, the tone of development quickly moves from passion and vision to effectiveness and ROI.

So now a lot of businesses want to get into social media, and their reasons are all over the map. Some truly want to engage their audience and see how it impacts their business, ostensibly by improving customer responsiveness and product development. Others want a new channel to connect with their target audience. Others just want to be part of something cool. But in the current business environment of intense focus on business process and metrics, almost everyone is asking how they can measure success with the new medium. The promise of cash flowing into social media development, but gated by pointed questions over metrics of success, has a lot of people scrambling to come up with incisive measurements to populate the CMO’s dashboard–like measuring the degree of influence certain bloggers have with an audience, or measuring the degree of engagement over a key issue. New metrics are a great idea, but I think there’s a deeper discussion that has to happen before this issue really shakes out.

First of all, why is measurement really needed? The most poignant answer is that anyone in the social media business is not going to get any money if they can’t put a value on what’s being purchased. The companies with money to invest want to buy insight, access and influence with a targeted audience. Ironically these same companies accept very tenuous metrics in mediums where they allocate huge budgets. Look at advertising. Across every traditional medium, from television to print, metrics are notoriously tenuous in linking investments to bottom-line returns, and businesses have learned how to negotiate pricing and enhance measurement of impact in creative ways. On the Web, where tighter measurement of traffic is possible, measurements of behavior and action are available–like click throughs and purchases–instead of just impressions. But if you really look at the dynamics of advertising on the web, it seems that tighter metrics are more a feature of less powerful channels, which need to provide more justification to potential buyers to stimulate demand. The heavy hitters–the large networks with massive audiences–don’t need to price their space on delivery of measurable actions, because the demand for impressions is high enough that they don’t have to.

I would argue that the same dynamic is shaping the development of metrics in social media today. Interest in moving into social media is high, but companies don’t have a context yet in which to place social media alongside other costs they already understand, like PR and advertising–even if the effective measurement of those other costs remains somewhat tenuous. So they’re going to ask a lot of questions about how to measure the value of what they’re buying, and they’re not going to be satisfied with the answer that measurements in social media are really no better than measurements in traditional media. Social media companies will race to provide metrics that show impressive results, but that’s not necessarily a good thing, especially given the ways measurement may shape investment and in turn shape the evolution of social mediums.

Last night, the metric that popped out as being the highest priority was “engagement”. If you’re going to have a social medium, it’s useful to know just how social it is. Are people actively commenting, debating, sharing ideas? If this becomes the metric of a successful channel, what does that really mean, and how will that narrow the perception of, and investment in, different types of social media channels? Some topics, and some communities, drive tremendous levels of dialog and engagement. The audience is comfortable engaging online, and passionate about their views. But some topics and communities attract a lot of “lurkers”, people who will hang out and listen, but not step out on a limb and comment, for any number of reasons ranging from fear of a slap-down, to concern over having their comments Googled when they apply for a job. One of the interesting things about SecondLife is that unlike a forum or blog, you can physically see all the lurkers watching from the sidelines. If your goal is to drive brand awareness, these lurkers are tremendously valuable, and just because they’re not engaged in the verbal discussion, doesn’t mean their presence isn’t valuable to a marketer. How should a metric of “engagement” be valued alongside “presence”?

This is already a long post, so I’ll pick up the thread again tomorrow on some other issues. But to wrap this up, I think it’s important for social media gurus to take a breath before racing down the track to deliver new metrics, and start asking some critical questions about why these measurements matter, who they matter to, and what the implications are of adoption. Are the measurements designed to aid in gaining insight into market issues and trends? Or to refine and direct influence over market dialogs and impressions? Are the proposed metrics really an accurate measurement of value, or just a reflection of a particular bias toward one form of social dynamic? And perhaps most importantly, what are the potential unintended consequences of leading investors to value particular measurements of social media interactions? (Think of Nielsen ratings for pricing television advertising, and the stupid spectacle of sweeps week.)

Kudos to Factiva and Jeremiah Owyang for putting together a great dialog, with a great meal in the bargain. This is only the first round of an important conversation.