Social Enterprise ROI: Measuring the immeasurable.

[tweetmeme source= “deb_lavoy” only_single=false]

image by Steve Harris 

Social Enterprise ROI: Measuring the immeasurable.
(The plural of anecdote is not data, or Is there a scientist in the house?)

The world has been chapping our collective hides about metrics for social business. Customers want them, and not without reason. Our typical answers (ROI is irrelevant, What’s the ROI of your mother, it depends on the business problem) have some merit, but in the end, we still need to demonstrate the efficacy of social approaches to business challenges. Probably.

In reality we have very little to prove the worth of the Social Enterprise. We have some academic studies, we have some anecdotal evidence a few (very few) published use cases where metrics are involved, and we have a whole lot of “it makes sense, we feel it working”. The reason adoption has gone as far as fast as it has, is not about ROI. Rather, its because of a) the extent to which the old models are failing and b) the extent to which many people deeply resonate with the new models.

Social ROI: We’re working on it.
Predicting the ROI of any enterprise investment can be tricky. At my company, we have a whole team of people, called “Value Engineering” that dedicate their time to calculating these things. But when the topic is social business or enterprise 2.0, the challenge is much, much bigger. The reason is that the objective is to qualitatively change how work is done – how we view challenges and how we make progress. We are rethinking how we structure organizations, projects, teams and work itself. Qualitative differences are tricky to measure. Further complicating the issue, is that the outcomes of these changes emerge over time. Since these concepts have only been adopted slowly, and over only the last 3 to 5 years, we lack experience in understanding what these emergent outcomes should be (though we have plenty of theories about it), how to detect these outcomes, what is required to achieve them (though again, lots of theory), and, most importantly, in what time frames we should expect to see these outcomes. My experience is that small changes can be seen almost immediately amongst small groups, but larger organization-wide changes can take two years or more to mature and emerge.

Social Marketing ROI is progressing
That said, we’re seeing significant progress on the marketing side. Marketing, of course, is one of the two or three oldest professions, and it has been diligently measured for decades, especially in the last two, as digital marketing became the norm. You may recall that in the first decade of digital marketing,  measurement came down to one key metric – page views. Ads were priced this way, much like magazine ads. You may also recall that this was not a great metric, and did little to predict ROI. A decade later, we are somewhat better at measuring the impact of digital marketing, and have begun to treat social marketing as a special case thereof.

Case studies and ROI calculations are now popping up regularly for Social Marketing and Social Customer Service (which are now gorgeously converging into “Customer Experience, (can I get an “Amen”)). See this list, and this one  and this. Brand value and perception are more elusive, and impact of social on same are measured by opinion surveys where cause and effect can each be quite hard to determine. And still most businesses still look at “followers” and “likes” and similar metrics. Why? First and foremost, because they can. Second, of course, is because even the unsophisticated understand what those metrics mean. They may not be good judges of their importance, but people “get” them.

Even now that we have nearly 100% adoption of digital marketing (having at least a website), we should attribute this less to our improved ability to calculate digital ROI, and more to the fact that a website is now a just defining characteristic of having a business. We may yet see this happen with social workplace. It’s likely that we will – in 5 or 10 years or so, but until that point is reached, metrics and ROI will play an important role in the adoption curve – a bridge across the chasm between the early adopting visionaries and the late-coming risk averse. And time is of the essence for many companies in these days where five years can chart both the apex and the nadir of major companies.

2. Metrics are evil but also essential.

Metrics themselves aren’t evil –  metrics abuse is. Metrics abuse is responsible for countless bad decisions and negative, unintended consequences. Metrics abuse is what happens when people replace thoughtful, meaningful goals and insights with measurable metrics. People do this because thoughtful, meaningful analysis is very hard, poorly understood, and rarely done. Metrics then amount to the drunk looking under the lamppost for the keys he dropped over yonder because there’s no light over there.

There’s plenty written about the evils of metrics, [including my article last month], but its summed up nicely by the “McNamara Fallacy” by which many believe we (that’s the USA) lost the Vietnam war. This quote has been variously ascribed by the internet to both Charles Handy and Daniel Yankelovich and I can’t tell you which, if either, it really belongs to yet (I’ll let you know. promise to let you know when I figure it out). Its good enough to quote even with shaky provenance:

“The first step is to measure whatever can be easily measured. This is OK as far as it goes. The second step is to disregard that which can’t be easily measured or to give it an arbitrary quantitative value. This is artificial and misleading. The third step is to presume that what can’t be measured easily really isn’t important. This is blindness. The fourth step is to say that what can’t be easily measured really doesn’t exist. This is suicide.”

3. So why do we need metrics? Learning, convincing and deciding
We have a lot to learn about social business – though we in the biz are crazy saturated with it day in and day out, its really only 3, maybe 4 years old. We have yet to determine what it can do, what makes it work, how to be successful, and when to expect what kind of results. (though we have lots of excellent theory). The right metrics, properly applied can help us learn. Are we progressing? Why or why not? Are we defining progress correctly? Does action A work better than action B? When properly considered, metrics help us chart and adjust course toward meaningful goals.

Metrics represent an hypothesis – one that can be right or wrong – giving us the chance to prove it one way or another. Metrics are guideposts – your weight can be an indicator of health, but it is not in and of itself, health. In fact, cancer patients are known to be quite thin. Goldman Sachs, Enron or BP – all were doing well by many measures of corporate performance and each met with disaster that “no one” predicted.

Your CEO/CIO/CMO is unlikely to be as hip to the widely held notions of how social enterprise makes everything better as you are. He or she (but likely he) has a budget that must be allocated and every option is being presented as having nothing but marvelous potential. How is (likely) he supposed to decide? She will hear 10 maybe 20 pitches for different initiatives that someone wants her to fund, each with different upside potential. One will make the company smell better, and another will teach golden geese to fly in through the windows.

ROI helps to standardize the comparison factors for people who have to make hard decisions.  The person that says “if you invest x dollars you will realize y increase in revenue” has a business case that is hard to beat, except by others who will do the same. We counter this with narrative, and vision and storytelling. And in the end these techniques may prevail. And there’s some merit to arguments that stories (which present a more multifaceted picture) make better arguments than data. But when the decisions are hard, then cold hard data comes into play, and we just don’t have it hanging around yet.

4. Three challenges and going forward
But is is right to potentially dismiss social enterprise/enterprise 2.0/social business due to its so far so-called lackluster performance in the metrics and ROI department? Sameer Patel calls us all out on this in a recent post, We are now seeing (as referenced in this article) success metrics being measured in terms of how many people “believe” there are results. Now I get that perception of success is an important factor, but is still a poor substitute for actual cause and effect information.

But alas, we simply do not have the data yet. We are living in fast-paced times with three hard challenges between us and meaningful ROI analysis for social business.

1. We do not know what to measure.
2. We do not know how to measure it
3. We do not know when to measure it.

Fortunately we are not the first people to have such problems. We can look to other fields of knowledge and see how they’ve handled it. Epidemiology (among others) relies primarily on statistical correlations to begin to discern cause and effect, and create hypotheses to be proven or un by other methods. We will likely have to go big and look at extent of social business adoption (which assumes we can roughly define it) and corporate performance over time. We’ll have to sort out which corporate performance indicators are most often affected, after what time period. We’ll have to create cause and effect hypotheses, which we’ll then go have to prove – much like Ester Dufflo did in this TED talk on using economic theory to prove or disprove the efficacy of poverty alleviation efforts.

So in the long term, we’ve moved our challenge from figuring out how to measure how social business works, to correlating the adoption of social business to traditional corporate outcomes. Ok – now how do we start working through this correlation. Anyone? Help? We need to be scientists. We need to create controls and comparisons.

In the short term, we look for patterns – that is for clusters of activity to show up, or for an indication that when x happens, y usually also happens, or cycles – a leads to b leads to c. A person who reads x tends to also read y. A person that contributes p generally sees q changes in outcome. We may not yet know if they are sustainable, scalable, or even beneficial, but we are learning to identify patterns, we are learning what those patterns have to teach us. We are building maturity models and benchmarks that help us look at our own initiatives alongside anecdotal success stories. From here we will form our hypotheses, and learn. With the help of patterns, metrics, and a bit of faith that what our guts tell us is right, is indeed right.

And we continue to build our vision of what the ideal state looks like and our pathway from here to there.

The best is yet to come.

This post originally appeared on CMSWire