Friday, February 19, 2010


Separating the relevant and useful metrics from the noise

The Twitter phenomenon marches on, however one thing that is still being hashed out is the set of metrics required to track activities in the space. There are countless tools out there, each with their own take on how to analyze Twitter data. One thing is immediately apparent - for any particular metric, different definitions are offered. Perhaps this is a reflection of the nascent nature of this space. There is also no one place where you can find a set of clearly defined, easy to understand and insightful metrics. There is a tendency to use overly complex calculations to develop exotic metrics which tell us little about what we need to know. I hope to examine a few existing metrics to put the focus where it really ought to be - on solid, simple, insightful metrics that tells the story as it is.

So what do we need to know?

The question is why are you on Twitter? Companies may use it for brand promotion/management, traffic acquisition or simply, as a communications tool. Each of these goals can be used as the basis for our analytic questions.  (I am restricting myself to questions only relevant to Twitter Activities)

Brand Promotion Questions

  • What is the rate of growth of your audience?

    Online communities are rarely finite, nor are they static. The best and most successful communities ebb and flow. The profile of membership changes, people move through phases of observation, participation, and back again. Some leave and never come back, others stay for the long haul and adopt different roles along the way. The only consistent factor of the makeup of a community is that it’s not consistent, but rather an organic thing in itself that changes along with the needs, interests, and profile of its members.

    Source: Radian6 - Community eBook (Feb 2010)

  • What is the health of your brand online?
    * See note about Sentiment Analysis in closing comments.

    In establishing the return on their investment, 73% of the panel currently look to the “quality” and “tone” of the feedback generated on sites like Facebook and Twitter..

    Source: World Federation of Advertisers (WFA) - Buzz and brand metrics key to social media, WFA finds (Feb 2010)

  • Who are the top influencers driving the conversation about your brand?
  • What are the top conversations about your brand?

Traffic Acquisition Questions

  • What is the volume of Twitter referrals?
  • What is the Click Through Rate of specific campaign links?
  • Who are the most Influential people driving traffic to your site?
  • What is the quality of Twitter referred traffic?

Communication Questions

  • What is the potential size of your audience?
  • What is the level of engagement with your audience?

    Are they listening and providing feedback to what you have to say? How responsive is your audience?

  • What is the scope of your conversations?

    Are you talking to the same small set of people every month?

 


The Basic Metrics

First principles, Clarice. Simplicity. Read Marcus Aurelius. Of each particular thing ask: what is it in itself? What is its nature? What does he do, this man you seek?

Hannibal Lecter, Silence of the Lambs (1991)

There is a dire need for a robust, accurate and timely twitter analytics package, and I know too many people who will pay for it. Take a look at what is available today - one key observation is that various reporting tools have very different metrics and don’t have all the “basic twitter metrics” in one place. What are the basics?

  • Total Followers

    Total number of your Twitter followers.

  • Total New Followers

    Total number of new followers.

  • Total Unfollows

    Total number of dropouts - (People who stopped following you).

  • Churn Rate

    Defined as (Total Un-Follows / (Total Followers - New Followers)).

  • Total Active Followers

    Total number of unique followers who either replied to your tweets (@) or retweeted any of your messages.

  • Reach

    Total unique Twitter USERS you have access to. Here, I prefer a count of the total unique Twitterers who see an impression of any of your tweets. This is essentially a head count of the unique number of people you reached with any of your tweets ( Followers and Non-Followers). I haven’t seen this metric anywhere. Note that this is not a count of the unique people in your second level network. The argument against using a count of second level network is, why stop at second level?  Some reporting tools also use “total impressions”. But this produces an implausible number very quickly simply because it counts everytime an impression is made - so for instance if user A saw 10 of your tweets, he is counted 10 times.  This is one metric that can get fuzzy very quickly.

  • Total Tweets

    Total number of tweets sent by you.

  • Total Mentions

    Total number of times you were referred to in a tweet.

  • Total Retweets

    Total number of times your tweets were passed on.

  • Total Unique Followers with @Tweets

    Total unique followers who communicated with you.

  • Total Unique Followers ReTweeting

    Total unique followers who retweeted your messages.

  • Total Tweet Impressions

    Total number of times your tweets were displayed.

  • Total Impressions with Links

    Total number of impressions with links

  • Click Through Rate of Links

    Percent of links that were clicked. (Total Click Throughs / Total Impressions with Links).  This particular metric isn’t usually available from the regular Twitter reporting services. Url shortening services are uniquely placed to capture such data. I don’t see it on any of their reports yet. Hopefully, it should be available soon.  They can also capture finer details of campaign parameters embedded in the url.

  • Top 10 Followers who retweeted any of your messages

    ... the bottom line is that a very small number of people with a high propensity to share easily trumps a very large number of people with a modest propensity to share. In my view, the goal is to become remarkable. Literally remarkable, in the sense that people remark about your stuff.

    Mark Drapeau - Social Media Metrics: Count Thank You’s, Not Click-Throughs

  • Top 10 Followers who replied to any of your messages (@)
  • Top 10 Conversation Topics
  • Top 10 Twitter users driving the discussion about your brand

    These should capture any Twitter user (Follower or Non-Follower) with a mention of your brand name.

These should be made available by various time segments, preferably, daily, weekly and monthly.


Answers to what we need to know

From this basic set, we can now attempt to answer the questions above:

Brand Promotion Metrics

  • Total Followers
  • Total New Followers
  • % Change in Followers (MOM)
  • Churn Rate
  • Top 10 Twitter users driving the discussion about your brand

    This should capture any Twitter user (Follower or Non-Follower) with a mention of your brand name.

  • Top 10 Conversation Topics
  • * The next 4 metrics are pulled from the Radian6 document mentioned above. See my notes on Sentiment Analysis below

  • Positive/negative/neutral ratios over monthly, quarterly, annual periods
  • Recovery time for sentiment ratios after a crisis
  • Emergence of evangelists: % of positive posts from single source
  • Emergence of detractors: % of negative posts from a single source


Traffic Acquisition Metrics

  • Total Tweets with Links
  • Click Through Rate of Links
  • Top 10 Followers who retweeted any of your messages
  • * The next set of metrics can be obtained from your regular Web Analytics tools lile Google Analytics - they capture the quality of the referred traffic. Again, I am using metrics pulled from the Radian6 Community eBook.
  • Referral traffic volume from Twitter
  • Time on site from Twitter referrals
  • Conversions from Twitter referrals
  • Conversion/click through percentages for specific Twitter campaigns

Communication Metrics

  • Total Active Followers
  • Reach
  • Ratio of Total Tweets to Total ReTweets
  • Ratio of Total Tweets to Total Replies
  • % Followers Retweeting
  • % Followers with Replies

Final Thoughts

  • You will notice that most of the basic metrics focuses on what followers (and non- followers in some cases) are doing in response to your tweets. Isn’t that what it’s all about? It is a two way communications channel, and the ultimate goal is to measure the quality and effectiveness of that communication activity.
  • You will also notice that there is no mention of sentiment metrics / analysis. After looking at what some tools offer, I am of the opinion that the methodology or technology to obtain an accurate gauge of user sentiment from a Tweet is not fully developed or accurate at this point. There are numerous posts and examples to highlight this issue.  I am staying away from any such analysis until someone comes up with a more accurate measure. Consider the sentiment metrics under “Brand Promotion Metrics” as placeholders.
  • Finally, I will like to encourage all developers in the Twitter Analytics space to at least consider providing the basic metrics listed above, regardless of what other metrics they consider important. Even better, an API to allow us to port this basic data into other analytic tools or visualization packages will be nice.

 


Comments

By Waldron Faulkner
Posted 08:09 pm
Fri, Feb 19, 2010

Hey Victor, thanks for this comprehensive overview. We’ll keep this bookmarked as a reference for building-out the GraphEdge product offering.

Waldron Faulkner
Founder, GraphEdge

1

Post comment