So you’ve agonized over whether or not social media is the right data source for your research goals and you have crawled through cyberspace to collect relevant tweets, blog posts and comments. When it comes time to analyze that data, many researchers quickly find themselves making decisions as to whether the data they have collected should be shaped into a qualitative or quantitative story for their clients. This usually leads to a debate (either among the project team or among the voices in one’s own head) as to the appropriateness of either approach given the unstructured, subjective and exploratory nature of the data itself. But let’s assume for the moment that you’ve rationalized a quantitative approach and want to make some summary observations about the data.

For many starting out with social media data, the inevitable first question tends to be something along the lines of: What is the attitude of this population towards my research topic? It seems that a natural starting point for many of us is simply understanding the prevailing sentiment of the population. Do they like brand X? Are they supportive of this new product idea? Will they vote for candidate Y? The approachability of these types of observations among researchers is perhaps in part aided by service providers’ eagerness to promote sentiment analysis solutions as part of the out-of-the-box functionality of many social media monitoring platforms. While many others in our industry have and will continue to debate for and against the mechanics and implementation of automated sentiment analysis solutions, I tend to steer my clients and towards a different, but related, discussion regarding more tactical considerations when looking to classify and quantify social media data.

To illustrate these considerations, we’ll use a tracking study we’re currently running in the telecommunications sector. Every month, our social media monitoring platform collects over 2,500 tweets that are discussing major wireless phone providers in any number of ways. Over a very short period of time (this study launched in Q1 2012), we’ve already begun to amass an incredibly large and rich dataset that we can use to help clients in this space better understand their customers’ needs. So how are we using this data to solve real client problems?

Let’s say that, when looking at a slice in time, we can very quickly measure the social media footprint of two leading wireless phone providers. Over a four to six week period this spring, Provider A was tweeted at or about 1,979 times, while Provider B was tweeted at or about 1,212 times.  My client, Provider B, wants to know if there are implications for having a smaller footprint and wants to know what insight she can glean from those who are tweeting about both providers. Following the natural inclination to classify this data into positive and negatives, I started by generating the chart below.

After classifying my data into positive, negative and neutral mentions about each brand (regardless of whether I used an automated sentiment program or coded it myself), I was able to make some preliminary comparisons about the two providers. Right off the bat, it’s clear that the tone of the conversations regarding Provider A are generally more positive than they are for Provider B – but, ultimately, this doesn’t give the client much in terms of actionable insight. Still, I’ve made the decision (for better or worse) that the volume of data in this social media data set requires some quantitative classification – there is just too much text to approach it from a purely qualitative perspective. So my next step was to classify the data on deeper thematic levels. After a thorough review of my data set, I was able to classify all of the Twitter conversations in my data into a number of categories and subcategories that reflected common behaviours, nature and contexts across each provider. This exercise allowed me to create a more comprehensive picture for the client.

Suddenly, I am able to make a greater number of observations that get to the heart of the similarities and differences between how these two providers are being discussed on Twitter. Right away, we can see that there are similar patterns in terms of the context of these conversations. For instance, in both cases roughly one-sixth of customers are tweeting about a specific customer service experience they had, while another 12 to 14 percent are tweeting a question or comment about their service provider’s pricing or fees. This summary also tells us that Provider B appears to have picked up some noteworthy mentions as of late because of a particular promotion they were running. If our client was really stuck on understanding the tone of these conversations, we could also have used sentiment scoring to learn that perhaps the majority of comments about Provider B promotions were positive during this time period, while maybe the inverse would have been true for Provider A – thus suggesting that Provider B was more successful in this regard. A quick note: While I often use quantitative analysis to summarize thematic content, I don’t mean to suggest that these numbers automatically have statistical importance, nor do I suggest that they are projectable beyond the immediate dataset. Rather, just as a focus group moderator may take a show of hands as a rough measure of purchase intent among participants, so too can quantitative summaries help us organize otherwise qualitative social media data to establish a frame of reference for other information.

Looking at the summary above a bit more though, I was actually more struck by the potential implications we could derive from other themes in this dataset. In both instances, about five percent of the conversations involving these providers were actually questions or comments about specific special features or apps. The thematic coding of these tweets made these questions leap off the page more so than they would have had we been wading through the thousands of other tweets. Ask yourself: If there were 60 to 100 customers asking questions about a new mobile phone feature or app every month, wouldn’t that be something your clients would want to act upon? From here, I actually made the jump back to a qualitative review of all the tweets that were classified as ‘questions about special features/apps,’ and found that there were a number of questions about the availability and compatibility of features like mobile TV viewing and new functionality that were being asked almost daily. Sentiment analysis was largely unnecessary in this case, as many of these were simple customer service inquires about where more info could be found. These findings led me to form recommendations for my client about improvements to the way information about special features and apps were organized on their website and at kiosks in retail centres. We had used social media data to identify gaps/opportunities for our client to respond in a dynamic marketing environment.

While this was just a small nugget of insight, it was one that wouldn’t have been possible if I hadn’t taken the time to develop a thorough understanding of the nature and context of the conversations happening in this space – a process that was ultimately more actionable than relying on sentiment scoring alone. And we’ve only really just begun to scratch the surface of how classifying social media data can help close the gap between research objectives and actionable insight. I believe that more work classifying and organizing unstructured social media data could result in more sophisticated and familiar metrics from ‘traditional’ market research methodologies making appearances in these increasingly relevant datasets. Net Promoter Scores, voting intention and purchase intent could all potentially be derived from careful consideration of the nature and context of social media interactions.

Up next: What can social media tell us about how different target populations interact online?

Before attempting any social media research analysis, it’s important to understand how the target population you are researching is using social media. This is an important first step to ensure you are considering the right medium, and also helps you focus your efforts to uncover the most informative and actionable insights. I have also found that this design-stage review can prove to be a useful exercise for contextualizing the information that you come across – or, in some cases, the absence of information.

Earlier this year, a marketing manager with a Canadian mutual company asked us to help them develop a better understanding of the perspectives and opinions held by both investors and professional investment advisors towards socially-responsible investments (SRIs). We had already been planning to do some qualitative research with these audiences, but the client was especially interested in the social media context, as they had been gearing up for an aggressive social media marketing campaign that was to begin in a few weeks’ time. The client was hopeful that there would be some conversations happening in social media that could give them a quick read on the views of advisors that could in turn serve as a bit of a temperature check as their campaign got underway. However, it didn’t take long for our team to realize that investment advisors were greatly under-represented in conversations regarding SRIs. Much of the content and opinions were being discussed by professional bloggers and journalists –  who were certainly demonstrating a firm grasp on the topic, but were clearly not coming from a position where they were providing investment advice to clients. While some of the bloggers touted credentials and affiliations with financial organizations, the nature of their conversations was very different from the type of discussions the client had been hoping to find. Specifically, those blogging about this category of mutual funds were considerably more concerned with the values and ethics of existing and prospective investment vehicles than they were about the advisor-investor relationship and discussions about how SRIs could fit into existing client portfolios. While the former was interesting for the client, the absence of the latter really diminished the usefulness of social media data in terms of satisfying the client’s objectives as they related to the advisor population. Further, as we were trying to refine our social media search queries to broaden our data set and (hopefully) include more relevant conversations involving advisors, our team was sinking a number of extra hours into the data collection and cleaning phases of our methodology.

Never satisfied with the uncertainty of missing data – we ran a short poll of financial advisors in order to better understand their absence in social media. A quick poll using the Environics’ Advisor Research Panel confirmed for us that many investment advisors were not using social media in a professional context – neither to communicate with clients nor to stay informed about new products or trends in the industry. In fact, almost a quarter of advisors went so far as to say that social media was a compliance headache that was best avoided outright.

Thankfully, these results gave us some context for understanding how this target population was engaging (or, in this case, was not engaging) in social media and helped explained why they were greatly under-represented in our dataset. Without this supplementary understanding, we might have felt like our research was incomplete or left our client with some unanswered questions. This experience also reinforced to us that, while the use of social media has grown exponentially in recent years, it’s not yet an integral part of how every target group communicates.

This project also emphasized to us the importance of understanding what social media platforms were most suitable to monitor our target populations. When embarking on this work, we were fairly certain that Facebook was unlikely to yield much insight on investing and financial services topics. Similarly, Twitter and other microblogging mediums didn’t seem like sufficient platforms for something as complex as mutual fund investing (still, we do think Twitter has relevancy in financial services research as a whole). Ultimately, we ended up focusing on blog content almost exclusively, as the medium had the greatest potential for listening to category exports discuss in great detail the pros and cons of various financial products. This experience prompted us to think more critically about what mediums we would consider as part of future research. As was the case with this project, we now recommend to clients that they focus on blog or Twitter data when looking to understand the views of opinion leaders or stakeholders, and reserve Facebook when looking at issues of more relevance to mass-market consumers/general population (in addition to Twitter). Similarly, when looking to understand users of specific consumer products or ‘brand fans,’ we turn to Facebook and Twitter in addition to review sites like TripAdvisor or Amazon when trying to mine conversations centred on a particular product or service.

Social media can provide new and unique perspectives on business problems in many different sectors, but the unstructured and organic nature of social media conversations can make it challenging and time-consuming to find the most relevant conversations unless you possess a thorough understanding of how your target population is using social media. Thus, often the best place to start with social media research is to begin by researching how your target population is engaging in social media itself.

In our next post, we’ll talk about some of the challenges of analyzing social media data, including the pros and cons of sentiment analysis.

There has been an abundance of discussion in the market research industry over the past couple of years regarding the growing significance of social media in understanding of consumer perceptions and behaviours. But for all the attention the subject gets, our industry has done an incredibly poor job of sharing their experiences working with social media and has left both our clients and our colleagues with much uncertainty about the potential applications for this powerful medium. Over the next couple of days we’ll be sharing some lessons we’ve learned over the past year about using social media to solve real client problems. This information is intended to help marketers and market researchers better understand how social media can help you close gaps in your research objectives and gain deeper insight into the perceptions and behaviours of your customers and stakeholders.

Lesson #1 – Deciding whether social media fits your research objectives largely depends on whether or not social media is an appropriate medium to access your target population.

Lesson #2 – Understanding the nature and context of social media conversations is just as important as their prevailing sentiment.

Lesson #3 — The more sophisticated the topic, the greater the role social media can play in bridging the gap between target populations.