Over recent months, the increasingly common theme within conversations regarding social media monitoring has been the 'man v. machine' debate. This is hardly a surprise – as the social media industry matures, and users gain more familiarity with the many monitoring tools on offer (over 200 at the last count), the flaws inherent in the less sophisticated ones becoming increasingly exposed.

In the old, pre-internet/online tracking days, a great many organisations used 'clipping' services that employed people to go through newspapers and magazines manually, as well as radio and TV broadcasts. It wasn't an easy process to automate and, as a result, it wasn't cheap. Now, technology has diminished the requirement for human involvement to a huge extent, and 'upfront' costs have fallen as a result.

Social media monitoring/tracking technology can undertake the vast majority of the grunt work by searching, collating, processing and organising millions of conversations – a task that's simply beyond the capability of people, given the sheer volume of data involved. But – and it's a pretty big 'but' – technology can’t completely replace human intervention, and nor should it. In short, it doesn’t much matter which of the many social media monitoring tools you choose to use, because the simple fact is that few of the tools really works … in the sense that most tools either don’t do what they claim to do, or they do it badly.

The central problem is that, in many cases, the tools are pretty stupid. Whilst some have a degree of sophisticated intelligence, they're only as intelligent as the analyst operating them. Most tools are essentially software platforms – genetic algorithms and search spiders that collate data and assemble it in an environment where you can find it easily. Some of them do a decent job of allowing you to access the data through an attractive dashboard interface and play it back via some funky charts and graphs, of course. But very few tell you the two vital things you really need to know: i) what it all means and ii) what you should do about it, strategically and tactically.

Why not? Because this stuff isn't easy. It takes hard work from highly experienced analysts who really understand the tools they're using (not least because, in the best firms, the analysts will have developed the tools themselves), and the environment in which they're operating. Indeed, the most common mistake we see brands making in this space is to put most of their efforts into tool selection whilst skimping on analysis. To really get to the bottom of what is being summarised in all of those charts and graphs, someone is going to have to analyse the actual conversations to make sure that any output is a genuinely accurate portrayal of what is going on and, more importantly, distil it into some actionable insights.

Worryingly, however, the monitoring of social media is often placed in the hands of junior, non-analytically trained employees because organisations – vendors and clients – misguidedly believe the tools themselves contain the value. They don't. In fact, the commonly accepted view amongst the expert community is that, for every $100 you have available, you should spend $10 on the tool and $90 on the analyst. Here's why.

1. Are you looking for the right stuff?
If your brand name is fairly generic (Orange, Virgin, Egg etc), you have a big problem as many painful hours of configuration and exclusion await you. Based on keyword configuration, the web crawler will pick up any mention of your keywords even if they're within a totally irrelevant context. Without the ability to construct robust, accurate and comprehensive taxonomies before embarking on your exploration, and then refining them over time, you're going to be wasting time and money wading through conversational treacle – in short, looking for the wrong needle in the wrong haystack. Getting this right upfront is critical.

2. Be sceptical, particularly about sentiment.
Of all the things that social media monitoring tools claim to be able to do, sentiment analysis is their most acute flaw, for the simple reason that the nuances of the written word are extremely difficult to detect. Sarcasm, to pick just one facet, is notoriously tricky to identify correctly through any form of automated tool. And be especially wary of vendors who claim to enhance accuracy through manual sentiment analysis across all posts. They can't. Just do the maths … the sheer volume of conversations makes this physically impossible for anything beyond a representative sample.

3. Can you accurately decode complex posts?
Most conversational threads are elaborate and protracted – in many instances, large numbers of participants will be contributing, and to varying degrees of detail. Is the sentiment algorithm within your tool sufficiently sensitive to decode individual posts which contain both positive and negative sentiment? And as the conversational thread develops, it's likely that your brand name will cease to be stated explicitly – again, can your tool ascribe non-branded sentiment correctly?

4. How global is your understanding?
The web is blissfully ignorant of geographical boundaries – are you confident your tracking tool is similarly unconstrained? It's not that online conversations tend to take place in multiple languages (unsurprisingly, the vast majority are in English), but more that individual posters will display the phraseological idiosyncrasies of their mother tongue. Interpreting these conversations is far from easy.

5. Can you demonstrate genuine capability in category verticals?
Most vendors have a broad reach across multiple industries by being agnostic to particular verticals. Many categories are highly complex, however – eg telco, tech, financial services, energy etc – and require genuinely deep understanding, not least because they have their own language, terminology and jargon. Can your vendor really deliver the level of understanding required?

6. Bespoke or off-the-shelf?
It's pretty rare to find two brands (even if they're in the same category) with identical tracking needs in the social media space. Accordingly, it's important you select a provider that's capable of tailoring its offering to your particular – what track, where you track it, how (and how frequently) you report it, how you access it etc. Most software-only tools tend to be sold on a shrink-wrapped, take it or leave it basis – it's unlikely that they'll be able to deliver what you really need now, let alone evolve with you as your needs expand.

7. How confident are you about data integrity?
In a data-driven field, data integrity is the sine qua non. Does your provider guarantee that its data are current, comprehensive and accurately documented to their source? Does it perform its data mining through proprietary systems directly from the source, or does it rely on third parties? And are data stored locally, enabling them to perform time-series analysis on data which are long departed from their original source? If not, you might want to think again.

8. Information or insights?
Even with the most sophisticated tools and dedicated social media analysts, you need to be delivering actionable insights not just information for your investment to deliver a solid return. What are consumers saying about you, and why? Where are you winning and losing relative to your competitors? What are the issues specific to your brand as opposed to the category? How can you best engage with and influence consumers? How are you going to integrate social media into your organisation – from quantitative brand tracking to customer services? Answering these sorts of questions early in the game will help you better understand the social media journey on which you need to embark. If you're new to social media, working with a full service partner is critical to avoid wasting both time and money.

9. Is the board listening?
How do you get CxO buy-in to social media? Without doubt, you are going to have to establish whether social media merits CxO visibility in your organisation/category. Most large brands are the subject of thousands of consumer conversations every month across a wide range of topics – is there a demonstrable value to the organisation in listening to these? More importantly, what are the commercial consequences of not listening, especially if your competitors are not only listening but actively engaging? It doesn't take long to understand your and your competitors' current position in social media. If you take the time to do this robustly, you'll know pretty quickly where the value lies … and that's something your board are going to want to hear. 

10. And finally, are you on your own?
As ever in life, you get what you pay for, and the cost/benefit equation is something you'll need to understand fully before making any commitment – there's a big difference between buying a stand-alone product and buying an integrated service. If you have the time, the inclination and the ability to run the software yourself, and interpret the findings independently, a desktop tool might just suffice. You'll be on your own because the software provider business model doesn't allow for very much at all in the way of ongoing service or support – which largely explains why they're so cheap! Choose a full-service monitoring and analytics provider, however, and they're likely to work with you, 24/7 –across time zones and language barriers, to build a bespoke tool and reporting dashboard, refine it over time, give you detailed and comprehensive debriefs at an agreed frequency, interpret the resultant findings, turn those findings into actionable insights and work with you to implement them.

So, there you have it – our Top 10 reasons why you should think seriously before rejecting a full service provider for the false economy of a software tool. But then again, they are cheap …