Research shows automated analysis “less accurate than flipping a coin”

Research shows automated analysis “less accurate than flipping a coin”

Research from FreshMinds has revealed the poor accuracy levels of automated sentiment tools.

  • FreshMinds conducted an experiment where they compared automatically coded data with manually coded data
  • Compared with manually coded data, the automatic tools were on average only 30% accurate.
  • The research tested tools from Nielsen, Brandwatch, Radian6, Alterian, Scoutlabs and Sysomos. The tools recorded data on the Starbucks brand.
 
FreshMinds stated “we were shocked that, without ‘training the tools’, they could be so wrong... For brands, the positive and negative conversations are of most importance and it is here that automated sentiment analysis really fails.”
 
Wavemetrix classifies all posts using human analysts. This allows us to understand, rather than read opinion. We believe that spending the time and money to code data manually has several key advantages.
 
A full copy of the report can be downloaded from the FreshMinds blog.

 

Add comment

The content of this field is kept private and will not be shown publicly.
By submitting this form, you accept the Mollom privacy policy.
  • Industry
    Stay
  • Social Platform
    Stay
  • Themes
    Stay