Social Media and BI are the sweet and sour, yin and yang, oil and vinegar topics of interest in BI these days. Can the real-time, user-generated, free flowing tweets and online conversations of social media benefit traditional enterprise BI?
In the past week, Information Management published the following, Social Media Will Play a Big Part in BI’s Future.
No doubt the volume of social media is growing exponentially. And surely, this data contains valuable information on competitive intelligence, product feedback, customer service, and even market trends.
But there’s a gap in social media data access. Traditional BI tools can’t access all this unstructured data and present it in a usable format, let alone filter out all the noise.
What’s needed is an automated, flexible way to access hundreds or even thousands of sites in real-time, extract only the relevant content, add structure to the data, and load it easily into a database. What’s needed is Web Data Services, and it exists today.
Social Media Data Access:
With hundreds of sites to monitor (most having no API access) and an already overburdened IT department, accessing social media data becomes the foremost hurdle to overcome. With Web Data Services, all of this can be achieved with no coding. Kapow robots (automated data collection processes) are easily created with visual point-and-click technology eliminating the need for complex, time-consuming coding and scripting. If you can see the data in a web browser, Web Data Services can extract it.
Enriching Unstructured Data:
The trick is taking disparate text based tweets, comments, blog posts, online conversations, etc. and structuring them in a way that lets your analyst understand when it occurred, who said it, and how it applies to your keywords or hypothesis. But getting there is harder than you might think. Web Data Services surgically transforms unstructured social media web data to provide superior data quality without the noise. Included, but not often talked about, is the ability to perform regular expressions (through a graphical interface), encoding and decoding, date formatting, string calculations, conditional expressions, numeric calculations, and multiple language support.
Making the data readily available:
Web Data Services makes it easy to output the structured social media data into multiple formats, such as a SQL database, vendor hosted database, Java or C# data structure, SOAP or REST Web service, RSS, CSV, or XML.
Social media is BI 2.0. It opens the doors to listen in on what people are saying about your brand, products and services, and also taps into untapped market opportunities and customer pain points. So rather than reacting, you are out in front predicting future events and gaining first mover advantage.
By: Rick Kawamura 
