Looking at the 2011 trends from ProgrammableWeb, the number of available APIs registered in their API Directory has once again doubled. What I find most interesting is that the REST web services now outnumber the SOAP web services by a factor of four.

Even Salesforce decided their original SOAP API could no longer stand alone and added a REST API in late 2010, acknowledging that REST is now the preferred choice for application integration due to easy implementation and deployment. Its lightweight nature is a good fit when creating mobile or web-based applications, both since the client app doesn’t need extensive libraries to be able to interact with the server and because SOAP has a much higher overhead in terms of bandwidth, which can negatively affect the perceived performance of the app.

Internally in many enterprises, SOAP is still used as the protocol over which the services in a service-oriented architecture (SOA) interact. However, despite the similarity of the acronyms, the components in a service-oriented architecture can just as easily interact using REST or other protocols. In fact, as many companies begin exploring hybrid integrations, leveraging the strengths of both in-house and cloud applications, it becomes crucial for any successful integration product to be able to consume both SOAP and REST web services.

In a hybrid world, the type safety of SOAP web services is of less importance, as the data transferred between cloud and in-house applications will most likely need to be validated and reformatted at the integration point anyway. Also, REST web services lend themselves easily to caching — riding on standard mechanisms — which can be important when the traffic between in-house systems and private and public clouds continues to increase.

My prediction is that in 2012, any CIO who is considering API-enabling an application — either to consume the service internally or to expose external APIs for partners and mobile or web applications — will be looking first and foremost at REST as the technology of choice.

By: Anne-Sofie Nielsen

Social Media and BI are the sweet and sour, yin and yang, oil and vinegar topics of interest in BI these days.  Can the real-time, user-generated, free flowing tweets and online conversations of social media benefit traditional enterprise BI?

In the past week, Information Management published the following, Social Media Will Play a Big Part in BI’s Future.

No doubt the volume of social media is growing exponentially.  And surely, this data contains valuable information on competitive intelligence, product feedback, customer service, and even market trends.

But there’s a gap in social media data access.  Traditional BI tools can’t access all this unstructured data and present it in a usable format, let alone filter out all the noise.

What’s needed is an automated, flexible way to access hundreds or even thousands of sites in real-time, extract only the relevant content, add structure to the data, and load it easily into a database.  What’s needed is Web Data Services, and it exists today.

Social Media Data Access:

With hundreds of sites to monitor (most having no API access) and an already overburdened IT department, accessing social media data becomes the foremost hurdle to overcome.  With Web Data Services, all of this can be achieved with no coding.  Kapow robots (automated data collection processes) are easily created with visual point-and-click technology eliminating the need for complex, time-consuming coding and scripting.  If you can see the data in a web browser, Web Data Services can extract it.

Enriching Unstructured Data:

The trick is taking disparate text based tweets, comments, blog posts, online conversations, etc. and structuring them in a way that lets your analyst understand when it occurred, who said it, and how it applies to your keywords or hypothesis.  But getting there is harder than you might think.  Web Data Services surgically transforms unstructured social media web data to provide superior data quality without the noise.  Included, but not often talked about, is the ability to perform regular expressions (through a graphical interface), encoding and decoding, date formatting, string calculations, conditional expressions, numeric calculations, and multiple language support.

Making the data readily available:

Web Data Services makes it easy to output the structured social media data into multiple formats, such as a SQL database, vendor hosted database, Java or C# data structure, SOAP or REST Web service, RSS, CSV, or XML.

Social media is BI 2.0. It opens the doors to listen in on what people are saying about your brand, products and services, and also taps into untapped market opportunities and customer pain points.  So rather than reacting, you are out in front predicting future events and gaining first mover advantage.

By:  Rick Kawamura

I just read an interesting blog post by Richard MacManus, “10 Ideas For Web of Data Apps” that explains the fundamentals of how people generate ideas for new applications.

My guess is that all examples described in the blog came from people combining what they saw on multiple websites into an idea of how to leverage the data in a new, valuable application.

However there is a problem here!

It’s not a given that all that data is available as Linked or Open data.  In other words, not all data necessarily has a documented method of programmatic access as, for example, an XML feed, RSS feed, or a REST or SOAP service. Without this programmatic access, no existing application or Mashup builder can get to the data which prevents these great ideas from ever materializing. WHAT A BUMMER!

More often than not the data you need to combine into your great new application idea is only available in a web browser. This means you have to either drop the idea or settle for a subset of the data available with documented programmatic access.

Wouldn’t it be cool if all the data you see in a web browser were always available?

Well that is what Web Data Services is all about. Check more of this blog to learn more.

As always, please send me your comments, my email is sa at kapowtech.com

By:  Stefan Andreasen