I just read an interesting blog post by Richard MacManus, “10 Ideas For Web of Data Apps” that explains the fundamentals of how people generate ideas for new applications.

My guess is that all examples described in the blog came from people combining what they saw on multiple websites into an idea of how to leverage the data in a new, valuable application.

However there is a problem here!

It’s not a given that all that data is available as Linked or Open data.  In other words, not all data necessarily has a documented method of programmatic access as, for example, an XML feed, RSS feed, or a REST or SOAP service. Without this programmatic access, no existing application or Mashup builder can get to the data which prevents these great ideas from ever materializing. WHAT A BUMMER!

More often than not the data you need to combine into your great new application idea is only available in a web browser. This means you have to either drop the idea or settle for a subset of the data available with documented programmatic access.

Wouldn’t it be cool if all the data you see in a web browser were always available?

Well that is what Web Data Services is all about. Check more of this blog to learn more.

As always, please send me your comments, my email is sa at kapowtech.com

By:  Stefan Andreasen

Many of you have asked for a deeper dive on the meaning of “Web Data Services”, so let me answer it here.

First, it’s important to understand the terms “Data Integration” and “Application Integration”.

Data Integration (DI) and (Enterprise) Application Integration (EAI) are not the same, though many vendors often confuse the two. Application Integration focuses on managing transactions or messages between applications while Data Integration focuses on managing the flow of data and providing standardized API’s to access the information. For more details, refer to Mark Madsen’s blog, Key Differences Between Data Integration and App Integration.

There are essentially three different types of Data Integration:

• Consolidation means moving all the data from the original data sources to a new repository, much like an ETL tool.

• Propagation means moving only the necessary data to a local storage for each application consuming the data.

• Federation means leaving the data at the original source and accessing it as needed in real-time.

Web Data Services is in reality all forms of Data Integration as well as Application Integration, with two distinct differences.  With Web Data Services:

• You primarily access data and business logic residing on the web (any application or data source you can access from a Web browser like Internet Explorer, Firefox or Safari). This includes applications inside your own organization and even at your business partners.

• You do not need to recode or have programmatic access to any of the data sources. As long as you have access from a Web Browser, you can access the data with no coding and be up and running in a matter of hours rather than weeks or months.

Web Data Services is the new highly productive way to access almost any of the data you need for Business Intelligence (BI), Data Validation and Acquisition, Enterprise Mashups, Partner Integration, or basically any solution that needs agile access to data or business logic. Web Data Services gives unheard of business agility and competitive advantage compared to traditional Data Integration or Enterprise Application Integration methods.

Try it free with our Kapow Web Data Server Trial Offer

Also check the Wikipedia entry on Web Data Services to read the definition from leading industry analysts.

By:  Stefan Andreasen

I’m excited about the recent buzz around real-time data and its importance in driving today’s business decisions.

I recently attended both the Web 2.0 Summit and IBM’s Information On Demand (IOD) conference.  I found it interesting that two very different venues both put a lot of focus on the growing importance of real-time data

At IOD, Stephen Mills, SVP and Group Executive of IBM SWG, was asked about the single most exciting, innovative aspect of Information Management today.  He replied that applications today are more predictive because they leverage more real-time data than ever.  Technology enables us to deal with larger quantities of data and apply intelligence in real-time to predict the future (and prevent catastrophes such as power outages, epidemics, and terrorist attacks), which help us do a better job of planning our businesses.  The net-net is making a “smarter planet” for the future.

Which reminds me of this “IBM Smart Products:  Creating Insight for your business” video where I talk about the importance of reporting with relevant and timely data to make your business more competitive.

The Web 2.0 Summit also placed a strong emphasis on the power of real-time data.  The closing session featured Tim Berners-Lee and Tim O’Reilly discussing the future of the internet. Tim B-L mentioned how he originally invented the World Wide Web based on a fellowship at CERN where he worked on distributed real-time systems for scientific data acquisition.  Basically the internet was invented to be a “sandbox” to find and share data. The discussion then moved to how the Web one day will be powering dynamic applications or widgets that can connect to any data available on the web. I could not help equating this to mashups connecting real-time to real-time web data.

My prediction is that the business landscape is becoming ever more competitive and distributed and real-time web data will be a fundamental necessity for businesses to make real-time predictions.

By:  Stefan Andreasen

Consider this scenario:  A large tanker carrying hazardous material suffers propulsion system failure in the Gulf of Mexico.  How does one actively manage this emergency in real-time and what is the correct course of action to get the vessel safely to the nearest port, avoid further disasters, and alert nearby ships, oil platforms, and environmental agencies?  The smart guys over at IBM created a very compelling Maritime Domain Situational Awareness Mashup using three key principles:

•    Discovery, normalization, and fusion of multiple disparate data sources
•    Display multiple layers of related data in a geospatial map aiding situational awareness and analysis
•    Applies business rules to detect situations; automate task procedures, and guide user interaction where dictated by procedure, best practices, or workflow

IBM defines Situational Awareness as “the understanding of data and events within a given space and time and understanding the meaning in the present and near future [and] deal with multiple sources of data… often rendered in a… map”.  Needless to say, pulling real-time web data from multiple sources to make time-critical decisions can be applied not only to emergency situations, but also to anti-terrorism, financial forecasting, price monitoring, competitive intelligence and fraud detection, just to name a few.

What is interesting here is the critical dependency on Web Data to deliver the full value of a Mashup. All the data used in the Maritime example (vessel information, weather, tide schedules, port locations, and nearby ship routes) is only available through public websites (most of which have no APIs).

The real-time access and REST service-enablement to this critical, situational data  was accomplished with Kapow OnDemand.

This is just another example of how Web Data Services can be used to power mission critical Situational Awareness by leveraging the gold-mine of data sources available on the Web (including public web, as well as federal, government and partner web sites).

We’re proud to the have creators of this mashup joining us in our booth at the IBM Information on Demand Conference to show this compelling demonstration.  If you are at the show, come by and check it out.

By:  Stefan Andreasen

Please tune in to hear Stefan and Jim Kobielus on Dana Gardner’s Briefings Direct Podcast, ”Web data services extend data access and distribution beyond the RDB-BI straitjacket”.  Dana moderates an intriguing discussion on making the most of Web Data Services for Business Intelligence, focusing on web data volume, relevancy and timeliness, as well as access, monetization, enablement, governance, security, and the unification and converging of structured and unstructured data.  And in looking towards the future, Jim and Stefan weigh in on the impact of cloud computing on Web Data Service tools.

Jim is a senior analyst at Forrester Research, and an expert on data warehousing, advanced analytics, and business intelligence.

Enjoy the Podcast!

Today, the founding of the Open Mashup Alliance (OMA) was officially announced to promote the interoperability and adoption of Enterprise Mashup Solutions.

This is an important step towards growing the understanding in the market of the value of Enterprise Mashups, as evidenced by the quality of companies supporting the Alliance: Adobe, Bank of America, Capgemini, Hinchcliffe & Company, HP, Intel, JackBe, Kapow Technologies, ProgrammableWeb, Synteractive, and Xignite.

Dion Hinchcliffe has written several excellent articles on the OMA and Mashups in the Enterprise over on his blog at ZDNet that would be well worth your time to check out.  In addition, consider this article by John Waters on Application Development Trends.

Dion sheds light on the origin of mashups and quite possibly the reason why many view them as lightweight consumer applications:  “In decades past, the new ideas in computing originated in the enterprise world and trickled down to the consumer world….  However in the Web 2.0 era… new ideas and approaches are germinating more on the consumer Web…  Mashups are example of this kind of hacker-style creation that emerged from the laboratory of the Web…  From an enterprise perspective, it gave a lot of people pause to see how easy it could be done (Paul Rademacher’s HousingMaps.com being the original example), compared to the methods used by formal and costly enterprise application integration and service-oriented architecture projects.”

The potential for Mashups in the Enterprise are bounded only by your imagination.  If you’re interested in learning more, please check out this webinar on solving real-world business problems to give your workers a competitive advantage with Enterprise Mashups.  Also, here is a demo on how to mashup data from multiple websites like Linkedin or Jigsaw to build a sales dashboard that enables sales executives to harvest relevant, valuable web data about existing and potential clients.  And lastly, learn more about how Cisco uses Mashups in the Enterprise.

If that’s not enough to whet your appetite for Enterprise Mashups, Business Insights just released their report, The Future of Enterprise Mashups: Demand, challenges and vendor opportunities, where they predict the mashup market will grow from $161M in 2008 to $1.74B by 2013. That’s a ten-fold increase in 5 years!

Alliances like OMA will certainly help to accelerate this to where Mashups will soon be mainstream.

By:  Stefan Andreasen

Our “No Coding” tagline has resonated with many of you.  Thank you for the support and positive comments in the media and your blogs.  Disruptive technologies often have the opposite effect e.g. making people uncomfortable.  The Kapow Web Data Server is truly a paradigm shift – radically changing how traditionally complex migration of data between source and destination systems or turning applications into web services is done.

Many customers have described to me the “aha” moment they’ve had.  Across the board, our customers kept saying to us, “the amazing thing about the Web Data Server is that it requires no coding”.  It’s great when your customers do your marketing for you!

Two revolutionary inventions with the Kapow Web Data Server are key to our ability to eliminate traditional coding associated with Business Intelligence, Data Integration, and application integration projects:

1.    Existing applications can be easily accessed as-is through their HTML/AJAX front-end through our point-and-click visual development tool, eliminating the need to deal with proprietary APIs.  Even better, it’s done in the context of the content owner or business user (through what they see in their web browser), eliminating any misunderstandings that occur when explaining to coders what needs to be done i.e. translating business/ functional requirements to technical specs.

2.    Instant, visual feedback allows results to be seen on the spot – delivering high productivity and time to market improvements.

We have several videos on our website demonstrating how users easily access, enrich, and serve web data with “no coding”:

•    Harvest the web for pricing and market intelligence

•    Access and serve web data from Linkedin, Jigsaw and more with no coding

In fact, the tool is intuitive and easy enough that even our marketers have been known to build their own Kapow robots to harvest web data.

But programmers love our product even more.  Why?  It makes them more productive.  With a visual interface, real-time feedback, and 100% accuracy, programmers get more done in a lot less time.

All Content Migration, data warehouse ETL, data integration and application development projects require an iterative process, from first attempt until solved.

With traditional methods, each iteration can take weeks and even months to get it right.  This is cumbersome and frustrating for most.

The Kapow Web Data Server cuts down each iteration to 3 simple steps (measured in minutes):

1.    Visually instruct what you want (without coding), just like working with the data in a web browser (point and click).

2.    See the end result in real-time, right in the middle of instructing how and what to do.  (Since this happens real-time, step 1 and 2 are basically the same… but would you  believe me if I called it a 2 step process?)

3.    Modify instructions based on the instant, visual feedback, i.e. go back to step 1.

This extremely quick turn-around per iteration with the Kapow Web Data Server allows you to get to 100% of what you want.  100% accuracy in 1/10th the time.  And did I mention “no coding”?

Don’t just take my word for it, please check out what Intel and Audi had to say about Kapow.

Technology continues to evolve.  The market demands it.  Register for our free trial.  It really works.

By:  Stefan Andreasen

Susan Bouchard, who leads the Web 2.0 and mobility program for Cisco and is author of the book Enterprise Web 2.0 Fundamentals, is an Enterprise Web 2.0 Expert and Blogger at Network World.  She recently wrote a great post titled Delivering Business Value with Mashups where she describes how “Mashups offer several key advantages and significant business value to the enterprise”.  She highlights how Cisco “uses Kapow robots to aggregate links to selling content from Cisco’s product and marketing business unit sites, eliminating the need for users to visit multiple sites and reducing search time” as a prime example of Cisco using Kapow Technologies to deliver business value.

Thanks Susan, great post!