Kapow Katalyst Blog

There has been a lot of buzz around the increasing need for APIs, especially around “free” and “open” APIs.

As Dion Hinchcliffe writes in his latest blog on ebizQ, (Open APIs Mature Into a Next-Generation Business Model and Is the Future of SOA Open Source?), APIs (or the lack thereof) are the biggest obstacle for developing next-generation business applications and SOA interoperability.

Why?  First, the APIs have to be available, but they also need to be simpler (for example using REST) and easier to consume by BI tools, agile application environments and mashup builders.

Dion describes how more and more companies are providing open or free APIs to their data as an important part of their business model. These APIs are supplemented by a new line of companies, like StrikeIron and Xignite, who provide APIs to other’s data through an easy data-as-a-service (DaaS) model (check out this article in WSJ, The New Information Goldmine).

This is well aligned with the Open Government Initiative and the new US government data site data.gov where more and more government data will become available. You can read more about this initiative in this article about Digital Democracy.

But can we realistically wait for all relevant data to become “API enabled”?

With more than 5 billion websites today, there is a vast amount of growing, relevant data that is not going to have an API any time soon – if ever. Add to this data locked internally in legacy applications and at your business partners and you can see how unrealistic it is to have APIs for all this data.

This is where Web Data Services and a product like the Kapow Web Data Server become critical.

Web Data Services allow business analysts and agile application developers to instantly create APIs where none exist. The only prerequisite is that you can navigate to and see the data in a normal web browser like FireFox, IE or Safari. This even includes data behind secure, password protected sites, and data on very complex websites powered by AJAX and Flash.

With the powerful combination of open APIs, free government data and the ability to rapidly turn any web application into an API on-demand, we finally have access to any data we need.

This lays the foundation to a new way of working, where business analysts and other decision makers can spend their time building better algorithms, better data visualization, and better analysis because the most critical ingredient behind any business decision today, the data, has become so easily accessible.

By:  Stefan Andreasen

Content Migration is hard work.  Dealing with thousands of HTML pages can be costly, time-consuming, and painful.  At the CM Pros and Gilbane Boston event last week, we enjoyed the focus on Content Migration.  One workshop in particular, Content Migration – the Dirty Little Secret of Content Management, provided a thorough and comprehensive investigation of  various approaches to succeed with enterprise content migration projects and to overcome the numerous hurdles and headaches associated with content migration.  For a summary of the workshop, check out Irina Guseva’s write-up at CMSWire.

One clear message from the workshop is that automation is clearly a winning strategy when it comes to complex migrations involving unstructured content and large volumes of HTML pages.

Key Benefits of Automated Content Migration:

1.    Save Time – Cut time to market up to 90% through automated data access, extraction, cleansing, transformation and QA.  Eliminate content freezes.  Accelerate your time to market for your production CMS.
2.    Save Money – Reduce manual, error-prone processes and cut resource requirements saving up to 80%.
3.    Accuracy -  Look for error detection and the ability to integrate complex JavaScript and AJAX content, as well as on-the-fly exception handling through real-time error identification.
4.    Project Completeness – Finish projects ahead of schedule, within budget, and no disruption to operations.

The alternative?  Manual cut-and-paste and black-box approaches which are complex, error-prone, expensive, and often times unable to complete the job.

There are a few vendors making bold promises around content migration.  We’ve stepped in and had to finish several of their projects because their tools and processes couldn’t get the whole project done and the client was losing their patience with the multiple delays and increasing costs.

Here are a few things to consider when choosing a Content Migration vendor and solution:

1.   Automation is King – Intel was able to reduce the number of developers on a large content migration project from 45 down to 2.  Check out how Intel reduced costs 80% and got their content migration with Kapow Technologies done in 1/10th the time.
2.    Don’t waste your time custom programming to a CMS’s APIs. Especially in cases where APIs don’t exits, visual UIs like Kapow’s provide full access to all functionality for proper loading of your content.
3.    Place your trust in “No Coding” through a robust Visual IDE which will save you peace of mind, frustration, headaches, and maybe even your job (just kidding).  Which is to say, avoid coding with JYTHON for transforming your data.
4.    Insist on Real-time Error Checking – don’t get caught in the ongoing process of checking your logs or QA’ing your site, only to find out it’s broken and then having to re-write your script and start again.

Kapow Technologies offers the industry’s ONLY solution to completely automate content migration.  Learn more on our website.  Here are three great resources to get you started:

1.    Whitepaper:  The Definitive Guide to Automating Content Migration
2.    Webinar:  A new approach to Content Migration – A use case featuring Intel
3.    Demo:  Extract data and load into CMS without APIs or coding

As a frequent traveler I often stay at Marriott hotels and noticed they recently upgraded and redesigned their website.

Why is this interesting in a blog about Web Data Services?

Marriott has just done what so many other companies are doing these days – they are modernizing their website with a user-friendly, dynamic AJAX-based interface to enhance user experience.

While AJAX is helpful for creating interactive web applications like a hotel reservation system, it’s very bad news for business users who depend on collecting web data with home grown scripts and primitive web scrapers to empower business applications like Market Intelligence, Financial Research, and Buzz Analytics.

But don’t despair. Kapow Technologies just released version 7.1 of Kapow Web Data Server which includes support for even the most sophisticated AJAX toolkits, including Google Web ToolKit.

That said I couldn’t write this blog post without testing it out myself.

So I powered up RoboMaker 7.1, typed in www.mariott.com, and did a simple search for hotels near San Francisco airport.

Then a dynamic map (powered by Microsoft bing) appeared showing the locations of the 10 nearest Marriott hotels.  I wanted to create a loop over the ten hotels on the map, so I simply clicked on hotel number 1, clicked the insert loop command, and in a few minutes I had created a Kapow robot that could extract hotels directly from a highly dynamic, AJAX-based map.

Tell me about any other product on the planet that can do this in 2 minutes!

Check out the picture below, and be sure to think about Kapow Web Data Server when your current Web Data Extraction tools break in the world of modern AJAX powered web sites.

By:  Stefan Andreasen