Kapow Katalyst Blog

Never before has it been more important for CEOs and CIOs to leverage information technology to automate their businesses. Increasing the operational efficiency of the existing workforce will be one of the primary drivers of business growth for years to come, and IT will be the means to deliver this.

This demand for process automation is creating a boost for BPM (Business Process Management) and BPA (Business Process Automation) products. Products like IBM Business Process Manager, Oracle Business Process Management or the new Visual Workflow from Salesforce.

So what does automation have to do with API-enabled applications?

Well, in order to improve workforce efficiency, business processes must be streamlined and manual processes must be fully automated.

This means that every application in a business process flow must be able to talk to other applications automatically, and until now the only way applications could talk to each other was through APIs.

So if you had no APIs, you could forget about automation. If you have been using any of the standard BPM products on the market, you know what I’m talking about. Basically this barrier is the biggest flaw with all of the existing BPM products — they depend on APIs that often don’t exist.

And it gets worse:

New web-based business apps are being cranked out at higher and higher volume. These applications are everywhere: inside your business, residing in the cloud (SaaS apps), and at your business partners’ or customers’ (B2B apps) locations. These are the applications that power some of the most important processes in your business:

1. Pricing Comparison
2. HR Processes
3. B2B Processes
4. Financial Account Aggregation
5. Supply Chain Optimization
6. Data Collection
7. Regulatory Compliance
8. Competitive Price Monitoring
9. Social Media Monitoring
10. Marketing Intelligence
11. Financial/Equity Research
12. Legal Evidence Collection
13. Management Reporting

These applications, more often than not, do not have documented APIs, making integration and, therefore, business process automation, almost impossible until now.

But there is a new approach to quickly API-enable any application in order to automate your business processes.

An enterprise-class application integration platform like Kapow Katalyst features a built-in Automation Browser that lets you wrap any web application as if it already had a documented API — essentially API-enabling the application and making it ready for those important automation projects.

This not only eliminates the need to rewrite internal applications, but also eliminates the challenge of working with external applications you cannot rewrite such as B2B and SaaS apps. With Kapow Katalyst you can create a complete integration or automation solution extremely efficiently.

Back to the Myth:

To automate business processes you need your applications to be API-enabled.

True for standard BPM products, False if you use Kapow Katalyst.

Let’s improve business efficiency today.

Stefan Andreasen, founder and CTO, Kapow Software

By: Stefan Andreasen

Companies across the world are employing lean enterprise practices to eliminate “wasteful” activities, i.e. activities that are not adding value. However, one is often left with a number of indispensable, rote tasks that consume staff resources but do not easily lend themselves to further optimizations. Should we as managers be satisfied with this? No — with any kind of repetitive task we should be looking for a way to automate it.

Consider how much time employees waste every day keeping systems in sync because no proper APIs or integration points exist between these systems. Take for instance the US Army Corps of Engineers that was facing the challenge of coordinating payments to contractors and employees across 13 districts, each with its own project management system. Using the Kapow Katalyst platform for process automation, they set up automated synchronization between the local systems and the central systems, resulting in savings of $2 million.

Similarly, the Spanish call center provider Atento had an army of people employed to type in data from printed forms, as this had to be done each time a new customer account was set up. Again, with the Kapow Katalyst platform for process automation, the account processing could be done automatically — leading to both the re-allocation of staff to more productive tasks as well as an improvement in the data quality due to the elimination of human errors in the data entry.

This ties in nicely with iSixSigma’s top four metrics of success of a lean project:

• Revenues
• Costs
• Risk
• Customer Satisfaction

As illustrated by the cases above, process automation can contribute to all of these key areas. It enables the organization to scale; growing the top line without incurring proportional costs in staffing to meet back-office demands. Employees can be freed to spend the bulk of their time on activities that yield core business value, leading to a more focused and agile enterprise. This is lean in its purest form.

By: Anne-Sofie Nielsen

Leveraging investments in legacy apps with application integration

I’ll bet that most people who hear the term “legacy application” think of an outdated and complex on-premise application that is bound to old mainframe platforms, has an old-fashioned user interface, and is very difficult to integrate or modernize.

But in reality any application is “legacy” as soon as it goes into production and we no longer develop it to meet new business requirements.

Today, applications are cranked out faster than ever before to meet increasing business demands resulting in an explosion of legacy applications. The stereotypical definition in the first paragraph above is as outdated as the applications it describes.

The consequence is that most “new” legacy applications are most likely “modern” web apps running on Java or .NET based app servers. Not particularly old-fashioned but legacy apps nonetheless.

I’m talking about those homegrown intranet and extranet web apps found all over most large enterprises today. Applications developed by people no longer with the company or by a systems integrator you no longer work with.

I often hear comments along the lines of, “We have thousands of legacy web apps sitting around our company, and we have no clue how to integrate them!”

Sound familiar?

The situation requires an attitude change. We need to rethink what we can do with legacy applications. Legacy apps and the content they hold are something we can leverage, just like any other investment, rather than something we have to try to replace. Call it IT’s version of re-use or recycle.

But in order to leverage legacy apps efficiently, we need easy integration.

Instead of spending a couple of years and a million dollars recoding your legacy apps or waiting for your external business partner to provide that vital API, you could spend a few weeks and maybe a hundred thousand dollars and not write a line of code. The latter is now possible using a revolutionary approach to application integration.

A new class of application integration solutions eliminates the dependency on missing APIs and can rapidly deliver full programmatic application control to any legacy web application. If the application can be used in a web browser, these new application integration solutions can automatically extract, transform, integrate, and migrate that data and application logic, without an API. This approach gives IT a hyper-agile way to perform enterprise, web, and cloud data integrations — as well as mobile enablement. Why make it more complicated than it needs to be?

Start leveraging those legacy applications now… you can’t afford not to.

By: Stefan Andreasen

Everyone talks about Big Data, and specifically about the three “V” drivers for Big Data: Volume, Velocity and Variety.

But most forget about a fourth and equally important dimension of Big Data, which is the spread of data across many different sources. It’s not how much or how fast or what type the data is, as in the three Vs above, but WHERE the data is.

Just look at all the data posted on social media, forums, and blogs. And then consider the thousands of new websites popping up every day. We are facing the largest “data silo” problem ever, and the situation is only going to get worse.

The challenge of “data spread” is how can businesses access, transform, and integrate the data from these sources to turn it into actionable information for competitive advantage? Because, as pointed out in the recent Forrester research report Enterprise Hadoop: The Emerging Core Of Big Data, “This growing tsunami of intelligence feeds downstream business processes in both the front and back office, helping organizations optimize their interactions and operations through powerful analytics.” Not if they can’t communicate with the source!

I recently posted an article entitled The Fourth Dimension of Big Data Nobody Writes About to start a discussion on how to best address these challenges. I encourage you to read it and to add your insights along with the other perspectives already posted.

I believe that the Kapow Katalyst Application Integration Platform for Big Data is the way to deal with this exploding data silo problem. Our approach is already tested and proven by more than 500 companies around the world. It solves the data spread problem like no other approach.

If you have “data spread” problems you need to address, I encourage you to check out Kapow Katalyst, whether it’s for large-scale web data collection from social media websites, or API enabling of data hidden in applications all over your business network, intranet, and the Internet.

By: Stefan Andreasen

I’m flying back from New York where I presented “The Moneyball Approach to Big Data – Creating an Unfair Advantage” at the Wall Street Technology Association’s Hot Technologies Forum. Big Data is an area technologists are curious about, but I’m concerned there’s a “wait and see” approach. My job is to create value for my customers, and I’d hate for any of you to miss out on this opportunity.

Skepticism or “late adopter” mentality is understandable – if you want to forego a low-risk, high-reward opportunity and let your competition gain the advantage. Everyone is benefitting from Big Data in some form or another – most probably don’t even know it. But believe me, there are 100s of scenarios I could walk you through that could save your company millions of dollars, grow revenue double digit percentages, create more personalized products that delight your customers, automate real-time feedback on your brand, products, and competitor prices, create your own custom research that gives allows you to see trends before your competitors, and overall make you a much more agile business that scales with your new found vigor and growth.

What’s the secret to Big Data rewards? “Relevance”, “Access”, “Intelligence” and “Action”.

The most common definition I’ve seen for Big Data relates to the 3 Vs:

• Volume: it’s Big – terabytes and petabytes
• Variety: it comes in many forms – internal, external, structured and unstructured
• Velocity: it grows fast and changes quickly – making real-time capture and action hugely important

And this is always supported by numbers showing how gynormous Big Data is:

• The New York Stock Exchange creates 1 terabyte of data per day (InformationWeek)
• 10,000 payment card transactions are made every second around the world (American Banker)
• 30 billion pieces of content shared on FB every month (McKinsey)
• Twitter feeds generate 8 terabytes of data per day (InformationWeek)

Before you go out and buy more storage, consider what you want to do with it. If there are 200M tweets a day equaling 8 terabytes of data, but only 1000 of the tweets relate to your product or company, do you need to store and analyze all 8 terabytes every day? Although Big Data is big, don’t get caught up in all the massive numbers. Stick with what’s relevant to your business.

Forrester Analyst Brian Hopkins made a great point in his blog “Big Data will help shape your markets next big winner”, stating that Forrester estimates enterprises use only 5% of their available data. So the playing field is wide open for anyone to quickly take advantage of the 95% they’re currently ignoring.

But slow down there pardner. Sybase published Big Data, Big Opportunity that stated, “for the median fortune 1000 company… a 10% increase in usability of data translates to an increase of $2.01B in total revenue per year, [and that] a 10% increase in accessibility to data translates to an additional $65.67M in net income per year.” So don’t think you have to go from 5% to 100%. You really only need to go from 5% to 5.5%.

The internet plays a huge role in the rapid growth of Big Data, giving individuals the ability to post and upload immense amounts of pictures, text, video, and mobile data, and businesses the channel to offer access to customers and partners through web-based applications (think Oracle, salesforce.com, social media, procurement, logistics, publishers, and so on).

In reviewing other articles about Big Data, despite all the discussion around the massiveness of Big Data, I didn’t find a single article mentioning the difficulty of accessing the data spread throughout all these applications. This is a HUGE POINT to understand because you are SOL if you can’t access the data you need. If I told you I could guarantee any app or data you can see in your web browser (customer data, bank transactions, twitter, blogs, supply chain vendors, government data, competitor prices, etc.) could be automatically accessed and loaded into the app, database, or spreadsheet of your choice, how many game-changing Big Data projects could you think of? Point-in-time cash position understanding of billions of dollars across 300 banks? No problem. Monitor competitor pricing on 50,000 SKUs every day? Simple. Automate a twenty-three step manual invoicing process to get paid millions of dollars 2 days faster? Done. Real-time, automated access to the data you need is the key to success with Big Data. Lest you think this all fantasy, learn how Kapow Katalyst Application Integration Platform provides real-time access to Big Data.

There’s huge difference between “I have terabytes of data – videos, satellite pictures, social media conversations, and research reports” and “I know where Osama Bin Laden is”. It’s Data vs. Intelligence. Data is useless if you can’t extract meaningful intelligence from it. And the quality of your intelligence is most likely much less dependent on the volume than the relevance and ability to access it.

And the whole point of having relevant, accessible, intelligent Big Data is that it is actionable. Otherwise it’s just a recommendation or a strategy without execution What’s incredibly cool about Big Data and the web-based nature of so much of it is that just as easily as you can access anything you can see, you can just as easily transform the data, perform an operation on it, and automate a resulting action for you. Huh? Here’s an example. You know consumers and even your B2B purchasers research prices online and that loyalty to any one vendor has deteriorated as buyers have more pricing knowledge a search and mouse-click away. But you are smarter than your competitors because you’re already doing the extra 10%. So you set up automated monitoring of your competitor’s pricing, and when their price drops below yours your Big Data Integration Platform calculates the difference plus 10%, logs into your ecommerce site and adjusts your prices automatically, all within a few ticks of the clock.

And the beauty is that this can all be set up in hours, if not a few days, and you don’t have to bring in an army of developers or consultants to create custom code to do any of this.

So let the Big Data party begin. Kapow Software is here to help. To learn more about Big Data Solutions or to set up a Big Data Sales Consultation, click either link, because you’ve read this far and deserve it!

By: Rick Kawamura

I was just rereading the article, “Choose your cloud with integration in mind” by David Linthicum for Infoworld’s Cloud Computing blog.

David knows this space very well, and in this article he discusses the challenge of integrating cloud-based and on-premise applications with the limits on integration interface instances imposed, understandably, by some cloud service providers.

I agree with David that it’s common sense to integrate your on-premise business applications with your SaaS-based business applications. Any company that has business applications inside the firewall as well as outside must integrate them to realize their full value.

I also agree that if you select a cloud-based integration solution, you need to be sure you have enough interfaces in the cloud to support the number and volume of connections from the cloud to your on-premise apps.

But is this the biggest challenge when doing on-premise to cloud integration?

No, or at least it’s not what I hear most often when discussing integration with industry people. They tell me that IT’s biggest concern is security. They ask, “How do you get your IT department to open up the firewall for integrating cloud apps to on-premise apps? Because I get a blank stare when I ask our IT to do it.”

So how do you deal with this?

You choose an integration platform that can be installed both on-premise and in the cloud. This is a must-have requirement.

With your integration solution installed on-premise, you can integrate applications inside-out rather that outside-in, which will be a lot easier for your IT department to accommodate because there’s less risk.

At the same time, you can have the flexibility and easy maintenance of a cloud integration solution for all those applications where firewall issues are not showstoppers.

To avoid any unpleasant surprises, I recommend looking for these features and benefits in addition to the security of a hybrid solution when choosing an application integration platform:

Does the integration platform you’re considering or using provide these features and benefits? How do you deal with IT’s concerns about security and hybrid integration? I’m interested in your experience and perspective.

By: Stefan Andreasen

In 2007, James Governor penned an article, Why Applications Are Like Fish and Data Is Like Wine, depicting how data gets better with age, while apps, like fish, begin to smell over time.

Earlier this year, Chuck Hollis from EMC offered his own wrinkle on the topic, also making a case for keeping the data (wine) and dumping those ‘lumbering apps’ (stale fish).

Both offer interesting reads, but too quickly dismiss the value of apps. Fish and wine go well together and depending on how you pair them, can make or break the experience or value of the meal. The same is true in the modern enterprise – the fish (application) is equally, if not more important than the wine (data), especially when the apps are kept “fresh and simple” with “many varieties to choose from…”

Consider if all the data trapped within applications were easily accessed and made even easier to interact with other applications.

There should be a straightforward way to interact and automate as many ‘fresh’ application sources as possible, in the shortest amount of time. IT organizations can no longer remain ‘comfortably numb’ in their avoidance of these agile Line Of Business (LOB) activities.

So, does the LOB really want a customary IT application integration fix or are they asking for something different? When the LOB says, “I really need access to this data now!” what they are really asking for is to interact with applications (or websites) in an impromptu manner in order to keep pace with emerging business dynamics and acquire key data advantages for the business.

The blistering pace and expansion of social media access, in all its forms (brand management, blog monitoring, anti-piracy, competitive intelligence, analyst research, prospect & customer mapping, consumer behavior analysis, marketing research, partner & reseller communication, on-line videos, risk compliance, legal monitoring, corporate reputation, background checks, R&D innovation research, consumer & customer research, sentiment analysis, predictive analytics, social CRM) makes the LOB integration demand on IT seem like an insurmountable task. The intensification of interactions from sources without API coverage (or even marginal API compliance) can no longer shackle the LOB data integration needs.

The traditional corporate IT blueprint has been to deploy long established application integration methodologies. This old design is already showing the strain of abandonment. I would propose a modern application integration approach consisting of the following components:

APPLICATION INTEGRATION PILLARS:

All four pillars are key to providing real-time integration, but they must also move toward a new form of LOB automation – a self-service component for the LOB. Ubiquitous access with the ability to quickly prototype and test application interactions are the most critical components to this offering. Applications need to be integrated in order for the LOB to interact and drive time-to-value (TTV) in the enterprise.

Adding to the turmoil for IT organizations are the hundreds (often thousands) of in-house applications constructed over the years. The vast majority of these have not been SOA enabled. Those ‘lumbering apps’ are also inclusive of the LOB need. There’s valuable data trapped in those in-house apps that must be freed.

Integrating to applications will require more frequent and varied connections with real-time and on-demand communications. These integrations may be more permanent links to applications or have ‘throw-away’ integration conditions.

The LOB is forcing the application integration challenge to the forefront of the IT stack. Applications are the real-time component of the raging BIG DATA frenzy. The day is coming, when the LOB will self-serve most of their application and website interactions.

The data wine cellars of the enterprise will be cared for by the IT stewards, but the Line Of Business wants FISH.

By: John Yapaola

Do you remember when JavaScript was merely used to add some snappy effects to plain HTML websites; perhaps a drop-down menu that rolled out when you moved your mouse over an object or an image that changed every 5 seconds?

We were a bunch of happy-go-lucky web programmers — script kiddies — back in the mid-90s who could not imagine how much business logic, data transformation, and interactive presentation would be taking place in the user’s browser 15 years later. Sure, data was sometimes dynamically generated, but all that magic took place on our server-side scripts.

When Kapow Software was simply extracting data from the web in the late 90s, we could safely ignore the sprinkles of JavaScript on a website; we could simply go straight to the HTML to get the data. Unfortunately, I still see people trying this approach.

Today even the best script kiddie is going to have an extremely hard time crafting a Perl script that can grab data from a modern website or web app where the information displayed in the user’s browser is the result of executing thousands of lines of JavaScript, pulling data from web services, and transforming that data in sophisticated ways. Even worse: Imagine doing web application integration or business process automation just scripting!

Here at Kapow Software, we quickly realized that disregarding JavaScript when trying to integrate web applications was insufficient. And we made the radical decision to base Katalyst Katalyst on our Integration Engine with Web App Interface that allows you to integrate any modern website or application, including all of the dynamic content generated by JavaScript.

That means that as a Kapow Katalyst user, you can focus on the data or apps you want, rather than having to concern yourself with how the website or web application was originally built.

While it was fun being a script kiddie in the nineties, with today’s dynamic web and the exponential growth in content and sources, scripting has proven extremely unproductive, if not impossible. It’s time to shift the focus away from technical issues and move toward solutions that help you solve your business problems.

By: Anne-Sofie Nielsen

Director of R&D at Kapow Software

The best thing about working in Business Development is meeting with partners and customers. It’s a great way to stay on top of technology trends, and my goal for this blog post is to keep you posted on developments I see on the road.

This year was Kapow Software’s first time exhibiting at Salesforce.com’s Dreamforce event where the buzz was all about the social enterprise and the value of collaboration and interaction in business and government. Kapow Software, together with our partner Threshold Consulting, made it to the final of the Salesforce Hackathon with a bi-directional integration between Salesforce Chatter and Google+ — a unique social integration feat because Google+ doesn’t support APIs.

We returned to Moscone Center in San Francisco for Oracle OpenWorld. Arik Hesseldahl, in his AllThingsD.com blog, offers an insightful analysis of the rivalry between Oracle’s Larry Ellison and Salesforce.com’s Marc Benioff, which we witnessed firsthand. Arik also explains the two visionaries’ divergent views of the cloud, which can be summed up as a hybrid environment vs. the pure cloud.

For our part, we knew our Founder and CTO Stefan Andreasen’s session on automating content migration into Oracle OpenWorld resonated when one attendee said, “it made the conference worth it in its own right.” Oracle and Kapow Software announced a Documentum trade-in campaign with a special offer for customers who are making the move from Documentum to Oracle WebCenter, using Kapow Software’s automated migration tools.

Next on my itinerary were two events for the intelligence community. I have never seen so many different national law enforcement agencies, as well as state and local police departments, as I did at ISS World in DC. They came for training on the technologies, techniques, and legal considerations of intelligence gathering and analysis. Back in California was Suits & Spooks, the so-called anti-conference designed to bring the greatest Silicon Valley entrepreneurs together with US intelligence agencies. (There wasn’t an actual suit to be seen anywhere.)

Having been involved with the technology side of intelligence for over 10 years, I’m astounded by how far we’ve come from simple reports and dashboards. The focus now is on social network analysis, geo-location-based visualization, and enhanced reality. But for all of the advances in analytics and visualization, the greatest challenge with intelligence continues to be getting access to the data, particularly as the majority of the data – big data – is outside the control of any one organization.

Last week presented the dilemma of choosing between two events: Pyxis Mobile’s Connect 2011 Summit and GEOINT 2011 Symposium hosted by the United States Geospatial Intelligence Foundation (USGIF). Kapow Software exhibited and presented at both, but I ended up choosing the Pyxis Mobile conference – and I’m glad I did. We met with a lot of great customers and partners, and my hat goes off to Chris Willis and Pyxis for organizing such a successful event. What was most enlightening for me is the impact that tablets (iPads and Androids) are having on enterprise strategies for mobilization. Most companies are developing strategies to mobile enable enterprise apps and were impressed with Kapow’s ability to integrate web application data without the need for APIs or any other programmable interface. Having resisted mobilization, IT seems to be forced to act finally by the ubiquitous “consumerization” of mobile devices. And tablets are starting to provide to field workers what has been promised for so long.

All in all, technological development in all of these areas is moving at neutrino speed. I’ll do my best to keep you informed. I’m back in the office this week, catching up on everything; hence, the timing of this post.

By: Rory Byrne

Earlier this week TechCrunch posted this article: Google+ API Launch Still Months Away. It was tweeted, liked, and shared over 1100 times, and the passion around why the Google+ APIs are not yet published is stirring quite a discussion as developers clamor to get their hands on the API.

For those of you anxiously waiting for the Google+ API, consider this: Kapow Software successfully integrated Google+ to Salesforce Chatter last week at Dreamforce, and entered it in to the hackathon where we made it to the final round. There was buzz in the audience as to how it was possible to integrate to Google+ when no API existed – enough that the judges had to ask the audience to stop asking questions.

How can Kapow Software integrate Google+ when no APIs exist?

Kapow Katalyst Application Integration Platform it’s actually very easy. Kapow Software designed a visual flow-chart editor that allows developers to integrate to web applications without requiring APIs. We’ve successfully done this with legacy apps (enterprise, ecommerce, banking, etc.), financial apps (hundreds of banks), partner apps (shipping, payroll, accounting, supplier, procurement, etc.), social media (Facebook, Twitter, Google+, TripAdvisor, LinkedIn), and hundreds more, including cloud and saas.

To integrate with G+ we used our Kapow Katalyst IDE, the Design Studio. It’s a visual flow-chart editor that automates the building of application integration workflows you can use to access and control any website (or API for that sake). As you create the flow, Design Studio is actually accessing the target website service in real time, so as you build a workflow you’re also testing it, in real-time. When the workflow is complete, you’re also done testing—it’s very productive.

We built automated workflows (“robots”) that login to G+, navigate to the Stream, and submit or extract posts from and to Chatter. A traditional API wasn’t needed because the Katalyst Application Integration Platform leverages the G+ presentation layer as the API. Our Kapow Automation Browser is a purpose-built browser engine, with full understanding of the web page’s structure and JavaScript state, so even sites such as Google+ that are built with cutting-edge AJAX techniques are easy for Katalyst to handle.

It’s easy to create a REST service in the cloud or on premise—with a click of the “Deploy” button you’ve built an API for use by anyone, anywhere, inside or outside your organization. These APIs can include website navigation and business rules as well as access to databases, spreadsheets, documents, file systems or other web services, and much more—all encapsulated in a single service call.

Learn more about Kapow’s Application Integration Solutions or request a free trial!

By: Stefan Andreasen