Kapow Katalyst Blog

Web data – external, unstructured, unwieldy, difficult to access and integrate, but oh-so-valuable – is a large part of what makes Big Data a goldmine. Today marks a major milestone for Kapow Software with the announcement of our new partnership with Informatica and the introduction of Informatica PowerExchange for Kapow Katalyst at InformaticaWorld, so that any data, anywhere on the web, is now available to Informatica customers to build their social media and Big Data strategies, as part of the Informatica 9.5 launch.

This further validates our unique competency to get critical and extremely strategic data from the web that no one else can – a capability that has been recognized and leveraged by over 500 customers worldwide – and now extends Informatica’s reach beyond the usual suspects; Facebook, LinkedIn, and Twitter.

When you think about it, wouldn’t it be great if you could access any data on any web app, even password protected apps, just as easily as accessing data in your own database? What if you could do this without writing a single line of SQL or taxing your already overburdened IT resources?

With Kapow Katalyst there is a rule of thumb: If you can see the data in your web browser then you are only a few clicks away from accessing and putting it to critical use in real-time. It’s that simple – and it’s scalable, secure and robust – used already by hundreds of companies around the world.

The internet, with 40-thousand new websites added every day, is the fastest growing source of information. It’s Big Data at it’s extreme and a discussion topic my colleagues and I have written about in the past.

Accessing the entire web as a database is very much about the 4th dimension of Big Data – the spread of data – and reflects one of the biggest challenges in Big Data – how to deal with accessing data from an explosion of different sources.

That problem is now solved.

With Informatica PowerExchange for Kapow Katalyst, Informatica customers can now access and extract all this data, noise-free and fully structured, to be used in a variety of Informatica products and Hadoop clusters.

I am very excited about this, and cannot wait to hear more about all the great projects Informatica customers will use this for – projects that I guarantee will drive significant value, driven by innovative ideas for new ways to delight customers, outwit the competition or simply operate more efficiently.

You would never gain this newfound value from not having direct access to the data, and certainly would not get it by buying the data from a data provider.

If you happen to be down in Vegas this week, come visit us at InformaticaWorld booth #S5 – we would love to show you how Kapow Katalyst can drive growth for your organization.

By: Stefan Andreasen

In a recent whitepaper for CIOs, Forrester analysts Ted Schadler and John C. McCarthy define mobile as the manifestation of a broader shift to new “systems of engagement”. These systems according to Forrester “help firms empower their customers, partners, and employees with context-aware apps and smart products to help them decide and act immediately in their moments of need”. This type of connectivity shifts more power to individuals, who can now serve themselves to check account status, make a purchase, view information or conduct a transaction – all via mobile devices.

There are multiple examples across many industries where mobility has taken the engagement to different levels, delivering business value and securing unique competitive advantages. Audi, who was recently honored the 2012 Connected Car of the Year awards, is one of these examples. The award spotlights vehicles with technology that strikes the right balance between safety, convenience, and infotainment. Audi A8 was selected for its unmatched Audi connect™ infotainment system. The leading automobile manufacturer wanted to provide real-time answers to location-based travel queries from its A8’s in-dash mobile navigation system. Audi recognized the power of engagement and leveraged mobility to innovate with a unique smart product that received recognition from industry experts and most importantly high-end customers. Audi delivers real-time data feeds from multiple sources about weather, restaurants, gas prices, traffic, construction and road updates directly to the Multi Media Interface in-car computer system. Audi is using mobility to redefine the driver’s everyday experience through access and connectivity to real-time information that is accurate, relevant and timely.

It is always a pleasure to help your customers introduce compelling new products, become leaders in their markets and receive industry-wide recognition for it. Kudos to Audi for their innovative business approach!

By: Hila Segal, Director, Product Marketing

Traditionally, when we talk about how data relates to applications, we think of data as the bottom of the three layers of the application stack:

But that distinction is evaporating fast, and soon we will end up with only one layer: the application itself.

Modern application frameworks also define the data model and tie it tightly with the application logic and the user interface. This means that even if you could access the data directly in the SQL database, the data won’t make much sense without the application logic that ties it together.

Similary application frameworks are also used to build the user interface, which means that application logic in modern applications is also woven into the the user interface; for example, into the JavaScript of a web application, leveraging one of many AJAJX frameworks to create a vivid user experience.

Now let’s look at BIG DATA applications. These don’t even use standard SQL databases anymore, but specialized NoSQL databases like HBase and Hive, build on the Hadoop processing framework. Now capabilities like search and analytics are tied together in one platform and the data can only be accessed through the BIG DATA application itself. If you need to fresh up on NoSQL technologies, check out this good (albeit a bit old) article, Decline of the Enterprise Data Warehouse by Road to Failure blogger Bradford.

The consequence of all this is that data and applications are becoming one and the same thing.

Similarly, data integration and application integration products are becoming one and the same thing, and anyone who needs to access data, will think of data in an application, not data standing alone.

To deal with this, we need to develop innovative new integration technologies that can access data through the application — a hybrid data integration and application integration platform, so to speak, but much more agile, since the pace of business and the impatience of business users today don’t permit waiting months or years to leverage that data.

The Kapow Innovation is the scalable Kapow Katalyst Enterprise Application Integration Platform that can live both on-premise or in the cloud; a technology that can access the data for the modern world, a world where there are no longer defined application layers.

Do you agree or disagree?

By: Stefan Andreasen

It’s Friday morning in sunny California. I’m heading to the office, listening to some good music. Usually I avoid the busy US-101 going south to Palo Alto and take the nicer I-280 route. This Friday I guess I was feeling lucky or simply tired of sifting through traffic reports and data feeds. I simply wanted the answer: 101 or 280? I took 101 based on a gut feeling. I was wrong. What usually takes 30 minutes became 1 hour due to a gravel spill on the road. There went 30 precious minutes all for making a “non-data-driven” decision.

As I was stuck in traffic regretting my decision to take 101, it made me think about data and how it is becoming the new currency. There is so much more data at our disposal as individuals for the decisions we make every day from traffic to shopping to booking a hotel or selecting a physician – which makes our lives better by saving time, money or simply feeling better informed. But at times this can be a little overwhelming when all we want are the answers.

When you think about what’s at stake for organizations it’s no wonder Big Data is becoming such a headliner and companies such as the world’s largest retailer, Wal-Mart, are turning to long-tail data on the web and social networks to glean insights into what consumers want in order to better compete with its online rival Amazon. But how much data do you really need to be able to make smarter decisions? How do you bridge the gap between increasing volumes of information and getting the right answer to the right question at the right time that will make you smarter and more successful?

In his article published in Forbes, Christopher Frank, Vice President in Global Marketplace Insights at American Express, talked about the “IWIK” (“I wish I knew”) discussion to identify wants and needs with respect to Big Data while uncovering the strategic, essential questions that can be answered through this information. I couldn’t agree more with Frank’s bottom line: “In the new reality of big data, hyper-competitive businesses are linking and labeling perception information, transactional data and layering in social behaviors and then using the IWIK to define the data rails to shape analytic-based decisions. Therefore, the focus should be less on ‘big’ data and more on ‘essential’ data”.

I wish I knew about the gravel spill on Friday…. Let’s start focusing on the data that matter the most for our strategic decision making and less about the volumes.

By: Hila Segal, Director, Product Marketing

My good friend Mik just gave his wife an iRobot for Christmas, and she loves it. The iRobot is intelligent enough not to vacuum the same place twice, avoid furniture, and recharge itself in the wall when running out of power. No more vacuum cleaning and instead Mik’s wife can spend her time painting, which is her great talent and passion.

Those of us with the latest iPhone 4S, love Siri. The kind female voice that helps you automatically perform tasks like sending a text reminds me to turn the oven on when I get home, for example, and even learns to do personal tasks for me like calling my wife.

Then there is the new service on the web, IFTTT (If This Then That). What a cool service! We don’t get a lot of rain in California, but when we do I’d like to know the day before — automatically. This is what IFTTT does very well. All I had to do was go to www.ifttt.com, select “rain in Palo Alto” for “If this”, and then “send me an email” for “then that”, and now I always know a day ahead when it’s going to rain.

iRobot, Siri, and IFTTT are just the beginning of a growing number of “intelligent robots” which will help us with our lives both in the physical and virtual world of the Internet.

Just as the Industrial Revolution automated the tedious tasks of weaving clothes, hammering hot iron into a horseshoe, and even building a new car, the information overflow and busy life of today has created demand for a next-generation Industrial Revolution, one built on services driven by intelligent software and smart devices with powerful CPUs.

I just read an article on AppleInsider about future updates to Siri. The article gives a hint to where Apple is going, and I predict that soon a next-generation Siri will become our 24/7 personal assistant, monitoring anything we want on the Internet and automatically performing actions such as buying a ticket for the next football game when one becomes available, or notifying us where to get the cheapest gas nearby when driving down the highway.

Soon all of the manual processes derived from the increasing information overflow — A. I. and the Next American Industrial Revolution on ptucker’s blog at the World Future Society website.

You might find this article interesting as well: Why it’s China’s turn to worry about manufacturing on the Berkley Blog.

Let’s go paint a beautiful picture.

By: Stefan Andreasen

There has been so much hype around Big Data and now finally there is a $ amount associated with it – Jeff Kelly form Wikibon, an open-source-style community of industry analysts, delivered the much anticipated forecast estimating Big Data market will reach the $50 billion mark worldwide by 2017, growing from $5 billion in 2012. IDC with a more modest forecast still predicts 40% a year growth rate reaching close to $17 billion by 2015.

Wikibon expects a significant growth for Big Data as “the new definitive source for competitive advantage across all industries”. There are plenty of examples of where data-driven decision making has made a significant impact in retail, financial services, logistics, consumer packaged goods and even sports with the well-known Moneyball Big Data story. Shipping companies use data on delivery times and traffic patterns for route optimization while retailers leverage data analysis for predictive targeting to increase number of shoppers, store visits and spend per visit.

Our customers are turning blogs, forums and social media commentary as actionable predictors for stock performance allowing them to make significant economic gains by being the first to buy certain stocks before prices go up. Others in the logistics space are leveraging real-time data from multiple sources to better compete for business in a timely and more economical way. The possibilities for agility, innovation and overall business growth are infinite and will drive more investments, as indicated by the analysts’ estimates.

While a lot of the focus in Big Data is driven by the new economical ways to store and process large volumes of data, getting the data is the first step to any Big Data initiative and can be equally challenging. This data is often scattered across unlimited number of internal and external data sources. These include a long tail of social feeds, review sites and news sources, cloud applications, government web-based applications (federal regulations, public data on housing, marriages, foreclosures etc.), channels, suppliers and competitor’s sites. A majority of these sources are difficult to connect to and the data they contain is constantly changing.

Removing this barrier with access to relevant data regardless of source or format will open up even greater opportunities for growth in this new wave of technology revolution called “Big Data”. I can’t wait to see what other game-changing use cases and applications are on the horizon.

By: Hila Segal, Director, Product Marketing

The debate continues – just what is Big Data? The definition of Big Data is all over the map, but the most important factor remains whether you can extract actionable intelligence from your big data. Splunk has a catchy phrase: “Listen to your data – It’s trying to tell you something”.

There’s a fascinating article in Forbes (How Target Figured out a Teen Girl was Pregnant before her father did) about how Target grew from $48B in 2002 to $67B in 2010 based, among other things, on a data-driven approach to analyzing historical purchase data to, in this particular example, predict the likelihood a shopper is pregnant to influence shopping habits that increase average purchase total per visit coupled with more frequent store visits. The title of the article comes from Target having sent baby related coupons to a teenager when they determined her shopping history gave her a high “pregnancy predication” score, which upset her father who complained to Target, only to later find out that his high school daughter was in fact pregnant.

From a marketing and ethical perspective, this is somewhat disturbing (albeit not surprising), but from an analytics point of view, fascinating. Reminds me of the social media monitoring fun I had predicting who would be eliminated from Dancing with the Stars and American Idol based on the insight I gained from twitter and facebook.

But what I found interesting about the Forbes piece was that nowhere in this article was hadoop or Big Data mentioned, yet I’m certain the volume of purchase data Target archives across their 1750 stores qualifies as Big Data.

The article mentions that the best time to change a person’s shopping habits is during a major life event such as having a baby, purchasing a house, getting married, and even divorce. Interestingly enough, all of this information is publicly available on various websites. Target could easily integrate those data feeds with their knowledge of purchase habits to further expand their big data analysis to incorporate real-time external data sources.

The value of Big Data is only just beginning. Stop just hoarding data and start taking action on it.

By: Rick Kawamura

Looking at the 2011 trends from ProgrammableWeb, the number of available APIs registered in their API Directory has once again doubled. What I find most interesting is that the REST web services now outnumber the SOAP web services by a factor of four.

Even Salesforce decided their original SOAP API could no longer stand alone and added a REST API in late 2010, acknowledging that REST is now the preferred choice for application integration due to easy implementation and deployment. Its lightweight nature is a good fit when creating mobile or web-based applications, both since the client app doesn’t need extensive libraries to be able to interact with the server and because SOAP has a much higher overhead in terms of bandwidth, which can negatively affect the perceived performance of the app.

Internally in many enterprises, SOAP is still used as the protocol over which the services in a service-oriented architecture (SOA) interact. However, despite the similarity of the acronyms, the components in a service-oriented architecture can just as easily interact using REST or other protocols. In fact, as many companies begin exploring hybrid integrations, leveraging the strengths of both in-house and cloud applications, it becomes crucial for any successful integration product to be able to consume both SOAP and REST web services.

In a hybrid world, the type safety of SOAP web services is of less importance, as the data transferred between cloud and in-house applications will most likely need to be validated and reformatted at the integration point anyway. Also, REST web services lend themselves easily to caching — riding on standard mechanisms — which can be important when the traffic between in-house systems and private and public clouds continues to increase.

My prediction is that in 2012, any CIO who is considering API-enabling an application — either to consume the service internally or to expose external APIs for partners and mobile or web applications — will be looking first and foremost at REST as the technology of choice.

By: Anne-Sofie Nielsen

Today is a huge milestone, not only for Kapow Software, but also for all those companies looking to extend their business on the Salesforce platform.

Last year as a sponsor at Dreamforce 2011, I was amazed to learn Dreamforce is now the largest conference in the enterprise software industry – even larger than Oracle Openworld.

But what was also apparent was the massive integration challenges Salesforce customers face. This was evident not only from the numerous customers I had the chance to speak with, but also from the large number of System Integrators and Cloud Integration sponsors (and the sizes of the booths) that solve SalesForce integration problems.

So here at Kapow Software we said, “Enough is enough. Let’s fix this once and for all. Salesforce customers should spend their money somewhere other than on Salesforce integration consultants.”

Our goal was to build this for Cloudforce this week, and I am proud to say that today we are releasing Kapow Katalyst Apex™ Generator for Salesforce®, the first and only truly self-service integration solution for the Salesforce platform, Force.com and Salesforce Visual Workflow.

It works in 4 easy steps, all with no coding, and all without any dependency on missing APIs or missing connectors. It’s completely self-service.

1. Build and test the integration to any web app in Katalyst DesignStudio. It won’t take much longer than to click through that web application once in a normal web browser. You can even combine multiple apps, Excel documents, XML feeds, SQL databases or web services all at once and from anywhere.
2. Click to upload it to the Katalyst Management Console for roles-based sharing with anyone in your company.
3. Click to launch the Kapow Katalyst APEX Generator, type in your Salesforce credentials/security token and click the bottom to generate, test and automatically upload the APEX connector.
4. Now you can use it directly from Salesforce Visual Workflow, or when building APEX apps on Force.com.

With more than 500 customers relying on Kapow Katalyst for busines critical integrations all over the world, I am now truly honored to offer a uniquely agile integration solution to all Salesforce users.

How about security? You decide where to do the integration – in the cloud, on-premise, or both. If your security policy won’t allow for inbound integration from the cloud, just install Kapow Katalyst on-premise.

You will be amazed how easy integrations can be and how fast you can respond to ever changing business needs, streamline manual business processes or build cool new innovative prototypes that can change your business forever.

By: Stefan Andreasen

Last week I attended a CIO Executive Leadership Summit here in Palo Alto. It was a great chance to interact with Silicon Valley IT execs driving transformation, innovation, and value creation in our rapidly changing connected world. Speakers included Tim Campos (CIO of Facebook), Kimberly Stevenson (CIO of Intel), Ralph Loura (CIO of Clorox), Mark Egan (CIO of VMWare), and Tom Keiser (CIO of Gap) just to name a few.

The event is structured to help IT leaders learn what it means to be transformational – to guide company-wide projects that result in significant changes to company strategy, value creation, revenue generation, and innovative products and services.

With the convergence of mobile, social, cloud and Big Data, CIOs cannot afford to ignore the importance IT has on the strategy, innovation, and ultimately business success of the companies they lead. Technology is core to a vast majority of initiatives that drive change.

Consumerization of IT is a passion of mine (and in particular the consumerization of integration), and there was no shortage of CIOs focused on the urgent need to make enterprise work environments equal to, if not better than, what many of their employees enjoy at home. Imagine that. Trying to be more convenient, efficient, and easy to use than what you can put together yourself from off the shelf products from your local electronics store.

But as IT departments look to provide a “consumer-like” enterprise environment for their employees, customers, and partners, why is there not a similar effort to make IT tools and platforms equally as easy to use? Whereas the iPhone “redefined” the smartphone market by resetting the consumer expectations of what a smartphone could be, when is someone going to “redefine” and “consumerize” the way IT departments integrate Big Data, Cloud, Mobile, and Social Media data and applications so enterprises can scale their businesses to not only keep up with the rapid pace of change their customers, partners, and employees demand of them, but to drive competitive advantage to innovate new products, develop new channels of revenue generation, and outwit their competition?

Transformational CIOs require transformational software. It exists, you just have to find it.

By: Rick Kawamura