This is the classic question that IT has asked line of business (LOB) in order to plan their data integration priorities. This has been the only way to ensure IT would have the data ready when demand arrived.

Not only has this caused tremendous waste of IT resources in building interfaces with expensive SOA technologies for data that might never be needed, but in today’s fast-paced world, the question almost becomes absurd.

Who really knows what data they will need in 6 months?

The fastest growing source of data is web data; data in cloud apps, social media sites, partner portals, government portals and competitor sites. How can you possibly know exactly what will happen in the market in 6 months?  You cannot.

The move to the cloud has been a great IT infrastructure cost saver, and has delivered innovative business apps like SalesForce, SuccessFactors, Eloqua just to name a few, but this “application spread” has added to  the growing challenges of accessing data that is spreading everywhere.

This week I participated in a very interesting discussion on exactly this topic, “All-Access Pass: How to Get the Data You Need, Fast”, on the Information Management DM Radio.

Click on the link and register to hear DM Radio hosts Eric Kavanagh and Justin Kern as they interview Wayne Eckerson of the BI Leadership Forum, Gagan Mehra of Terracotta and myself around new integration technologies which can deliver on this question:

Can you plan for the data you will need tomorrow?

Yes that is possible!

Authored by: Stefan Andreasen, CTO and Co-Founder

Whether it’s petabytes of data or just a few gigabytes, Big Data is really all about new and creative ways of leveraging data to extract significant value. Many companies are already using Big Data without calling it Big Data. It doesn’t have to be large volumes; it’s really about surgically getting the right data to make a difference in the enterprise.

Here are 5 use case examples for leveraging data in the enterprise:

1)  Competitive pricing: The on-line commerce market is highly competitive. Prices of products are dynamically changed several times throughout the day. To stay competitive retailers monitor and extract in real-time critical pricing intelligence and inventory information of millions of SKUs from multiple competing websites. With this data they’re able to offer consumers the best price, drive higher customer acquisition and grow market share.

2)  Regulatory compliance: Pharmaceutical companies need to build and maintain a healthcare professionals’ exclusion list of individuals with whom the company should not do business because they had lost a license or were otherwise disqualified. In order to meet U.S. Food and Drug Administration (FDA) compliance requirements they extract such information from various federal government websites, feeding into compliance systems.

3)  Sales prospecting: To speed up the sales cycle, companies are equipping sales executives with background and contact information for prospects aggregated from multiple public sources such as LinkedIn, ZoomInfo, business publications, Google, white pages and others. With a comprehensive profile of their prospects, sales reps are able to speed up the velocity and quality of their sales calls and increase win rate.

4)  Market research: In the fast moving and super competitive financial services market what you know today that no one else knows can yield significant financial gain. Companies in this industry are turning to various websites and government portals — looking for unique competitive advantage by leveraging market data that only they have. They’re creatively using this data to predict market conditions that guide critical investment decisions.

5)  Customer feedback: Today, users’ feedback and commentary about products is readily available on the web. Software companies are turning to a growing number of external and internal blogs, user forums and discussion groups to collect information about use case scenarios, product reviews, support cases and feature wish lists across industries.

Companies use this information to guide product development plans and competitive strategies.

Are you also strategically using data (big or small) in your organization? Comment here and share a common use case.

Stay-tuned for the next post discussing 5 more use cases examples….

By: Hila Segal, Director, Product Marketing

We’re so excited to announce the KapowNow webinar series. This is not just another set of webinars. We’ve specifically designed them so everyone can fit them into their busy schedules. They are short! In less than 30 minutes, you can learn about application integration, process automation, rapid creation of business applications, Big Data, social media monitoring, mobile enablement and much more.

Watch live demos and get all the tips and tricks you need to make integration more stress free, your business users more empowered, your company more agile and innovative. This series will show you how to use Kapow Katalyst and Kapow Kapplets as the single application integration platform, solving disconnects across the organizations such as getting analysts their data, providing employees mobile access to legacy applications and eliminating manual process – just to name a few examples.

Register today and spread the word by tweeting and using the hashtag #kapownow.

By: Hila Segal, Director, Product Marketing

Informatica Roadshow Seattle, October

The theme of the roadshow is centered around solving the return on data equation which Informatica defines as the value of data divided by the cost of data:

Return on Data = Value on Data / Cost of Data

Specifically I am talking about how this applies to what is quickly becoming the most important and fastest growing data domain – “Big Interaction Data”. It is your customers’ blogs and tweets, your competitors pricing information and the data on your partners’ portals. It is unstructured, rapidly changing and provides unique insights that can turn your company into the market leader.

From being almost non-existing a few years back, Big Interaction Data is today growing explosively, with most of that happening on the internet and available as “Web Data”. My previous blog Dynamic pricing with automated competitive price monitoring provides a good example of how the use of Interaction Data can create huge differentiation, drive topline growth and turn companies into winners in their industry.

For many, understanding the huge importance of Big Interaction Data is just in its infancy, but I can assure you this is quickly changing. In my presentation, I give a few other examples of how companies leverage Interaction data. One example is about, Fiserv, a large financial services company who had to monitor point-in-time cash position across 300 partner banks for regulatory compliance. They had no other way of getting the data other than manually logging in to 300 portals and retrieving bank balances. Being able to get this type of data in an automated way was game changing for this organization. So as you can see Big Interaction Data is far more than just social media and provides significant value.

The other side of the equation is the “Cost of Data”. If the data is impossible to get, or very expensive to get, or cheap but out-of-date, then the return will be lower.

The most successful companies will be the ones that put data first, the most relevant data, and leverage technologies to get all that data at a low cost – in real-time.

Here is a snippet video of the presentation I gave in New York earlier this month:

On November 7th I will be presenting again in Atlanta, but you can also meet my colleagues at the San Francisco event on the same day. I will also be speaking in Dallas on November 13th if you happen to be in that area.

Come see me and let’s discuss the topic.

Stefan

Sales reps spend a lot of time searching for leads and sourcing critical information on their prospects. This information includes contact details, new hires, recently launched products, new locations, etc. According to a recent survey, “CSO Insights 2012 Impact of Data Access on Sales Performance Report: Why Big Data Should Be a Big Deal for Sales”, sales reps search and review on average as many as 15 different sources before calling a prospect. Among them business-oriented membership groups such as LinkedIn are used more than traditional internal information sources. Other sources include search engines, press releases, social networks and financial statements. More than 80% of participants feel challenged by the amount of data available and the difficulty of accessing it. Think about how much time is being wasted by sales reps searching for data instead of making calls.

At the same time, the “decision window”, defined by Aberdeen’s BI practice as the period of time during which information can materially impact a business decision, is shrinking. That decision window can range all the way from real-time to weeks or months. When asked how that window is changing, two-thirds of Aberdeen’s surveyed decision makers reported it is shortened indicating the increased urgency for relevant data.

According to CSO Insight survey, nearly 90% of sales organizations feel they are missing opportunities because of inability to effectively leverage all information available on prospects. It all comes down to getting the right data into the hands of the frontline managers who make daily critical decisions that impact the top-line – such as dynamically changing product prices ahead of the competition, monitoring social media to prevent a major product recall or predicting market trends for profitable investment decisions. According to McKinsey director David Court, in his interview “ Putting big data and advanced analytics to work”, powerful but simple tools for managers are a key success factor for transforming companies to exploit data that can provide a broader view on the business, operations and customers.

Are you missing opportunities or are you putting the right business intelligence into the hands of the people responsible for growing your topline?

By: Hila Segal, Director, Product Marketing

Will toilet paper soon be priced like airline tickets?

The fast-moving Internet pricing games used by airlines and hotels are now moving deeper into the most mundane nooks of the consumer economy.

I just read this an article in the Wall Street Journal about how critically important it is for anyone selling anything online to monitor and react to the prices of their competitors in real-time.

The article has a nice figure that shows how Amazon, BestBuy and Sears are changing their prices on a specific GE microwave oven during a 24 hour period.

Source Wall Street Journal September 5th, 2012 – Don’t Like This Price ? Wait a minute

We all know who the most successful online retailer is of the three, and this figure explains a lot of the reason: Frequent real-time price adjustments. Amazon adjusted prices on the microwave oven nine times during the day, BestBuy twice, and Sears not once.

So how do they do it? Not by having humans visit competitor stores, but by software which basically turns the internet into a database of information so competitor prices can be monitored in real-time directly from any E-Commerce site on the web.

If you get your hands on the article (here is a link if you have a WSJ subscription), you will read that it mentions multiple companies who use this kind of software for real-time price monitoring and intelligence. One such company is Wayfair who sells a “Zillion Things for your Home”.

In the article, the CEO of Wayfair, Niraj Shah, states: “The new software has greatly speeded up the process. Online home goods retailer Wayfair changes hundreds of thousands of prices daily”.

Wayfair is one of several companies mentioned in the article using Kapow Katalyst from Kapow Software for competitive price monitoring and intelligence. Wayfair also recently posted a review of Kapow Katalyst (click the review tab) on SalesForce AppExchange which speaks for itself.

If you want more information on using Kapow Katalyst for real-time price monitoring, then check our website under Competitive Price Monitoring.

Now I wonder, is Toilet Paper next?

By: Stefan Andreasen

It was a great pleasure to co-host a Big Data webinar with Dominic Sartorio from Informatica. The topic of Big Data continues to make headlines, but companies are only in the early stages of starting to understand what Big Data means to them, and IT organizations are beginning to build out the required infrastructure to analyze this data. What is becoming clear is that the first step to any meaningful insight from Big Data is the ability to access and integrate information from a variety of internal, external, structured and unstructured sources.

Kapow and Informatica recently announced a partnership to address the need to capture hidden value found throughout the web in social platforms, portals and a variety of web applications. Through the joint solution, companies can maximize the return on Big Data with the ability to integrate information from blogs, forums, customer review sites, collaboration tools, partner and competitor portals, government sites and social networks to glean insights about customers, products, markets and competitors – information that is very valuable but typically trapped in data sources that do not have APIs.

More recently we are seeing a very rapid growth of what Informatica calls Big Interaction data which represents the various interactions of customers, partners, competitors and other external stakeholders with the organization and with each other. Dominic gave the example of a large retailer using a variety of systems like CRM and others to capture consumer identity and track various transactions like purchase history, returns, etc. As useful as this information is, there is tremendous value to be derived from consumers’ interaction outside of the organization which can help drive more effective business decisions related to the way you engage with customers and provide meaningful value without relying on anecdotal gut feel or intuition about what’s going on with your customers.

We’ve discussed a few other interesting examples of harvesting valuable Big Interaction Data including consumer goods company making product development adjustments based on social conversations on the web, an ecommerce player shifting its pricing strategy based on pricing information of its key competitors, a logistics company significantly improving efficiencies and operating margins by integrating and automating data from its partner portals, and a large financial services company integrating data from 300 banking partners’ web applications.

The opportunity is huge and Informatica and Kapow Software are happy to be working together to bring a tightly integrated solution that enables organizations to pull in social web data to be combined with internal data for better decision making. Don’t miss the follow up webinar: Putting Big Web Data to Work on September 25 which will offer a detailed demonstration of integrating web data harvested with Kapow Software into the Informatica Master Data Management (MDM) product to augment customer data records with information from blogs, forums and community sites.

By: Hila Segal, Director, Product Marketing

The age of connectivity is here. Smartphone and tablets have given consumers the power to connect with businesses from nearly anywhere. Companies that take advantage of the heightened connectivity are at a significant advantage, according to a Forbes article written by Marita Scarfi, CEO of digital ad agency Organic. The main challenge is that social media Big Data is unstructured, and, traditionally, it hasn’t been easy or cost effective to use that social media information to measure consumer engagement. However, more and more consumers are beginning to interact with brands through social media websites, making them a valuable source of business intelligence.

“To meet those ROI needs, we need to turn social media into actionable data or we may end up harming the brand instead of helping it,” Scarfi wrote. “The Holy Grail of marketing has always been creating great emotional engagement with utility. Social media has the power to connect consumers with brands in ways that go far beyond even the best word of mouth. But we still need to know if that emotion is turning into profit.”

Social web data such as blogs, forums, community sites and customer review sites provide extremely valuable insight about customers like and dislikes, brand loyalty and other related sentiment information. At the same time, these portals and web-based data sources are typically difficult to access and integrate due to the lack of available APIs.

With only 0.0017% of all websites having APIs – integrating valuable external web-based big data for your business becomes a key challenge.

By: Kapow Software Marketing

We recently came across a ZDNet blog post that touches on one of the issues we feel strongly about. Integration. The post highlights comments from Cisco CIO Rebecca Jacoby, who spoke at GigaOm Structure 2012. Jacoby had a few interesting things to say about how Cisco leverages its data.

According to Jacoby, Cisco begins with trying to understand data in context. The company first identifies whether the data is structured or unstructured, whether it’s in video or some other format and other factors to give the data contextual information.

Once context is added, Cisco determines what business value the data could add in productivity, growth opportunities or user experience. She also identified elements she considers core to data analysis regardless of the strategy used to conduct it. These include: where you’re going to gather data from, how to store it and how you’re going to package it.
Jacoby identified a challenge with traditional business analytics solutions, which is dealing with data from where it is sourced. This limits the speed at which the data can be used.

We agree with Jacoby there. It is important for businesses to be able to utilize valuable data to make better decisions. In order to do that in a timely manner, information needs to be accessible regardless of its source. According to Netcraft, as of 12/2011, there are over 366M websites. And according to Programmable Web, there are 6348 sites with published APIs. That’s 0.0017% that have APIs.

How can businesses integrate data from any web-based source, so they’re leveraging the full potential of the web?

By: Kapow Software Marketing

Since the Beijing Olympics, the number of Facebook users has surged to 900 million from just 100 million. There are over 500-million active users on Twitter, compared to just 6 million in 2008. The explosion in the use of social media and smartphones in the past four years means the London 2012 Olympics will be the most liked, tweeted, pinned, blogged and reported event in history. Like no other medium, the social channels will provide fans with access to athletes, results, special events and details about the Olympic Games.

This is the new world of informed consumers, empowered with information and freed from the reliance on traditional media and news providers. Now not only can I instantly know who won the gold in 200-m freestyle right when it happens but I also have access to information about this athlete’s previous wins, exact times for the last 3 Olympic Games and maybe even his favorite ice-cream flavor. Wouldn’t it be great if I can get all that with a single click? Pulling in athlete’s tweets, maybe weather data for the day of the race, info from Wikipedia and other sources about competing athletes — without entering these sites one by one. That would be golden….

But it can even get better. Automatically getting information on my favorite athletes, updates on their results, press mentions and links to photos and videos of key performances. Information collected for me and delivered via my preferred choice of delivery like text or email.

As the world becomes always-on always connected, we will see consumers demanding self-service access to information via tools that require less time, less training and less clicks. Let the games begin!

By: Hila Segal, Director, Product Marketing