Data Science Vs BI & Predictive Analytics

Business intelligence (BI) has been evolving for decades as data has become cheaper, easier to access, and easier to share. BI analysts take historical data, perform queries, and summarize findings in static reports that often include charts. The outputs of business intelligence are “known knowns” that are manifested in stand-alone reports examined by a single business analyst or shared among a few managers. For example, who are the probable high-net-worth clients to sell them a premium bank account. There can be some consideration like the average account balance etc.

Predictive analytics has been unfolding on a parallel track to business intelligence. With predictive analytics, numerous tools allow analysts to gain insight into “known unknowns”. These tools track trends and make predictions, but are often limited to specialized programs. In the previous example, the probable high-net-worth client could also be the spouse of an existing high-net-worth client that can be figured out using predictive analytics.

Data Science on the other hand is an interdisciplinary field that combines machine learning, statistics, advanced analysis, high-performance computing and visualizations. It is a new form of art that draws out hidden insights and puts data to work in the cognitive era. The tools of data science originated in the scientific community, where researchers used them to test and verify hypotheses that include “unknown unknowns”. Here are some of the examples:

  • Uncover totally unanticipated relationships and changes in markets or other patterns. For example the price of a house based on nearness to high voltage power lines or based on brick exterior.
  • Handle streams of data—in fact, some embedded intelligent services make decisions and carry out those decisions automatically in microseconds. For example analyzing the users click pattern to dynamically propose a product or promotion to attract the customer.

As discussed, Data Science different from from traditional business intelligence and predictive analytics in the following way.

  • It brings in data that is orders of magnitude larger than what previous generations of data warehouses could store, and it even works on streaming data sources.
  • The analytical tools used in data science are also increasingly powerful, using artificial intelligence techniques to identify hidden patterns in data and pull new insights out of it.
  • The visualization tools used in data science leverage modern web technologies to deliver interactive browser-based applications. Not only are these applications visually stunning, they also provide rich context and relevance to their consumers.

Data science enriches the value of data, going beyond what the data says to what it means for your organization—in other words, it turns raw data into intelligence that empowers everyone in your organization to discover new innovations, increase sales, and become more cost-efficient. Data science is not just about the algorithm, but about deriving value.

 

Disclaimer: The postings on this site are my own and don’t necessarily represent IBM’s positions, strategies or opinions.

 

Advertisements

Watson Analytics

Need for Watson Analytics
If an organization is good at analyzing data and extracting relevant insights from it then decision makers can make more informed and thus more optimal decisions. But the decision makers are forced to make decisions with incomplete information. The reason?  Decisions makers/ Citizen Analysts, for the most part, tend to be mainly consumers of analytics and they rely on more skilled resources (Like Data Engineer, Data Scientist, Application developer) in the organization to provide the data driven answers to their questions. Moreover the answer to one question is just the start of another. Think of a detective interrogating a suspect. The consumer/builder model is hardly conducive to the iterative nature of data analysis. Therefore, the time it takes for these answers to be delivered to the decision makers is far from optimal – and many questions go unanswered every day.

watsonlogoWatson Analytics
So a logical solution is to provide an easier to use analytics offerings. Watson Analytics provides that value add so that more people will be able to leverage data to drive better decision making using analytics.

When we think of Watson, we think about Cognitive. And when we think about Analytics, we think  about traditional analytics (querying, dashboarding), along with some more advanced analytic capabilities (data mining, and social media analytics). So Watson Analytics is a Cloud based offering which can make analytics a child’s play even for a non-skilled user.

Watson Analytics helps users understand their data in a guided way using a natural language interface to ask a series of business questions. Example, a user can ask “What is the trend of revenue over years?” and get a visualization in response. So, Instead of having to first choose a visualization and working backwards to try answer the business question, Watson Analytics allows you to describe your intent in natural language, and it chooses the best visualization for you. Even better, Watson Analytics gives you some initial set of questions which you can keep refining.

Watson Analytics for Social Media
Watson Analytics can work on Social Media data to take the pulse of an audience by spotting trends and identifying new insights and relationships across multiple social channels allowing greater visibility into a given topic or market. It combines structured and unstructured self-service analysis to enrich your social media analytics experience for exceptionally insightful discoveries. All on the cloud!

Summary of Steps:
Watson Analytics does the following to provide insights hidden in your Big data. Mouse-over the below images to get the details of the steps.

  • Import data from a robust set of data source (on Cloud and on premise) options, with the option to prepare and cleanse via IBM Bluemix Data Connect.
  • Answering What: Identifying issues, early problem detection, finding anomalies or exceptions, challenging conventional wisdom or the status quo.
  • Understanding or explaining outcomes, Why something happened.
  • Dashboarding to share results

Match and Manage your Data on Cloud

We left the last blog with two questions.

A few weeks back I wrote on IBM Bluemix Data Connect. If you missed it, then watch this video on how you can put data to work with IBM Bluemix Data Connect.

Now, Business Analysts will be able to leverage Entity Matching technology using Data Connect. The Match and Manage (BETA) operation on Data Connect identifies possible matches and relationships (in plethora of data sets, including master data and non-master data sets) to create a unified view of your data. It also provides a visualization of the relationships between entities in the unified data set.

For example, you have two sets of data : One containing customer profile information and the other containing a list of prospects. A Business Analyst can now use intuitive UI to do the Match and Manage operation to match these two data sets and provide insights to questions such as:

  •  Are there duplicates in the prospect list?
  • How many of the prospects are already existing customers?
  • Are there non-obvious relationships among prospects and customers that can be explored?
  • Are there other sources of information within that could provide better insights if brought together?

The two data set are matched using Cognitive capabilities which allows the MDM– matching technology to be auto-configured and tuned to intelligently match across different data sets:

dataconnect

Business Analyst can understand the de-duplicated datasets by navigating through a relationship graph of the data to understand how the entities are related across the entire dataset. Now they can discover new non-obvious relationships within the data that were previously undiscoverable. The following generated canvas enables them to interactively explore relationships between entities.

dataconnect1

In the above example it was illustrated as how clients can now easily understand the data they hold within their MDM repositories and how now they can match their MDM data with other data sources not included within the MDM system. This simplifies the Analytical MDM experience where MDM technologies are accessible to everyone without the need to wait for Data Engineers to transform the data into a format that can be matched and rely on MDM Ninja’s to configure matching algorithms.

Summary:

IBM Bluemix Data Connect provides a seamless integrated self-service experience for data preparation. With addition of entity analytics capability, business users are empowered to gain insight from data that wasn’t previously available to them. Now organizations can extract further value from their MDM data by ensuring it is used across the organization to provide accurate analytics. Entity analytics within Data Connect is now available in beta. Go ahead and experience the next evolution of MDM.

Cognitive 7 – Challenging Cyber-Criminals

It’s estimated that cyber-crime will cost the global economy more than $2 trillion by 2019—and could be the greatest threat to every company in the world! Since I have your attention now, lets spend some more time detailing this statement that I made.

cybercrimeOrganized criminals, hacktivists, governments and adversaries are compelled by financial gain, politics and notoriety to attack organizations’ most valuable assets. Their operations are well funded and business-like. However, cybersecurity solutions of the past often fail to protect against increasingly sophisticated attack methods. The result is more severe security breaches happening more frequently (50% increase in past year alone) which are geared towards stealing higher-value records such as health-related personally identifiable information (PII) and other highly sensitive data.

3 Top Security concerns of companies:

  • Optimizing their security program to unite security silos, reduce complexity and lower costs as opposed to buying dozens of products from dozens of vendors to nail down all kind of security risks.
  • Stopping advanced threats to prevent, detect and respond to known and unknown threats throughout the entire attack lifecycle. Currently on average a company takes 201 days to identify a data breach!
  • Safeguarding critical assets across users, applications, sensitive data and infrastructure, in the cloud, on mobile or on-premise. Note more than 55 percent of all attacks are carried out by malicious insiders or inadvertent actors

Protecting against security threats:

Cybersecurity  can be compared to an immune system. When you get a cold or virus, the immune system understands the virus and sends data through your central nervous system (security intelligence) to create white blood cells and antibodies to gather info, prioritize and take action. This is what’s called the “immune response.” Faced with an attack—whether to a server, mobile device, application, etc.—an effective security solution should detect that breach and work to stop it, similar to how an immune system would function.

Continuing with our comparison to fighting infection, think about how global health organizations share data to stop a pandemic. For cybersecurity, a partner ecosystem serves that same purpose. When organizations collaborate and share their data on new or changing threats, the other organizations can more quickly assess their own vulnerability based on that data and put appropriate protections in place.

Why Cognitive security?

Because even with major advances in technology, the tools available to today’s security analysts aren’t enough to keep up with the number of security events they see every day. There’s a tremendous amount of security knowledge created for human consumption, yet most of it is untapped. 80% of all data on the internet, including blogs, articles, videos, reports, alerts and other information cannot be processed by traditional security tools, and it’s this unstructured data which often proves most valuable in detecting and stopping threats before they cause harm.

Cognitive computing has the ability to tap into and make sense of all of the security data that has previously been dark. A cognitive systems continues to observe events and behaviors—distinguishing the good from the bad—the ability to use integrated defenses to block new threats gets stronger and stronger. By helping to make security analysts more effective and accelerating the response to emerging threats, cognitive security will help to address the current security skills gap, bringing heightened levels of confidence and risk control.

Cognitive 6 – Meet The Watson Family

Watson is a technology that can learn, compare, analyze, reason, justify and process human language, so there must be countless ways to create benefit from it. Watson doesn’t replace people, but acts as a trusted advisor that can quickly read and understand massive amounts of information and provide suggestions or warnings to help people make the best decisions. In fact, Watson is already interacting with 1 billion people in 8 languages. Let us look at some specific Watson offerings that are helping clients.

Some specific Watson offerings helping clients

Watson Knowledge Studio
Enables subject matter experts and developers to teach Watson the linguistic nuances of industries and knowledge domains—without writing a single line of code. Click here to get a series of videos on Watson Knowledge Studio. It can help an auto manufacturer to proactively identify safety defects using traffic incident reports as shown below.
knowledgeexplorer
Watson Explorer
Accesses and analyzes structured and unstructured content—presenting data, analytics and cognitive insights in a single view, giving users the information they’re looking for while uncovering trends, patterns and relationships.

Watson Company Analyzer
Watson Company Analyzer applies cognitive technology to enhance understanding of business accounts – creating holistic company profiles. It does this by collecting, filtering and analyzing structured and unstructured information. By building these comprehensive company profiles enables companies to better understand customers and accelerate opportunities.

Watson Engagement Advisor
Interacts with customers, listens to questions and offers solutions, learning with every human interaction and growing its collection of knowledge—quickly adapting to the way people think.

 Watson Health Solutions
From advancing personal training to cancer treatment, from clinical trial matching to insulin pumps that think, Watson Health is transforming the way organizations deliver health and wellness. Imagine you’re a doctor treating cancer patients. It’s impossible for you to keep up with the vast amount of new data from medical journals, clinical trials, academic research and other sources that would help you provide the best treatment recommendations. That’s where Watson comes in. It can ingest all of this data and find potential connections or correlations that you wouldn’t find on your own—providing you the insight to make the best decisions about treatment options for the specific type of cancer your patient has.

To discover more about Watson through education modules and courseware, visit IBM Watson Academy.

Cognitive 5 – IBM Watson as a cloud service

After its success on Jeopardy!, IBM research worked to make this technology open for public use. Later, IBM established a separate business unit for Watson called The Watson Group and a dedicated workforce to continuously improve Watson’s capabilities. The aim is to bring the power of Watson and cognitive computing to market using cloud delivery models. With those efforts, some of the Watson capabilities are now available on IBM Bluemix. These are available “as a service,” meaning you can use them in your own applications and services, embed them anywhere using Watson APIs and enhance your application capabilities dramatically. This also means that soon you might be able to do all the magic behind the Jeopardy! challenge within your application, with just a couple of clicks!

We can open the Bluemix dashboard and start using these services. Here they are:

watsonapis

AlchemyAPI
Using AlchemyAPI, developers can perform tasks such as extracting the people, places, companies, and other entities mentioned in any publicly-accessible webpage, posted HTML/text document, or a predefined corpus of news articles.

Conversation:
Using Conversation APIs, you can add a natural language interface to your application to automate interactions with your end users. Common applications include virtual agents and chat bots that can integrate and communicate on any channel or device.

Document conversion
The IBM Watson Document conversion service converts a single HTML, PDF, or Microsoft Word™ document into a normalized HTML, plain text, or a set of JSON-formatted Answer units that can be used with other Watson services.

Language Translation:
It dynamically translate news, patents, or conversational documents and can instantly publish content in multiple languages. As a result French-speaking staff can be empowered to instantly send emails in English.

Watson Personality Insights:
Personality Insights derives insights from transactional and social media data to identify psychological traits which determine purchase decisions, intent and behavioral traits; This can be utilized to improve conversion rates.

Retrieve and Rank service
Watson Retrieve and Rank service helps users find the most relevant information for their query by using a combination of search and machine learning algorithms to detect “signals” in the data. The Retrieve and Rank Service can be applied to a number of information retrieval scenarios. For example, an experienced technician who is going onsite and requires help troubleshooting a problem can use this.

The Text to Speech and Speech to Text
Text to Speech service processes text and natural language to generate synthesized audio output complete with appropriate cadence and intonation (multi language support exists). The Speech to Text service converts the human voice into the written word.

Tone Analyzer
This API leverages cognitive linguistic analysis to identify a variety of tones at both the sentence and document level. It detects three types of tones, including emotion (anger, disgust, fear, joy and sadness), social propensities (openness, conscientiousness, extroversion, agreeableness, and emotional range), and language styles (analytical, confident and tentative) from text. This insight can then used to refine and improve communications.

Tradeoff Analytics
This API helps people make better choices while taking into account multiple, often conflicting, goals that matter when making that choice. The service can be used to help make complex decisions like what mortgage to take, and also for helping with more everyday ones like which laptop to purchase.

Using these Watson APIs now you can build cognitive apps that help enhance, scale, and accelerate human expertise. In our next blog, we will explore some of these cognitive apps. Stay tuned.

Cognitve 4 -Cognitive Computing in Action

There is growing interest and imagination about how Cognitive computing can help organizations to think in whole new ways. To see cognitive in action, here are some Industry specific examples of organizations benefiting from power of cognitive computing.

macyRetail

Cognitive systems are driving more personalized shopping experiences and helping reveal customer trends. Macy’s, a U.S. department store chain, is using Watson cognitive technology to create an in-store mobile companion that assists in servicing customer needs. It allows customers to input natural language questions regarding each participating store’s unique product assortment, services and facilities and receive a customized response to the inquiry. Click here for a video.

Banking

Cognitive systems are improving the client experience and driving more efficient operations. Bradesco, the second largest bank in Brazil, used Watson as a virtual agent assistant to answer banking questions, reducing the number of calls; improved its client experience and delivery using Watson Self Service Assistant for a new virtual channel; and kept its innovation leadership position by implementing cognitive computing and being the first Watson Portuguese implementation.

Health

Cognitive systems are transforming global health. Medtronic, a U.S. and Ireland-based medical device company, has insulin pumps which, combined with Watson, will help predict dangerous spikes or drops in blood sugar hours in advance, then notify people living with diabetes so they can take action before it becomes a problem.

Energy and Utilities

Cognitive systems are acting as a trusted advisor. Woodside, an Australian oil and gas producer, needed to access knowledge from decades worth of projects to save the company millions of dollars. With Watson Engagement Advisor, they created a service that culls through 30 years of documented expertise, recognizing patterns and continually learning from them. Watson acts as a trusted advisor answering questions from engineers, while learning through newly adapted knowledge. Here is a video on it:

In next blog we will explore some of the Watson offerings (aka APIs) available for consumption. So stay tuned.