Information Governance – Revisited

IIGIt has been more than 5 years that I wrote on Information governance. Over the period of last 5 years some areas of Information Governance became more matured and I thought of re-visiting this topic. In a simple analogy, what library do for books, Data governance does for data. It organizes data, makes it simple to access the data, gives means to check for validity/ accuracy of data and makes it understandable to all who need it.  If Information Governance in place, organizations can use data for generating insights and also they are equipped for  regulatory mandates (like GDPR).

There are six sets of capabilities that make up the Information Management & Governance component:

  1. Data Lifecycle Management is a discipline that applies not only to analytical data but also to operational, master and reference data within the enterprise.  It involves defining and implementing policies on the creation, storage, transmission, usage and eventual disposal of data, in order to ensure that it is handled in such a way as to comply with business requirements and regulatory mandates.

2. MDM: Master and Entity Data acts as the ‘single source of the truth’ for entities – customers, suppliers, employees, contracts etc.  Such data is typically stored outside the analytics environment in a Master Data Management (MDM) system, and the analytics environment then accesses the MDM system when performing tasks such as data integration.

3. Reference Data is similar in concept to Master and Entity Data, but pertains to common data elements such as location codes, currency exchange rates etc., which are used by multiple groups or lines of business within the enterprise.  Like Master and Entity Data, Reference data is typically leveraged by operational as well as analytical systems.  It is therefore typically stored outside the analytics environment and accessed when required for data integration or analysis.

4. Data Catalog is a repository that contains metadata relating to the data stored in the Analytical Data Lake Storage repositories.  The catalog maintains the location, meaning and lineage of data elements, the relationships between them and the policies and rules relating to their security and management .  The catalog is critical for enabling effective information governance, and to support self-service access to data for exploration and analysis.

5. Data Models provide a consistent representation of data elements and their relationships across the enterprise.  An effective Enterprise Data Model facilitates consistent representation of entities and relationships, simplifying management of and access to data.

6. Data Quality Rules describe the quality requirements for each data set within the Analytical Data Lake Storage component, and provides measures of data quality that can be used by potential consumers of data to determine whether a data set is suitable for a particular purpose.  For example, data sets obtained from social media sources are often sparse and therefore ‘low quality’ but that does not necessarily disqualify a data set from being used.  Provided a user of the data knows about its quality, they can use that knowledge to determine what kinds of algorithms can best be applied to that data.

 

Match and Manage your Data on Cloud

We left the last blog with two questions.

A few weeks back I wrote on IBM Bluemix Data Connect. If you missed it, then watch this video on how you can put data to work with IBM Bluemix Data Connect.

Now, Business Analysts will be able to leverage Entity Matching technology using Data Connect. The Match and Manage (BETA) operation on Data Connect identifies possible matches and relationships (in plethora of data sets, including master data and non-master data sets) to create a unified view of your data. It also provides a visualization of the relationships between entities in the unified data set.

For example, you have two sets of data : One containing customer profile information and the other containing a list of prospects. A Business Analyst can now use intuitive UI to do the Match and Manage operation to match these two data sets and provide insights to questions such as:

  •  Are there duplicates in the prospect list?
  • How many of the prospects are already existing customers?
  • Are there non-obvious relationships among prospects and customers that can be explored?
  • Are there other sources of information within that could provide better insights if brought together?

The two data set are matched using Cognitive capabilities which allows the MDM– matching technology to be auto-configured and tuned to intelligently match across different data sets:

dataconnect

Business Analyst can understand the de-duplicated datasets by navigating through a relationship graph of the data to understand how the entities are related across the entire dataset. Now they can discover new non-obvious relationships within the data that were previously undiscoverable. The following generated canvas enables them to interactively explore relationships between entities.

dataconnect1

In the above example it was illustrated as how clients can now easily understand the data they hold within their MDM repositories and how now they can match their MDM data with other data sources not included within the MDM system. This simplifies the Analytical MDM experience where MDM technologies are accessible to everyone without the need to wait for Data Engineers to transform the data into a format that can be matched and rely on MDM Ninja’s to configure matching algorithms.

Summary:

IBM Bluemix Data Connect provides a seamless integrated self-service experience for data preparation. With addition of entity analytics capability, business users are empowered to gain insight from data that wasn’t previously available to them. Now organizations can extract further value from their MDM data by ensuring it is used across the organization to provide accurate analytics. Entity analytics within Data Connect is now available in beta. Go ahead and experience the next evolution of MDM.

3 Compelling Use cases for Entity Analytics

Entity analytics is used to detect non-obvious relationships, resolve entities, and find threats and vulnerabilities that are hiding in your disparate collections of data. Through the medium of three use cases, let’s try to understand how Entity Analytics can help organizations enhance their customer experience.

entityanalytics1Scenario 1

Entity Analytics can detect non-obvious relationships between entities. It can also analyze new data sources in context leading to new insights and opportunities. In this scenario you have some data in an MDM system and another set of data in a spreadsheet file. Suppose you want to run a marketing campaign to target high-net-worth clients to sell them a premium bank account. The information in the one MDM system in isolation doesn’t give you the needed information. You want to bring these two sources together and determine if you can identify individuals that can be targeted for the new account.

In the MDM system, John Smith lives with Mary Smith. The spreadsheet file shows that John Smyth (spelled differently) is actually a high-net-worth client. Combining this information we can say that John Smith is actually the same person across the data sets. He’s a high-net-worth client, and he has a wife. With this information you want to target Mary Smith with a premium bank account because she lives with a high-net-worth individual. Entity analytics enables you to discover and understand this opportunity.

entityanalytics2

Scenario 2

Entity Analytics can find where threats and vulnerabilities are hiding in big data and respond efficiently. In this scenario for a risk assessor in an insurance firm, severe rainfall is predicted within a geographical area that includes the client’s residential location. When pulling up the client data from MDM and the flood warnings being issued from the environmental agency, we can match across the data sets to identify that a number of properties are at risk. So, the client can then be provided an early warning to help mitigate risk and increase the flood risk value on the client’s property renewal. Also, if you have an elderly customer that is at severe risk; you can take action to notify the emergency service to ensure a proactive resolution to any potential threat.

entityanalytics3

Scenario 3:

Lets see how using Entity Analytics, MoneyGram International Inc.,  a money transfer company gets notified of questionable activities in real time for faster predictive and preventive decision making. This helped them to save $200 million in just two years!

Summary

Entity analytics help organizations by launching more target-oriented campaigns and by reducing the risk of fraud. With the help of entity analytics, organizations can predict and preempt suspicious activity faster and with reduced costs. Entity analytics further help by allowing enterprises to detect the entities that are the same, regardless of whether the entities are hidden or masked. So following questions can be raised:

  • Does this Analytics require an MDM Ninja or can something be set up easily by a Business user?
  • Do we have Entity Analytics available on Cloud for decisions that cannot waaaaiiiiittttt?

Stay tuned for my next blog.

 

Need for Master Data Management

Incidentally this is my 100th blog on this site. Since some time I was reading on Master Data Management and in this blog I wish to explain the Need of Master Data Management (MDM).

Imagine you are a bank with a huge clientele. You want to know how many clients you have? Is it enough to count the number of credit card customers, saving account customers and loan account customers? What if there is an overlap? What if you have information in one account that will help you to serve the customer better in the another account? For example a customer with gold account in Credit card calls but this time as a loan account customer and we fail to recognize this and treat him as ordinary customer? So wouldn’t it be nice for the bank to have a consolidate view of each of the customer? That’s where master data management (MDM) comes in. MDM makes it possible to distill a single view of the client—or of the patient, supplier, partner, account or other critical ‘entity’—from the incomplete or inconsistent bits of data that are scattered across the enterprise. The resulting view, now unified across disparate silos, provides the insight that you need to make better decisions and create superior outcomes.

Master data is the information about customers, products, materials, accounts and other entities that is critical to the operation of the business. But companies hold pieces of master data in many different applications, such as enterprise resource planning (ERP) and customer relationship management (CRM) systems. Each of those source systems creates and holds the data in its own unique way. As a result, information does not match from one system to the next. Critical data elements may be missing, duplicated or inconsistent. Further, each department can only operate from within its own compartmentalized view.MDM_Success

MDM software manages the creation, maintenance, delivery and use of master data, both to ensure that it is consistent and trustworthy, and to make it possible to see the data in an organization-wide context. Consider an insurance company with multiple divisions: Without MDM, an agent in the auto division will offer rates to a prospective client based on the home address and ages of drivers in the household. These standard rates might be higher than those offered by a competitor. What if the company had an MDM “hub” pulling together customer data from across divisions? Then, the agent could see that the customer already owns a  homeowner’s policy, and could offer them discounted, more competitive auto rates. In this simple example, MDM provides a single view of the client that empowers the agent to secure a better business outcome.

So to summarize, MDM delivers a single, unified, trusted version of truth about an organization’s critical entities—customer, supplier, product and more. Armed with this single, trusted view, organizations can make better decisions and improve business outcomes— which can lead to higher revenue, better customer satisfaction, lower cost and lower risk.

Further Reading