Entity analytics is used to detect non-obvious relationships, resolve entities, and find threats and vulnerabilities that are hiding in your disparate collections of data. Through the medium of three use cases, let’s try to understand how Entity Analytics can help organizations enhance their customer experience.
Due to new modernization strategies, data analytics is architected from top down or through the lens of the consumers of the data. In this blog, I will describe the four roles that are integral to the data lifecycle. These are the personas who interact with data while uncovering and deploying insights as they explore this organizational data.
Citizen analysts/knowledge workers
A knowledge worker is primarily a subject-matter expert (SME) in a specific area of business—for example, a business analyst focused on risk or fraud, a marketing analyst aiming to build out new offers or someone who works to drive efficiencies into the supply chain. These users do not know where or how data is stored, or how to build an ETL flow or a machine learning algorithm. They simply want to access information on demand, driving analysis from their base of expertise, and create visualizations. They are the users of offerings like the Watson Analytics.
Data scientists can do a more sophisticated analysis, find a root cause to a problem, and develop a solution based on an insight that he discovers. They can use SPSS, SAS, etc or open source tools with built-in data shaping and point-and-click machine learning to manipulate large amount of data.
They focus enable data integrations, connections (plumbing) and data quality. They do the underlying enablement that a data scientist and citizen analyst depend on. They typically depend on solutions like DataWorks Forge to access multiple data source and to transform them within a fully managed service.
Application developers are responsible for making analytics algorithms actionable within a business process, generally supported by a production system. Beginning with the analytics algorithms built by citizen analysts or data scientists, they work with the final data model representation created by data engineers, building an application that ties into the overall business process. They use something like Bluemix development platform and APIs for the individual data and analytics services.
Putting it all together
Image a scenario where a Citizen analyst notices (from a dashboard) that retail sales are down for the quarter. She pulls up Watson Analytics and uses it to discover that the underlying problem is specific to a category of goods and services in store in a specific region. But she needs more help to find the exact cause and a remedy.
She engages her data scientists and engineer. They discuss the need to pull in more data than just the transactional data the business analyst already has access to, specifically weather, social, and IoT data from the stores. The data engineer helps create the necessary access – the data scientists can then form and test various hypothesis using different analytic models.
Once the data scientist determines the root cause, he then shares the model with the developer who can then leverage it to improve the company’s mobile apps and websites to be more responsive in real-time to address the issue. The citizen analyst also shares the insight with the marketing department so they can take corrective action.
For data integration projects, DataStage has been the work horse for many years. It is used by Data Engineers to extract data from many different sources, transform and combine the data, and then populate them for applications and end users. DataStage has many distinct advantages over other popular ETL tools.
Until recently, these capabilities were only available with the on-premises offering. Now DataStage is available on the Cloud as a hosted cloud offering. Customers can take advantage of the full capabilities of DataStage and without the burden and time consumption of standing up the infrastructure and installing the software themselves. Customers can quickly deploy a DataStage environment (from ordering to provisioning it on the cloud) and be up and running in a day or less. There is no up-front capital expenditure as customers only pay a monthly subscription based on the capacity they purchase. Licensing is also greatly simplified.
Using DatasStage on Cloud, existing DataStage customers can start new projects quickly. Since it is hosted in the IBM cloud, the machine and operating system are managed by IBM. The customer will not have to spend time to either increase the current environment or create a new one. In other words, Cloud elasticity makes them ready to scale and handle any workload. DataStage ETL job developers can immediately be productive and the data integration activities can span both on-premises and cloud data if necessary, as the DataStage jobs can be exported from the cloud and brought back to an on-premises DataStage environment.
As an example; A customer has data sources such as Teradata, DB2, etc. in their data center as well as SalesForce, MongoDB and other data residing in the Cloud. They need access to their existing data sources and their cloud data sources for a new customer retention project . This project requires some sophisticated data integration to bring it all together but they don’t have the IT resources or budget to stand up a new data integration environment in their own data center for this project. So, an instance of DataStage on the Cloud can be deployed for their use. The customer can access the DataStage client programs on the Cloud to work with DataStage. The access would be either through the public Internet or a private connection via the SoftLayer VPN. DataStage ETL jobs running in the Cloud can access the customer’s on-premise data sources and targets using secured protocols and encryption methods. In addition, these DataStage jobs can also access cloud data sources like dashDB as well as data sources on other cloud platforms using the appropriate secured protocols.
- Extend your ETL infrastructure: Expand your InfoSphere DataStage environment or begin transitioning into a private or public cloud with flexible deployment options and subscription pricing.
- Establish ad hoc environments: Extend your on-premises capacity to quickly create new environments for ad hoc development and testing or for limited duration projects.
- Start new projects in the cloud: Move straight to the cloud without establishing an on-premises environment. Realize faster time-to-value, reduce administration burden and use low-risk subscription pricing.
Go here for more information: https://developer.ibm.com/clouddataservices/docs/information-server/
To stay competitive and reduce cost, several Enterprises are realizing the merits of moving their data to Cloud. Due to their economies of scale cloud storage vendors can achieve lesser cost. Also Enterprises escape the drudgery of [capacity] planning, buying, commissioning, provisioning and maintaining storage systems. Data is even protected by replication to multiple data centers which Cloud vendors provide by default. You can read this blog listing the various advantages to move data to cloud.
But now the BIG challenge is to securely migrate the terabytes of Enterprise data to Cloud. Months can be spent coming up with airtight migration plan which does not disrupt your business. And the final migration may also take a long time impacting adversely the users, applications or customers using the source database.
Innovative data migration
In short, database migration can end up being a miserable experience. IBM Bluemix Lift is a self-service, ground-to-cloud database migration offering from IBM to take care of the above listed needs. Using Bluemix Lift, database migration becomes fast, reliable and secure. Here’s what it offers:
- Blazing fast Speed: Bluemix Lift helps accelerate data transfer by embedding the IBM Aspera technology. Aspera’s patented and highly efficient bulk data transport protocol allows Bluemix Lift to achieve transport speeds much faster than FTP and HTTP. Moving 10 TB of data can take a little over a day, depending on your network connection.
- Zero downtime: Bluemix Lift can eliminate the downtime associated with database migrations. An efficient change capture technology tracks incremental changes to your source database and replays them to your target database. As a result, any applications using the source database can keep running uninterrupted while the database migration is in progress.
- Secure: Any data movement across the Internet requires strong encryption so that the data is never compromised. Bluemix Lift encrypts data as it travels across the web on its way to an IBM cloud data property.
- Easy to use: Set up the source data connection, provide credentials to the target database, verify schema compatibility with the target database engine and hit run. That’s all it takes to kick off a database migration with Bluemix Lift.
- Reliable: The Bluemix Lift service automatically recovers from problems encountered during data extract, transport and load. If your migration is interrupted because of a drop in network connectivity, Bluemix Lift automatically resumes once connectivity returns. In other words, you can kick off a large database migration and walk away knowing that Bluemix Lift is on the job.
Speed, zero downtime, security, ease of use and reliability—these are the hallmarks of a great database migration service, and Bluemix Lift can deliver on all these benefits. Bluemix Lift gets data into a cloud database as easy as selecting Save As –> Cloud. Bluemix Lift also provides an amazing jumping-off point for new capabilities that are planned to be added in the future such as new source and target databases, enhanced automation and additional use cases. Take a look at IBM Bluemix Lift and give it a go.
I have been tracking the development on IBM Bluemix Data Connect quite closely. One of the reason is that I was a key developer in the one of the first few services that it launched almost two years back under the name of DataWorks. Two weeks back I attended a session on Data Connect by the architect and saw a demo. I am impressed at the way it has evolved since then. Therefore I am planning to re-visit DataWorks again, now as IBM Bluemix Data Connect. In this blog I will reconcile the role that IBM Bluemix Data Connect play in the era of cloud computing, big data and the Internet of Things.
Research from Forrester found that 68 percent of simple BI requests take weeks, months or longer for IT to fulfill due to lack of technical resources. So this entails that the enterprises must find ways to transform line of business professionals into skilled data workers, taking some of the burden off of IT. It means business users should be empowered work with data from many sources—both on premises and in the cloud—without requiring the deep technical expertise of a database administrator or data scientist.
This is where cloud services like IBM Bluemix Data Connect comes into picture. It enables both technical and non-technical business users to derive useful insights from data, with point and click access—whether it’s a few Excel sheets stored locally, or a massive database hosted in the cloud.
Data Connect is a fully managed data preparation and movement service that enables users to put data to work through a simple yet powerful cloud-based interface. The design team has taken lot of pain to design the solution in most simplistic way, so that a basic user can quickly get started with it. Data Connect empowers the business analyst to discover, cleanse, standardize, transform and move data in support of application development and analytics use cases.
Through its integration with cloud data services like IBM Watson Analytics, Data Connect is a seamless tool for preparing and moving data from on premises and off premises to an analytics cloud ecosystem where it can be quickly analyzed and visualized. Furthermore, Data Connect is backed by continuous delivery, which adds robust new features and functionality on a regular basis. Its processing engine is built on Apache Spark, the leading open source analytics project, with a large and continuously growing development community. The result is a best-of-breed solution that can keep up with the rapid pace of innovation in big data and cloud computing.
So here are highlights of IBM Bluemix Data Connect:
- Allow technical and non-technical users to draw value from data quickly and easily.
- Ensure data quality with simple data preparation and movement services in the cloud.
- Integrate with leading cloud data services to create a seamless data management platform.
- Continuous inflow of new and robust features
- Best-of-breed ETL solution available on Bluemix – IBMs Next-Generation Cloud App Development Platform
It’s estimated that cyber-crime will cost the global economy more than $2 trillion by 2019—and could be the greatest threat to every company in the world! Since I have your attention now, lets spend some more time detailing this statement that I made.
Organized criminals, hacktivists, governments and adversaries are compelled by financial gain, politics and notoriety to attack organizations’ most valuable assets. Their operations are well funded and business-like. However, cybersecurity solutions of the past often fail to protect against increasingly sophisticated attack methods. The result is more severe security breaches happening more frequently (50% increase in past year alone) which are geared towards stealing higher-value records such as health-related personally identifiable information (PII) and other highly sensitive data.
3 Top Security concerns of companies:
- Optimizing their security program to unite security silos, reduce complexity and lower costs as opposed to buying dozens of products from dozens of vendors to nail down all kind of security risks.
- Stopping advanced threats to prevent, detect and respond to known and unknown threats throughout the entire attack lifecycle. Currently on average a company takes 201 days to identify a data breach!
- Safeguarding critical assets across users, applications, sensitive data and infrastructure, in the cloud, on mobile or on-premise. Note more than 55 percent of all attacks are carried out by malicious insiders or inadvertent actors
Protecting against security threats:
Cybersecurity can be compared to an immune system. When you get a cold or virus, the immune system understands the virus and sends data through your central nervous system (security intelligence) to create white blood cells and antibodies to gather info, prioritize and take action. This is what’s called the “immune response.” Faced with an attack—whether to a server, mobile device, application, etc.—an effective security solution should detect that breach and work to stop it, similar to how an immune system would function.
Continuing with our comparison to fighting infection, think about how global health organizations share data to stop a pandemic. For cybersecurity, a partner ecosystem serves that same purpose. When organizations collaborate and share their data on new or changing threats, the other organizations can more quickly assess their own vulnerability based on that data and put appropriate protections in place.
Why Cognitive security?
Because even with major advances in technology, the tools available to today’s security analysts aren’t enough to keep up with the number of security events they see every day. There’s a tremendous amount of security knowledge created for human consumption, yet most of it is untapped. 80% of all data on the internet, including blogs, articles, videos, reports, alerts and other information cannot be processed by traditional security tools, and it’s this unstructured data which often proves most valuable in detecting and stopping threats before they cause harm.
Cognitive computing has the ability to tap into and make sense of all of the security data that has previously been dark. A cognitive systems continues to observe events and behaviors—distinguishing the good from the bad—the ability to use integrated defenses to block new threats gets stronger and stronger. By helping to make security analysts more effective and accelerating the response to emerging threats, cognitive security will help to address the current security skills gap, bringing heightened levels of confidence and risk control.
Watson is a technology that can learn, compare, analyze, reason, justify and process human language, so there must be countless ways to create benefit from it. Watson doesn’t replace people, but acts as a trusted advisor that can quickly read and understand massive amounts of information and provide suggestions or warnings to help people make the best decisions. In fact, Watson is already interacting with 1 billion people in 8 languages. Let us look at some specific Watson offerings that are helping clients.
Some specific Watson offerings helping clients
Enables subject matter experts and developers to teach Watson the linguistic nuances of industries and knowledge domains—without writing a single line of code. Click here to get a series of videos on Watson Knowledge Studio. It can help an auto manufacturer to proactively identify safety defects using traffic incident reports as shown below.
Watson Company Analyzer
Watson Company Analyzer applies cognitive technology to enhance understanding of business accounts – creating holistic company profiles. It does this by collecting, filtering and analyzing structured and unstructured information. By building these comprehensive company profiles enables companies to better understand customers and accelerate opportunities.
Watson Engagement Advisor
Interacts with customers, listens to questions and offers solutions, learning with every human interaction and growing its collection of knowledge—quickly adapting to the way people think.