How to Modernize Enterprise Data and Analytics Platform (Part 1 of 4)

Data and Analytics Platform is a sub platform of the Enterprise Digital Business Technology Platform. It contains information management and analytical capabilities.

Introduction

Building the digital business technology platform is a core aspect of enterprise endeavours to support its digital business transformation and therefore gain a sustainable competitive advantage. The digital business technology platform includes 5 building blocks: the information systems platform, the customer experience platform, the internet of things (IoT) platform, the ecosystem platform, and the data and analytics (D&A) platform. The D&A platform is located at the heart of the enterprise digital business technology platform.

In many cases, modernizing the traditional D&A platform is one of the essential steps an enterprise should take in order to build its digital business platform, therefore enabling its digital business transformation and gaining a sustainable competitive advantage. Figure 1 depicts an example of a roadmap for building the enterprise digital business platform.

Figure 1: Example of a roadmap for building the enterprise digital business platform

So, this series of articles is intended to help enterprises determine what parts of the existing traditional D&A capabilities are missing or need to be improved and modernized. It provides conceptual-level reference architectures of traditional and modern D&A platforms. It also explains how these reference architectures can be used (along with the business strategy) to determine what parts of the traditional D&A platform are missing or need to be improved and modernized.

Part 1 of this series provides a conceptual-level reference architecture of a traditional D&A platform.

Part 2 provides a conceptual-level reference architecture of a modern D&A platform.

Parts 3 and 4 explain how these two reference architectures can be used to modernize an existing traditional D&A platform. They illustrate this by providing an example of a Transmission System Operator (TSO) that modernizes its existing traditional D&A platform in order to build a cyber-physical grid for Energy Transition. However, the approaches used in this example can be leveraged as a toolkit to implement similar work in other vertical industries such as transportation, defence, petroleum, water utilities, and so forth.

It is worth noting that in this series, traditional D&A capabilities that normally exist in many enterprises will be referred to as “traditional Data & Analytics platform” and that for simplification purposes – although they may not be built on services-based principles and architecture.

Reference Architecture of Traditional Data and Analytics Platform

The reference architecture of traditional D&A platform comprises several functional capabilities (see Figure 2).

Figure 2: Reference architecture of traditional D&A platform

(The dashed-line rectangle surrounds the other sub platforms that should be connected to the D&A platform after it is modernized)

From an enterprise architecture perspective, each D&A capability shown in Figure 2 includes a grouping of functionality and can be considered as an Architecture Building Block (ABB). ABBs are mapped into real products or specific custom developments (Solution Building Blocks). Following are the most common ABBs of traditional D&A platform reference architecture, sorted in alphabetical order:

Advanced Analytics is a collection of related analytic techniques and tools, which usually includes predictive analytics, data mining, statistical analysis, and complex SQL

Alerts is used to deliver automatic notifications via reports, dashboards, email, or mobile device according to a set of prespecified conditions or rules

Analytic Services is used to provide the ability to embed analytics capabilities into other software applications

Big Data Analytics is applications used to apply advanced analytics into big data sets

Big Data Distributed Cluster is used to support big data analytic applications by obtaining results and reporting them to an interface (e.g. Hadoop MapReduce processing clusters)

BPA (Business Process Analytics) is used to analyse business processes. This class of analytics normally presents a part of a business process management solution

CDC (Change Data Capture) is used to track and determine the data that has changed so that action(s) can be taken using it

CEP (Complex Event Processing) is an event-driven computation capability used to track and process real-time events. It supports continuous-intelligence applications used to enhance situation awareness and support real-time decisions.

Collaboration is used to provide capabilities such as annotations, discussion threads, blogs, wikis, and Really Simple Syndication (RSS)

Dashboards typically display information into a user interface to focus user attention on important exceptions, trends and conditions

Data Modelling is used to build different levels and types of data models (e.g. conceptual, logical and physical level data models; relational and dimensional data models)

Data Virtualization is used to provide virtual views that represent information in forms that applications and users need in real time or near-real time

Data Warehouse as defined by Bill Inmon is a subject-oriented, integrated, time-variant and non-volatile collection of data in support of management’s decision-making process

Discovery is used to enable users to develop and refine views and analyse structured and unstructured data by visually navigating data or applying guided advanced analytics

DQ (Data Quality) is used to address various aspects of the data quality problems such as data profiling, cleansing and enrichment

ECM (Enterprise Content Management) is used to create, store, distribute, discover, archive and manage unstructured content

EMM (Enterprise Metadata Management) is used to manage the descriptive data about information objects. It supports establishing policies and processes that ensure information can be integrated, accessed, shared, linked, analysed and maintained to best effect across the organization.

ETL (Extract, Transform and Load) is used to automate the process of finding data, integrating it and placing it into a data warehouse

ILM (Information Lifecycle Management) is used to align the business value of information with the most appropriate and cost-effective IT infrastructure from the time information is created through its final destruction.

IMD (In Memory Database) is a full-featured, standalone database running in-memory that supports data processing APIs (e.g. SQL)

MDM (Master Data Management) is used to ensure the uniformity, accuracy, stewardship, semantic consistency and accountability of the enterprise’s shared master data

Mobile Apps is used to extend portal and collaboration capabilities to mobile devices so that mobile users can access the same content

Networking Services is used to provide robust interconnectivity between the D&A platform (after modernizing it) and the other platforms of the enterprise digital business platform (i.e. the customer experience platform, the ecosystem platform, the information system platform, the IoT Platform) and their associated subsystems

ODS (Operational Data Store) is used to support operational decision support system applications

Office Applications is used to deliver analytic results through industry standard office applications such as MS Word, Excel and PDF files

OLAP (Online analytical processing) is used to enable users to analyse multidimensional data interactively from multiple perspectives

PM (Performance Management) is used to monitor and manage the business performance of an enterprise

Portals is a website with wide range of content, services and links, used to offer such services as Web searching, news and communications capabilities including e-mail and chat rooms.

Query & Reporting is used to provide analytical and operational reports and ad-hoc queries

RT Operational Intelligence is a real-time dynamic business analytics that delivers visibility and insight into data, streaming events and business operations

Taxonomy & Auto Classification

(Taxonomy) is a large classification of data used to organize and understand textual, unstructured data (equivalent to a data model for structured data).

(Autoclassification) capability is used to classify textual, unstructured data according to specific taxonomies.

Conclusion

The traditional D&A platform reference architecture includes four main architectural building blocks, namely, the data management, the analytics, the analytics applications, and the delivery. In many cases, for an enterprise to build its digital business technology platform, it must modernize its traditional D&A architecture. Such modern D&A platform should be built on a services-based principles and architecture. It should also be able to provide the context for compliance and reporting; real-time event analysis and adjustments to processes; data and models that allow decisions to be made; and algorithms that automate decisions and prescribe courses of action that can be executed in all of the other digital business sub-platforms. The next article (Part 2) will provide a reference architecture of a modern D&A platform, and Parts 3 and 4 will explain how these two reference architectures can be used to modernize an existing traditional D&A platform.

Credit: Data Science Central By: Alaa Mahjoub