Unifying Data

Data will live in disparate systems because organisations utilise an array of platforms, systems and services.

Extracting, transforming and loading data into one unified location source for analytics, data sharing, data science, machine learning and prediction is one of the many data hurdles every large organisation faces today.

Data Systems and Architectures

Today's best in class data systems involve being cloud first, separating compute and storage and abstracting data from source systems.

Organisations build Data Warehouses or Data Lakehouses to store and centralise massive amounts of data across their organisation. The goal of centralising data is to perform fast analytics on data that is as close to real-time as possible to drive decision making for analysts and business units down to the lowest level employee.

Data Wrangling

Significant efforts are made to setup the Organisation's ETL layer commonly known as Extract, Transform and Load.

ETL exists because in larger organisations, data lives in siloed source systems. The schema, format, fields, data structures may be incomparable. The first step is to extract the data into a staging platform where transforms can be applied.

Once data transforms have been completed, the data can be loaded into a data warehouse for various activities some of which could involve machine learning and predictive analysis.