Conceptual Modeling for Data Vaults

Data Vault has become a key technology in the enterprise data pipeline because it is focused on business needs, provides an agile and manageable integration process, and meets historical reporting requirements. This architecture manages enterprise data in multiple different forms:

Normalized data held in operational systems
Semi-structured data available from diverse sources
Data Vault data in the enterprise data warehouse
Dimensional data in data marts for reporting/business intelligence

None of these forms provides a viable solution to the problem of managing data semantics throughout the lifecycle, which leads to the resultant transformation processes being slow, ad-hoc and degrading the value of the data asset. We show how Fact-Based Modeling can be used as a unified modeling approach that maintains conceptual integrity across the entire data lifecycle. Because a Fact-Based Model is more fully decomposed than other forms of model, it can be used as a Rosetta Stone to describe all the other composite forms. We can generate the transformations required to perform a Data Vault “decomposition” from the source systems. In fact, all forms of decomposition and recomposition can be unified around the underlying semantics of the conceptual data model, including the design of normalized, messaging, vault or dimensional schemas, and the generation of ETL code to transform between these forms.

Case studies will be provided from the Justice and Insurance industry domains.

Clifford Heath is a software innovator, toolmaker, product architect and designer. Clifford is a masters level CDMP, a Fact-Based Modeling expert participating in the Working Group for standardisation, the inventor of the Constellation Query Language and the architect behind the Infinuendo system. Clifford has applied Fact-Based Modeling to a wide range of systems development and data integration environments that have all contributed to the breadth of his approach. Clifford has presented and published at international scientific and data modeling conferences, workshops and events.

Dr. Graeme Port is a senior IT manager and leader specialising in enterprise-scale software development and systems deployment. Graeme has held a wide range of leadership roles in product development, marketing and sales, and has held leadership roles in major systems implementation in both the commercial and government sectors. Graeme received his PhD from the University of Melbourne in the field of Logic Programming.