The Enterprise Data Architecture Framework: Connecting Business, Operational and Industry Data

Most industrial businesses are not short of data. They are short of connected, trusted data they can act on.

A typical mid-market group runs finance in SAP S/4HANA or Dynamics 365, customer records in a separate CRM, warehouse movements in a WMS, transport in a TMS, production on an MES, and project controls in Primavera P6 — with the gaps filled by spreadsheets and SharePoint. Each system works on its own. The trouble starts the moment a question crosses two of them.

• Which customers are actually profitable once logistics and operational cost is loaded back in?
• Why is inventory rising while service levels fall?
• Which projects will overrun, judged on procurement, resource and schedule performance together?
• Which production and supply-chain factors are quietly eroding margin?

None of those is a dashboard problem. Answering them needs an architecture that connects the business — not another report stacked on top of one system. That is what this framework is for. It is not about replacing the systems you run; it is about connecting the data they generate and turning it into decisions.

The MyData Insights Enterprise Data Architecture Framework is a blueprint for building a data platform around how an organisation runs — not around a single reporting request.

It connects one chain: business and operational systems, then integration, then a governed data platform, then reusable business data models, then analytics, then AI, then decisions. Instead of a separate stack for finance, another for supply chain and a third for projects, you build one enterprise foundation and reuse it.

The framework has eight layers:

• Business and operational source systems
• Data integration and ingestion
• Bronze — raw data
• Silver — standardised, integrated data
• Gold — business data products
• Semantic and KPI layer
• Analytics and decision intelligence
• AI, automation, governance and data operations

The platform layers stay industry-neutral. What changes from one business to the next is the business data model, the source systems, the KPIs and the decisions the platform has to support.

Every business has a different technology landscape, but the shape is familiar. Enterprise applications — ERP, CRM, finance, HR, procurement — sit on SAP, Dynamics 365, Oracle, NetSuite or Business Central. Supply-chain systems cover the WMS, TMS, demand and supply planning, and supplier management. Operational systems vary by industry: MES, SCADA, IoT, fleet management, POS, field service and asset management.

For engineering, procurement and construction businesses, the estate adds Primavera P6, project controls, cost management, engineering systems and document management. And behind all of it sit the usual unofficial systems — Excel, CSV extracts, SharePoint, partner feeds, SaaS APIs and legacy databases.

The framework starts by mapping what data exists, where it lives, and which business process it supports. You cannot connect what you have not first named.

Connecting sources is the part most teams get wrong — usually by treating every feed the same way. The framework supports three patterns and picks per source.

Batch suits finance, historical ERP data, HR, periodic planning and master data. Incremental — Change Data Capture, change tracking, watermarks, modified-timestamps or incremental APIs — pulls only new or changed records instead of re-extracting whole tables, which cuts data movement, processing time, source-system load and infrastructure cost. Streaming fits IoT, fleet telemetry, machine events and real-time logistics, where a decision genuinely depends on the last few minutes.

The principle is simple and it saves real money: use the movement pattern the business requirement needs — not the one a platform happens to support.

The platform organises data in three refinement tiers — the medallion architecture — so trust is built in stages rather than assumed.

The Bronze layer is a landing zone. Data is kept close to its original form to support traceability, reprocessing, auditing, historical analysis and troubleshooting. Raw ERP, raw logistics, raw project and raw production data all land here without being reshaped.

The Silver layer is where fragmented source data becomes enterprise information. Data is cleansed, standardised, deduplicated, validated, integrated and conformed. Where three systems each hold a different customer identifier, Silver resolves them into one representation — customer to orders to deliveries to payments to service. The same applies to supplier-to-inventory and project-to-cost chains. For any group running multiple systems, business units or locations, this layer does the heavy lifting.

The Gold layer turns standardised data into reusable business datasets — the subject of the next section.

The Gold layer converts standardised data into business data products: governed, reusable datasets built around a domain, not around one report.

Rather than designing the platform around individual dashboards, design it around domains that many applications can consume:

• Finance — revenue, cost, gross margin, receivables, payables, cash flow
• Customer — orders, revenue, profitability, service, retention
• Supply chain — demand, procurement, inventory, supplier, fulfilment, delivery
• Logistics — shipment, route, carrier, vehicle, delivery, freight cost
• Project — project, WBS, activities, resources, cost, schedule, progress
• Operations — production, equipment, quality, downtime, productivity

Build these once and a dozen analytical applications can draw on the same governed data. Build a pipeline per report instead, and you inherit dozens of pipelines with the same business logic copied — and quietly diverging — inside each one.

The strength of the framework is that the platform stays consistent while the business domain changes. The systems and the metrics differ; the layers do not.

This is where the platform stops being a generic data warehouse and becomes a business-specific decision platform. The engineering is reused; the meaning is local.

Two problems break cross-functional analytics more often than any technical fault: entities described differently across systems, and metrics calculated differently across teams.

The same customer might be "Customer A" in the ERP, "CUST-1001" in the CRM and "CUSTOMER-001" in the TMS. Without a common enterprise definition of customer, supplier, product, material, location, plant, vehicle, equipment, project and cost centre, every cross-system number is a negotiation.

A shared platform alone does not guarantee a single version of the truth — business definitions have to be standardised too. Finance and Sales calculate revenue differently. Supply Chain and Logistics define on-time delivery differently. Project Controls and Finance measure project progress differently. The semantic layer fixes the definitions of KPIs, measures, business rules, dimensions and hierarchies in one place, so every application consumes the same governed metric. Raw data becomes business data, business data carries a KPI definition, and the semantic model serves Power BI, analytics and AI from one source.

A common mistake is to measure success by the number of dashboards shipped. A better question is: what decision does this data help someone make?

Take a logistics example and walk it up the ladder. Traditional reporting says 15% of shipments were delayed. Analytics says 65% of those delays came from three routes. Advanced analytics shows the delays on those routes track specific carrier and time-window patterns. Decision intelligence acts: reallocate shipments and adjust carrier capacity on those routes.

The value was never the dashboard. It was the decision the data made possible. The framework is deliberately built to carry data all the way from reporting, through analytics, to a decision someone owns.

AI does not rescue a weak data foundation — it exposes it. Predictive models and assistants need reliable data, historical context, business definitions, lineage, security, governance, metadata and trusted KPIs. Every one of those is a property of the layers below, not of the model on top.

Once the foundation exists, the AI work becomes tractable: demand, inventory, delivery, project-delay and equipment-failure prediction on the predictive side; assistants that answer "which customers had declining margins last quarter?" or "why did logistics cost rise this month?" against governed data; and agents that work across trusted business data to investigate a question and surface a trend.

Treat AI readiness as a design constraint on the platform from day one, and the AI roadmap stops being a science project. Bolt it on afterwards, and you rebuild the foundation anyway.

The moment data is centralised, weak governance turns a data platform into a centralised data problem. The framework treats four things as first-class.

Ownership — a named owner for customer, product, financial, project and operational data. Quality — completeness, accuracy, timeliness, consistency and uniqueness, monitored not assumed. Security — role-based access, row-level security, and business-unit, plant, location and department boundaries, with sensitive data protected. Lineage — the ability to trace an executive KPI back through the semantic model, the gold product, the silver dataset, the bronze record and the source system.

That lineage does double duty: it builds trust when the number is questioned, and it collapses troubleshooting time when something breaks. Tooling such as Microsoft Purview supports this, but governance is a discipline first and a product second.

No business needs a sophisticated lakehouse the week it starts. The framework is deliberately maturity-driven — you enter where you are and progress as the value justifies it.

The point is not to reach level five fastest. It is to sit honestly at your current level, then take the next step that pays for itself.

The approach starts with the business, not the platform. Six steps, sequenced so value arrives before the full build is done.

• Discovery — business processes, data sources, existing reports, critical KPIs, data owners, integration needs and pain points
• Current-state assessment — architecture, data quality, integration, reporting, governance, technical debt and scalability
• Target-state architecture — data platform, integration, lakehouse, data model, semantic layer, governance, analytics and AI roadmap
• Data engineering — pipelines, incremental ingestion, transformation, data quality and business data products
• Analytics — executive, functional and operational dashboards, KPI frameworks and self-service
• AI and automation — predictive analytics, assistants, agents and intelligent automation, once the foundation is stable

We design the architecture before we choose the technology, and we prove value on one business domain before expanding across the rest. First working output measured in weeks, not a fifty-slide roadmap.

Most failures are not technical. They are sequencing and discipline failures, and they repeat across industries.

• Starting with dashboards before designing the data foundation — which guarantees duplication and technical debt
• Building one pipeline per report, so the same business logic is copied into dozens of pipelines
• Ignoring master data, which makes every cross-functional number unreliable
• Treating every requirement as real-time, adding cost and complexity where no decision needs it
• Ignoring governance, turning a central platform into a central problem
• Selecting technology before defining the architecture, because a popular platform is not the same as a fitting one
• Measuring success by dashboard count instead of outcomes

On that last point: a business can run 200 dashboards and still have no decision intelligence. Measure the outcomes instead — faster decisions, less reporting effort, better data quality, wider visibility, lower operational cost, sharper forecast accuracy and higher productivity.

The whole framework reduces to nine layers, each with one job.

The stack is technology-agnostic. A Microsoft-centric build might run Azure Data Factory for integration, OneLake and Azure Data Lake Storage for storage, Microsoft Fabric or Azure Databricks for the lakehouse, Azure SQL or a Fabric Warehouse for the database, Power BI for the semantic and analytics layer, Microsoft Purview for governance, and Power Automate for automation. Other estates run Snowflake, BigQuery, Databricks or hybrid architectures. The principle holds either way: architecture first, technology second. And when the task is consolidating an older estate onto this platform, our Fabric migration framework sets out the path — assess, prioritise, migrate, validate.