The bottom line
Lakehouse governance is harder than warehouse governance because a warehouse enforced controls structurally — one write path, one schema owner — while a lakehouse lets many engines and teams write the same storage by design. Every control now has to be deliberate. The disciplines that matter: named ownership per domain and data product with an arbiter; a small maintained catalogue of certified tables; access designed across storage, table and row/column levels; classification at ingestion; data quality as contracts with thresholds and owners; lineage for impact and deletion scope; a written delete–purge–vacuum–verify runbook; schema-evolution control; cost as an availability control; and an operating model that survives a departure. For a team of two to four, run the minimum viable version — one page per discipline, not a programme.
In This Article
The old warehouse governed itself, and almost nobody noticed
It governed itself because it was hard to write to. One team owned the loading process; one schema owner controlled structure; one release process gated change. A lakehouse removes that door on purpose — its entire value proposition is that many engines and teams can read and write the same open-format storage: Spark notebooks, SQL endpoints, streaming jobs, BI tools, a data science workspace, an ingestion service, all pointed at one set of files.
This is the platform-neutral version: the disciplines that apply whether you run Microsoft Fabric, Azure Databricks, Snowflake or open Delta Lake on your own storage. Every control that used to come free from write friction now has to be a deliberate decision.
Why is lakehouse governance harder than warehouse governance?
Because a warehouse enforced controls structurally and a lakehouse allows many write paths by design. Three consequences bite mid-market estates first. Multiple write paths mean multiple definitions — two teams can both create dim_customer in the same lakehouse without ever speaking. Convenience becomes the security risk — everything sits in one storage account, so granting one person access to "the lakehouse" is a single click, a very different decision from granting access to fourteen tables, usually made with the same amount of thought. And history persists — Delta keeps prior versions of a table, so a deleted record still exists in time-travel history until it is expired.
Ownership comes before tooling
Lakehouse ownership requires three named roles before any catalogue is purchased. A domain owner, accountable for a business area's data, mapped to how the business is organised (Manufacturing, Supply Chain, Finance) not how IT is. A data product owner, one level down, accountable for a specific curated table or model, with an access policy attached — more useful day to day. And an arbiter, the role most estates skip: when Commercial says a shipment counts on despatch and Finance says it counts on invoice, someone decides, and it is not the data engineer.
Ownership recorded as a team name rather than a person is not ownership — a team cannot be paged, and a team does not settle a definition dispute.
The catalogue: why an unmaintained one is worse than none
A catalogue is only useful when entries are trusted, and an unmaintained one is worse than none because it teaches users that documented metadata is unreliable — once a team learns to ignore catalogue descriptions, they revert to asking colleagues, and the catalogue absorbs maintenance effort while delivering nothing.
The catalogue has three jobs: tell someone what a table means, whether it is fit for use, and who to ask. Most estates do the first badly, skip the second, and never do the third. The fitness signal matters more than the description — Fabric provides endorsement badges (Promoted by anyone with write permission; Certified, restricted to specified reviewers; Master data for authoritative sources). Keep the maintained surface small: twenty certified tables with accurate owners beat 400 auto-scanned assets with blank descriptions.
An unmaintained catalogue is worse than none — it teaches users that documented metadata lies. Twenty certified tables with real owners beat four hundred auto-scanned assets with blank descriptions.
Access control and classification
Access operates at three levels — storage/folder, table, and row/column — and all three must be designed together. The characteristic lakehouse failure is granting broad workspace-wide access because all data conveniently sits in one place: a warehouse grant covered what existed; a lakehouse folder grant covers what will exist. OneLake security supports folder, table, row and column roles across Fabric experiences, with users not assigned to a role seeing no data. The design rule: grant at Gold, restrict at Bronze — almost everyone who thinks they need raw landing data actually needs a curated table with a filter on it. (And know the trade-offs: Unity Catalog time travel does not work on tables with row filters or column masks.)
Classification must be applied at ingestion, not retrofitted — retrofitting means inspecting every column in an estate that has already grown, turning minutes-per-source into a multi-week audit. Classify at source-system level first (this HR extract contains personal data, this SCADA feed does not), then refine at column level only where it matters. Purview labels inherit downstream from Fabric item to Fabric item and to Power BI — but they do not persist in CSV or TXT exports, which is exactly the boundary people cross.
Data quality as a contract, and lineage for operations
Data quality works as a contract with four elements per critical table: the expectation, the threshold at which it fails, the named owner notified, and the agreed action when it breaches. A dashboard without those four produces red indicators nobody is accountable for. "Customer master completeness above 98%, owner: Commercial Operations Manager, action: block Gold refresh and notify" is a contract; a gauge is not. Purview Data Quality supports rules across completeness, consistency, conformity, accuracy, freshness and uniqueness, with scores at rule, asset and data-product level — but the tool is not the hard part, writing the threshold is.
Lineage is used for three operational tasks: impact analysis before a change, root-cause tracing when a number is wrong, and scope determination for deletion and access requests — not for architecture diagrams. Both major platforms capture it automatically, but know the gaps: Unity Catalog does not preserve lineage for renamed objects or jobs submitted through spark-submit, and Purview coverage varies by source. You will find those gaps at the worst moment, so keep a manual register for renames.
Retention, deletion, schema evolution and cost
Honouring a deletion request requires deleting the record from Bronze, Silver and Gold, then physically rewriting the affected Parquet files, then expiring the retained history that still contains them — a DELETE alone removes the row from current queries while leaving the underlying data in place (deletion vectors mark rows modified without rewriting the file). The honest sequence: identify every table holding the record (where lineage earns its keep), delete in all three layers, run REORG TABLE … APPLY (PURGE), run VACUUM, and confirm the retention window has elapsed. Two design decisions make this survivable: keep personal identifiers in as few tables as possible (ideally one keyed reference table the layers join to), and write the runbook before you need it.
Schema-evolution control covers two risks — mergeSchema lets columns be added automatically, which is convenient and is also how an unreviewed upstream change silently enters a governed table. The governance decision: allow auto-evolution in Bronze, require explicit change in Silver and Gold. And cost governance belongs in the model because compute on a shared capacity is a shared resource — one badly written notebook degrades every other workload, so cost control is an availability control: a named capacity owner, a monthly top-consumer review, workspace limits, and sizing heavy workloads before scheduling.
The operating model, and minimum viable governance
A governance operating model needs three cadences: weekly operational checks on quality alerts and failed jobs, monthly review of access grants, cost and new assets, and quarterly review of ownership, classification and retention. For a mid-market team this is not a department — it is one platform owner spending 20–25% of their week, plus named business owners an hour a month each. The only test that matters: if the person who built this left on Friday, what stops working? If the answer includes "nobody would know which tables are certified" or "nobody knows the deletion sequence", that is a dependency, not a governance model.
For a team of two to four, run the minimum viable version — one page per discipline, not a programme.
| Discipline | Minimum viable version for a small team |
|---|---|
| Domain & data product ownership | One named owner per domain and per certified table, in a shared document |
| Definition arbitration | One named business arbiter per contested metric; rulings written down |
| Catalogue & metadata | 20 certified tables with owner, description and refresh cadence; nothing else |
| Access control | Three tiers: raw (engineers), curated (business read), restricted (row-filtered) |
| Classification | Source-level at ingestion; column-level only for personal data |
| Data quality contracts | 5 rules on decision-feeding tables, each with threshold, owner and action |
| Lineage & impact | Automated where available, plus a manual register for renames and gaps |
| Retention & deletion | A written runbook: delete, purge, vacuum, verify — tested once before needed |
| Schema-evolution control | Auto-evolution in Bronze only; Silver and Gold require explicit change |
| Environment & release | Two stages minimum, no direct production edits, code in source control |
| Cost governance | Named capacity owner, monthly top-consumer review, workspace limits |
| Operating model | Weekly, monthly and quarterly cadences written down and diarised |
Where this breaks, and what it does not fix
Governance does not fix a definition the business has not agreed — tooling records a decision, it does not make it. Automated lineage has documented gaps you will find at the worst moment. Deletion cannot be guaranteed across everything downstream — you can purge Delta tables and expire history, but a CSV someone exported last month is beyond your reach. Row and column controls carry functional trade-offs, including breaking time travel.
A small team cannot run twelve disciplines at full depth, and pretending otherwise produces zero — the minimum-viable column is the realistic first-year target. And none of this improves a source system that records data badly: if the MES records downtime reasons as free text and the WMS has no consistent SKU key, governance documents the problem accurately, it does not solve it.
What to do first
Answer these five this week — under an hour, and they locate the gap precisely:
- Name the person — not the team — who would settle a dispute about how OTIF is calculated. If you cannot in ten seconds, ownership is your first gap, not tooling
- Take your three most-used tables: who owns each, when did each last refresh successfully, and can a business user see that without asking IT?
- If a deletion request arrived today naming one individual, write the sequence across Bronze, Silver, Gold and retained history. If it is not written, that is your highest-risk gap
- List everyone with write access to your lakehouse storage — was each grant a decision, or a convenience?
- If your platform owner left on Friday, name the three things that stop working. Document those first
We set this up during the build rather than after it, because retrofitting governance onto a live estate costs two to three times what designing it in does — usually a Fractional Data Consultant shape: set the disciplines, hand over the runbooks, stay available for the quarterly review.
The fastest way to find your biggest governance gap is the departure test: if the platform owner left on Friday, what stops working? If "nobody knows which tables are certified" or "nobody knows the deletion sequence" is on the list, start there. Book a diagnostic with Amit — no slides, no pitch deck, no obligation to proceed. We set governance during the build, because retrofitting it costs two to three times as much.
Free Assessment
Where does your operation sit on the data maturity curve?
8 questions. 3 minutes. You get a scored breakdown across data infrastructure, analytics readiness, and automation potential — with a specific next step for your industry.