Data Governance and Quality
Who can see what, where a number came from and whether it is right should all be answerable in writing and on demand. Lineage, cataloging, access policy, masking and quality observability are configured inside the platform itself.
The problem
Legal asked us to document who can see European customer records and we couldn't answer in writing. Somebody had to go and read the grants one by one.
Governance programs fail in two recognizable ways. The first stays theoretical and produces a policy document plus a catalog that stops being updated after the first quarter. The second is strict enough that teams keep private copies of the data outside the platform. Either way, an entitlement question from legal or an auditor takes days of manual work to answer.
Our approach
We implement governance as configuration inside the platform. A masking policy attached to a column is applied on every query and leaves an audit record. Evidence for an auditor then comes out of the platform's own logs.
- 01
Classification
We find where sensitive data sits, including the copies that were never meant to persist. Classification is recorded as column-level tags inside the platform. Policy is then attached to the tag.
- 02
Access model
We build a role hierarchy of functional roles granted to access roles, which makes entitlements possible to reason about and report on. The design target is that the platform can produce a current list of everyone able to read a given column, on demand and without a manual review.
- 03
Policy enforcement
We apply dynamic masking and row-level security through tag-driven policies, so a newly created column inherits protection from its classification. Sharing paths are tested separately, because a policy that filters correctly inside the owning account can behave differently once the table is shared to another account.
- 04
Lineage and cataloging
We generate column-level lineage from the transformation code, which keeps it current as the models change. The trace runs from the source column to the field on the report.
- 05
Observability
Freshness, volume, schema drift and distribution checks are put in place with routed alerts. The incident process is written down alongside them. That includes who responds and how a data incident is communicated to the business.
How it fits together
What you get
- Data classification applied as platform tags, with the classification standard documented
- Role hierarchy and access model, implemented and documented, with an entitlement report that can be run on demand
- Masking and row-level security policies attached to classified columns
- Column-level lineage generated from transformation code
- Data quality monitors with alert routing and defined ownership
- Data incident response runbook
Technology
Platform controls
- Snowflake dynamic data masking
- row access policies
- object tagging
- Unity Catalog
- Microsoft Purview
Catalog and lineage
- dbt docs
- OpenLineage
- Atlan
- Collibra
- Alation
Quality and observability
- dbt tests
- Great Expectations
- Monte Carlo
- Soda
Engagement
- Duration
- Ten to twenty-four weeks for an initial implementation, commonly followed by a lighter ongoing review
- Team
- A governance lead, a platform engineer and an analytics engineer, working alongside your security and compliance stakeholders
- Starts with
- An access and exposure review comes first. It maps who can currently read what, and where copies of sensitive data have spread to. The findings usually set the scope for everything that follows.