Data governance works in practice — but only when it is treated as an engineering problem, not a policy problem. Most programmes fail not because the framework is wrong, but because governance stays in documents and never reaches the pipelines, the catalogues, or the people who actually touch the data. For data engineering teams in manufacturing, energy, and agritech, the difference between governance that works and governance that gathers dust comes down to how it is implemented.
Why do so many data governance programmes fail to deliver?
Most data governance programmes fail because they are designed by committees and never adopted by engineers. Governance becomes a set of Word documents, a steering group that meets quarterly, and a data dictionary nobody updates. The result is a framework that exists on paper but has zero influence on how pipelines are built, how data is labelled, or how quality is enforced.
There are a few specific failure patterns worth naming:
- Top-down mandates without tooling. Telling engineers to “document their data assets” without giving them a catalogue or integrating documentation into their workflow guarantees non-compliance.
- Ownership without accountability. Assigning data owners in a spreadsheet means nothing if those owners have no mechanism to actually enforce quality rules or flag issues.
- Governance divorced from engineering. When governance is managed by a separate team that does not deploy code, it cannot keep pace with the data platform it is supposed to govern.
The fix is straightforward in principle: governance rules need to live in code, not in policy. Quality checks need to be enforced in the pipeline, lineage tracked automatically, and ownership baked into the data catalogue. That is when governance stops being overhead and starts being infrastructure.
What does data governance actually include?
Data governance is the set of policies, processes, roles, and technical controls that determine how data is created, stored, accessed, used, and maintained across an organisation. In practice, it covers data quality management, data lineage, metadata management, access control, and data ownership — all working together to ensure data is trustworthy and usable.
Breaking that down into concrete components:
- Data quality rules — defined standards for completeness, accuracy, and freshness, enforced at the pipeline level
- Data lineage — a traceable record of where data comes from, how it has been transformed, and where it flows
- Metadata management — descriptions, tags, and classifications that make data discoverable and understandable
- Data ownership — clear assignment of who is responsible for each domain or dataset
- Access control — role-based permissions that determine who can read, write, or modify data
- Compliance controls — audit trails and documentation required by regulations such as NIS2 or the EU AI Act
For teams working with OT data sources like SCADA systems, IoT sensors, and MES platforms, governance also needs to cover the boundary between operational technology and the data platform — defining how raw sensor data is normalised, validated, and contextualised before it enters analytical workflows. This is a core part of what robust software and data engineering looks like in practice for industrial environments.
What’s the difference between data governance and data management?
Data management is the broader discipline of handling data across its entire lifecycle — ingestion, storage, processing, and delivery. Data governance is the framework of rules and accountability structures that sits inside data management and ensures data is handled correctly. Think of data management as the engine and data governance as the quality controls built into it.
A practical way to see the distinction: a data engineering team building ETL pipelines is doing data management. When they add automated quality checks, document the schema in a catalogue, and assign a domain owner who gets alerted when a pipeline breaks, they are applying data governance within that data management work.
The two are inseparable in a well-run data platform. Governance without management has nothing to govern. Management without governance produces data that nobody trusts.
How do you know if your organisation actually needs data governance?
If different teams in your organisation regularly disagree about which numbers are correct, your organisation needs data governance. Other clear signals include pipelines that break overnight without anyone noticing, multiple versions of the same dataset with no clear source of truth, and an ML team that spends more time cleaning data than training models.
More specific indicators to look for:
- Business units pulling their own exports and trusting nobody else’s figures
- No clear answer to “where does this metric come from?”
- Data access is either too open (anyone can modify anything) or too closed (analysts wait weeks for access)
- Compliance requests — such as GDPR audits or NIS2 assessments — require manual investigation to answer basic questions about data location and usage
- New engineers take months to understand the data landscape because there is no documentation
In manufacturing, energy, and agritech environments, there is an additional trigger: when OT data from sensors and production systems starts feeding into analytical platforms, the absence of governance becomes immediately visible. Without clear ownership and quality rules, sensor data arrives inconsistently formatted, with gaps, and with no way to trace anomalies back to their source.
How can data governance work in practice without becoming bureaucracy?
Data governance avoids becoming bureaucracy when it is enforced through tooling rather than process. Instead of asking engineers to follow governance policies manually, the goal is to encode those policies into the data platform itself — quality checks run automatically, lineage is captured as a byproduct of pipeline execution, and catalogues are populated through code rather than manual entry.
Several principles make this work in practice:
- Governance as code. Quality rules, schema contracts, and access policies are defined in version-controlled code alongside the pipelines they govern. This makes governance reviewable, testable, and deployable.
- Lightweight ownership models. Rather than elaborate RACI matrices, assign clear domain owners with a specific responsibility: they are alerted when quality drops and they sign off on schema changes. Simple and enforceable.
- Incremental rollout. Start with the datasets that matter most — the ones feeding executive reports or ML models — and extend governance coverage progressively. Trying to govern everything at once guarantees nothing gets governed well.
- Observability built in. Pipeline monitoring and alerting means issues surface automatically, reducing the reliance on humans to catch problems.
We work this way with our clients — building governance into the platform layer rather than layering it on top as an afterthought. The result is governance that engineers actually use because it is part of their normal workflow, not an additional burden. For teams that need ongoing support maintaining these standards at scale, managed services can provide the operational continuity that keeps governance effective over time.
What tools support data governance — and which ones are worth it?
The tools worth investing in for data governance are those that integrate directly with your data engineering workflow rather than sitting alongside it. The most impactful categories are data catalogues, data quality frameworks, and lineage tracking tools — and the best implementations connect all three.
Data catalogues
Tools like DataHub, OpenMetadata, and Atlan allow teams to document datasets, assign ownership, and make data discoverable across the organisation. The key differentiator is whether the catalogue can be populated automatically from pipeline metadata rather than requiring manual input. If engineers have to manually update a catalogue, it will fall out of date within weeks.
Data quality frameworks
dbt tests, Great Expectations, and Soda are widely used for defining and enforcing quality rules at the transformation layer. These tools allow quality checks to be written as code, run as part of CI/CD pipelines, and integrated with alerting systems so failures surface immediately rather than silently corrupting downstream reports.
Lineage and observability
Tools like Monte Carlo, Marquez, and OpenLineage provide visibility into how data flows through the platform and where transformations occur. For compliance purposes — particularly under frameworks like the EU AI Act — being able to trace a model’s training data back to its source is increasingly non-negotiable.
The honest answer on tool selection: the best tool is the one your team will actually use and that integrates with your existing stack. A sophisticated catalogue nobody populates is worth less than a simple one that is kept current. If you are unsure where to start, talking through your current architecture with engineers who have implemented these tools in production is usually the fastest way to identify what will actually move the needle.
Frequently Asked Questions
How do we get buy-in from engineers who see data governance as extra overhead?
The most effective approach is to show engineers that governance tooling removes friction rather than adding it — automated quality checks catch pipeline issues before they escalate, catalogues reduce the time spent answering ‘where does this data come from?’, and clear ownership means fewer interruptions from downstream teams chasing broken reports. Involve engineers in designing the governance implementation from the start, not just the rollout. When engineers help define the quality rules and choose the tooling, adoption follows naturally because they have built something they trust.
What's a realistic starting point for a team with no existing governance in place?
Start with your most critical datasets — the ones feeding executive dashboards, production ML models, or regulatory reports — and implement three things: automated quality checks at the pipeline level, schema documentation in a catalogue, and a named domain owner for each dataset. Resist the urge to govern everything at once. A well-governed core of ten datasets is far more valuable than a half-implemented framework spread across hundreds. Once the pattern is established and trusted, extending coverage becomes straightforward.
How should data governance handle OT data sources like SCADA systems and IoT sensors specifically?
OT data requires governance rules that address the unique characteristics of operational technology: irregular sampling rates, sensor drift, unit inconsistencies, and the absence of schema enforcement at the source. In practice, this means defining normalisation and validation rules at the OT/IT boundary — the point where raw sensor data enters the data platform — and treating those rules as first-class pipeline code. Ownership should also be shared between OT engineers who understand the physical context and data engineers who manage the platform, with clear handoff points documented in the catalogue.
What's the biggest mistake teams make when rolling out a data catalogue?
The most common mistake is choosing a catalogue and then relying on manual input to populate it. Within weeks, the catalogue is out of date and engineers stop trusting it — which means they stop using it. The fix is to treat catalogue population as an engineering problem: connect it directly to your pipeline metadata, dbt models, or data contracts so that documentation is generated automatically as a byproduct of normal development work. Manual curation should be reserved for business context and ownership assignment, not technical schema descriptions.
How does data governance need to adapt to meet NIS2 or EU AI Act compliance requirements?
Both NIS2 and the EU AI Act require organisations to demonstrate control over their data — where it comes from, how it is used, and who has access to it. For NIS2, this means having audit trails for data access and the ability to quickly identify which systems were affected in the event of an incident. For the EU AI Act, high-risk AI systems must be able to trace training data back to its source, which makes automated lineage tracking non-negotiable rather than a nice-to-have. Building these capabilities into your data platform now, rather than retrofitting them at audit time, is significantly less costly and more reliable.
Can data governance be implemented incrementally, or does it require a big-bang rollout to be effective?
Incremental implementation is not only possible — it is the recommended approach. A big-bang rollout typically produces a governance framework that is too abstract to be useful and too broad to be enforced. Instead, identify the highest-risk or highest-value data domains, implement governance there first, and treat that initial rollout as a template you refine before extending. Each new domain you bring under governance should take less effort than the last, because your tooling, patterns, and team familiarity all compound over time.
How do you measure whether your data governance programme is actually working?
The most meaningful indicators are operational, not procedural: how often do data quality failures reach downstream consumers before being caught, how long does it take a new engineer to locate and understand a dataset, and how quickly can your team respond to a compliance data request? Tracking the rate of pipeline incidents, catalogue coverage across critical datasets, and mean time to resolve data quality issues gives you a clear picture of whether governance is having a real impact. If those numbers are not improving, the governance framework exists on paper but is not embedded in the platform.