Your data is good enough to trust when it is accurate, consistent, complete, and timely enough to support the decision at hand. That threshold is not absolute — it shifts depending on whether you are building a weekly operations report, training a machine learning model, or meeting a compliance requirement like the EU AI Act. The sections below unpack each dimension of data trustworthiness, from the warning signs that your data has a reliability problem to the engineering practices that fix it for good.
What makes data reliable enough to base decisions on?
Data is reliable enough to base decisions on when it is accurate relative to its source, consistent across systems, complete for the scope of the decision, and delivered within a timeframe that makes it actionable. Reliability is not a binary state — it is a spectrum, and the acceptable level depends entirely on the stakes and context of the decision being made.
For a daily production report in a manufacturing environment, freshness and completeness matter most. For a financial forecast, accuracy and consistency across business units are non-negotiable. For a machine learning model predicting equipment failure, lineage and historical integrity become critical. The mistake most data teams make is treating data quality as a single standard when it is actually a set of fitness criteria tied to specific use cases.
The practical starting point is agreeing on what “reliable” means for each domain in your organisation. That means defining quality dimensions explicitly: accuracy, completeness, consistency, timeliness, uniqueness, and validity. Without that shared definition, every business unit ends up with its own interpretation, and trust erodes quickly.
What are the most common signs your data cannot be trusted?
The most common signs that your data cannot be trusted are: different teams reporting different numbers for the same metric, pipelines failing silently overnight, no single agreed source of truth, and business users who have stopped using dashboards because they no longer trust them. These are not technical edge cases — they are symptoms of structural data engineering problems.
In practice, the warning signs tend to cluster into recognisable patterns:
- Contradictory reports: Two business units pull their own exports and arrive at different revenue or production figures for the same period.
- Silent pipeline failures: A pipeline breaks at 2am, no alert fires, and the CEO sees a corrupted report at 9am.
- Undocumented transformations: Nobody can explain why a column is calculated the way it is, or which upstream system it came from.
- Stale data presented as current: A dashboard shows data from three days ago with no indication of its freshness.
- Shadow spreadsheets: Teams maintain their own Excel files because they do not trust the central data platform.
Each of these is a signal that the data engineering layer — pipelines, quality checks, documentation, and alerting — has not kept pace with the organisation’s data consumption. The problem is rarely the data itself. It is the absence of the infrastructure around it.
How do data teams measure data quality objectively?
Data teams measure data quality objectively by defining and tracking quantifiable metrics across six core dimensions: accuracy, completeness, consistency, timeliness, uniqueness, and validity. These metrics are implemented as automated checks that run continuously in the pipeline, not as manual audits performed periodically.
The most effective approach embeds quality checks directly into the data pipeline using tools like dbt tests, Great Expectations, or Soda. Each check produces a pass or fail result that is logged, monitored, and — when thresholds are breached — triggers an alert before bad data reaches downstream consumers.
Quantitative quality metrics worth tracking
Concrete metrics give teams an objective view of data health over time. Useful examples include: the percentage of records with null values in required fields (completeness), the rate of duplicate primary keys (uniqueness), the lag between an event occurring and the data being available (timeliness), and the proportion of values that fall within expected ranges (validity).
Quality checks versus quality thresholds
A check tells you whether a rule passed or failed. A threshold tells you how much failure is acceptable before action is required. Both are necessary. A pipeline that rejects every dataset with a single null value will grind to a halt. A pipeline with no thresholds at all will pass corrupted data downstream without warning. Setting sensible thresholds by domain — and reviewing them regularly as business requirements evolve — is part of mature data engineering practice.
What is data observability and how does it differ from data quality checks?
Data observability is the ability to understand the health, state, and behaviour of data across your entire pipeline at any point in time. It differs from data quality checks in scope: quality checks test whether data meets defined rules at a specific moment, while observability continuously monitors the full pipeline — tracking anomalies, lineage changes, volume shifts, and freshness degradation — even when no explicit rule has been violated.
A useful analogy is the difference between a medical test and a continuous health monitor. A quality check is like a blood test — it tells you whether a specific value is in range on the day you run it. Observability is like a heart rate monitor — it watches for unexpected patterns across time, even ones you did not think to define a rule for.
In practice, observability covers four pillars: freshness (is the data arriving on schedule?), volume (are record counts within expected ranges?), schema (have column names or data types changed unexpectedly?), and lineage (which upstream sources and transformations produced this dataset?). Tools in this space — such as Monte Carlo, Bigeye, or built-in features in modern data platforms — surface anomalies automatically, giving data engineering teams the ability to detect and resolve issues before business users are affected.
For organisations working with OT data sources like SCADA systems, IoT sensors, and historian databases, observability is especially critical. These sources behave differently from SaaS APIs: they can go silent without error codes, produce duplicate timestamps during network interruptions, or shift their schema when firmware is updated on the shop floor. Observability tooling that accounts for these patterns is not optional in industrial data environments — it is foundational.
When is data good enough to use for AI and compliance?
Data is good enough for AI when it is representative of the problem being modelled, free from systematic bias, traceable to its source, and documented well enough to explain model behaviour to a non-technical auditor. For compliance purposes — particularly under frameworks like the EU AI Act and NIS2 — the bar extends further: you must be able to demonstrate data lineage, show that quality controls were in place, and provide evidence that the data used to train or run a model was fit for that purpose at the time.
For machine learning specifically, the most common failure is not that the data is wrong — it is that the data is undocumented. An ML team can build a technically sound model on data whose provenance is unclear, whose transformations are undocumented, and whose quality was never formally assessed. That model will perform well in development and fail in production, or worse, perform well in production until an edge case exposes a fundamental gap in the training data.
The EU AI Act, which came into full effect in 2026 for high-risk AI systems, requires organisations to demonstrate that training data meets specific quality criteria and that those criteria were enforced through documented processes — not just assumed. This means data governance is no longer a best practice. For any organisation deploying AI in regulated contexts, it is a legal requirement. Data lineage tracking, quality check logs, and ownership documentation are the engineering artefacts that make compliance auditable rather than aspirational. If you are navigating these requirements, Itility’s software and data services are built to help organisations put the right foundations in place.
How do you build a data foundation that teams actually trust?
You build a data foundation that teams actually trust by combining three things: reliable pipelines with built-in quality checks and alerting, a governance structure that assigns clear ownership to every dataset, and documentation that lives in the platform rather than in a separate wiki that nobody reads. Trust is not built by buying a new tool — it is built by making the existing data provably dependable over time.
The engineering layer comes first. Pipelines need to be deterministic, testable, and observable. Quality checks should be enforced in code, not described in a policy document. Alerts should fire before business users notice a problem, not after. This is the foundation that prevents the 2am failure from becoming a 9am crisis.
The governance layer comes second, but it only works if it is operationalised. Data ownership should be assigned to specific people and enforced through access controls and quality accountability. A data catalogue is only useful if it is automatically populated from the pipeline — not manually maintained by a team that has other priorities. Governance that lives in a spreadsheet is not governance; it is documentation theatre.
The cultural layer comes third. When teams see that the data platform catches problems proactively, that the numbers are consistent across business units, and that someone is accountable when quality drops, they stop maintaining shadow spreadsheets. That shift from distrust to reliance does not happen because of a communication campaign — it happens because the engineering is reliable enough to earn it.
This is the work we do at Itility every day: designing and building data platforms that are reliable in production, not just on an architecture diagram. If your team is spending more time firefighting pipelines than delivering value, our managed services are designed to take that operational burden off your plate. Get in touch and we can start with a concrete conversation about your current landscape.
Frequently Asked Questions
How do we prioritise which data quality issues to fix first when everything seems broken?
Start by mapping quality issues to business impact: which broken datasets are causing the most expensive decisions to be made on bad information? Fix those first, regardless of how technically complex the fix is. A useful triage framework is to ask three questions for each issue — how many downstream consumers are affected, how frequently does the problem occur, and what is the cost of acting on incorrect data? That ranking will give you a defensible priority order that business stakeholders can also understand and support.
What is the minimum data governance setup a small data team should have in place?
At a minimum, every dataset that feeds a business decision should have a named owner, a documented definition of what it contains and how it is produced, and at least one automated quality check running in the pipeline. These three things — ownership, documentation, and automated validation — are the foundation everything else builds on. A small team does not need a full data catalogue on day one, but it does need these basics codified in code and version control rather than stored in someone’s head or a shared drive.
How do we handle data quality when our sources are third-party systems we cannot control?
When you cannot control the source, you control the ingestion layer. Implement validation checks at the point where third-party data enters your pipeline — before it touches any internal system — and treat every external source as untrusted until it passes those checks. Define a clear contract for what you expect from each source (schema, volume, freshness, value ranges) and alert immediately when that contract is violated. For critical sources, maintaining a buffer of the last known good dataset allows downstream processes to continue operating while the issue is investigated.
What is the difference between a data catalogue and data lineage, and do we need both?
A data catalogue is an inventory of what data assets exist, what they mean, and who owns them — it answers the question ‘what data do we have and what does it represent?’ Data lineage tracks how data moves and transforms across systems — it answers the question ‘where did this number come from and what happened to it along the way?’ For operational reporting, a catalogue alone may be sufficient. For AI, compliance, and debugging complex pipeline failures, lineage is essential and the two work best together: the catalogue tells you what exists, and lineage tells you how to trust it.
How do we convince business stakeholders to invest in data quality infrastructure when the ROI is hard to quantify?
The most effective approach is to quantify the cost of the current state rather than the benefit of the future state. Calculate how many analyst hours per week are spent reconciling conflicting reports, how many decisions were delayed because data was not available in time, or identify one specific instance where a decision was made on incorrect data and estimate its financial impact. These are concrete, business-readable numbers that make the case for investment far more effectively than a technical argument about pipeline architecture. Shadow spreadsheets and manual reconciliation are expensive — they are just invisible costs that nobody has added up yet.
At what point should we consider dedicated data observability tooling versus relying on dbt tests and custom alerts?
Custom checks and dbt tests are the right starting point and remain valuable at any scale for rule-based validation. Dedicated observability tooling becomes worth evaluating when your pipeline has enough complexity that unknown unknowns — anomalies you did not think to write a rule for — are causing incidents. If you are regularly discovering data issues only after a business user reports them, and your existing checks are not catching the problem, that is a strong signal that ML-based anomaly detection and automated lineage tracking would add real value. For industrial environments with OT data sources, that threshold tends to arrive earlier than in purely cloud-native SaaS stacks.
How should we approach data quality differently when preparing data for an AI model versus a standard BI report?
For BI reporting, data quality is primarily about accuracy, freshness, and consistency at the time of reporting — a snapshot that is correct when a user views it. For AI, data quality requirements extend across the entire historical dataset used for training: the data must be representative of real-world conditions, free from systematic bias introduced by collection or transformation processes, and fully traceable so that model behaviour can be explained and audited. Additionally, AI use cases require ongoing quality monitoring of inference-time data, because a model trained on clean historical data can degrade silently if the live data it receives begins to drift from the distribution it was trained on.