Start with the decision, then set the clock

A dashboard can look current because it opened successfully. That does not tell a team whether the information behind it is recent enough to use. Start with a concrete decision, such as assigning available stock or prioritizing a service queue. Ask how old its inputs may be before someone should pause. Set that limit with the people responsible for the decision, including exceptions for weekends, delayed suppliers, and planned maintenance.

Define what the timestamp actually means

Keep the time of the business event, the time data arrived, and the time a report refreshed distinct. Reprocessing an old file can produce a recent pipeline completion time without adding new business information. Our recommendation is to show which timestamp a freshness check measures and which source it covers. A recent record from one location should not silently reassure users about another location whose feed has stopped.

Turn an age limit into an observable check

dbt documents freshness rules with separate warning and error thresholds. Its configuration supports a column or a query for determining the latest loaded timestamp, with metadata-based checks available where supported. The business design still needs an owner and a response for each threshold. Decide who receives the warning, what they investigate, and when downstream automation should wait for human review. Check the documentation for the dbt version and adapter in use before implementing a rule. dbt freshness documentation.

Check the contents as well as the clock

Freshness alone does not establish that the data is usable. Google Cloud documents Dataform assertions for non-null columns, unique keys, and custom row conditions. Manual assertions fail when their query returns rows. Select checks that reflect the meaning of your dataset: a required order identifier, a valid status, or a relationship that must hold. A successful transfer and a recent timestamp should not substitute for these checks. Google Cloud Dataform documentation.

Make a failed check useful to the person deciding

Our suggested interface shows the last verified update, the affected source, and a clear warning when the agreed limit is exceeded. Keep the last verified result identifiable rather than presenting an old value as a new success. Test a missing delivery, a replayed file, a duplicate identifier, and a recovery after corrected input. Confirm that the right person is alerted and that normal processing resumes only after the required checks pass. A useful data pipeline explains when its output can support the next decision.

Sources & further reading