Data Sketch
Back to gallery

Episode 06 / Data Quality

A Simple Reconciliation Workflow

A count mismatch is a starting point for investigation. Compare the date range, filters, grain, keys, pipeline stage, and late updates to understand why source and target results differ.

By Mario Caesar

Reconciliation funnel checking count mismatch, date range, filters, grain, keys, pipeline stage, and late updates

Reading the sketch

  1. Confirm the mismatch

    Identify which source and target counts differ before changing the pipeline.

  2. Align the date range

    Compare the same date range or partition window on both sides.

  3. Compare the filters

    Check that the WHERE conditions select comparable records.

  4. Check the grain

    Confirm what each row represents. Counts at different grains are not directly comparable.

  5. Inspect the keys

    Look for missing keys and duplicate rows that could explain the difference.

  6. Locate the pipeline stage

    Determine which processing stage each dataset represents before comparing the results.

  7. Check late updates

    Review delayed records and source changes that may not have reached the target yet.

View the original LinkedIn post (opens in a new tab)