Data Sketch
Back to gallery

Episode 09 / Warehouse

Partitioning: Why Scanning Less Data Matters

Scanning more data than a query needs can increase runtime and cost. Partitioning can reduce that work when the query can skip irrelevant partitions; it does not make every query faster.

Partitioning diagram showing how scanning less data can reduce query work and improve performance and cost

Reading the sketch

  1. Review how much data is scanned

    Check whether the query reads data beyond what it needs for the result.

  2. Organize data into partitions

    Partitioning separates data into groups that the query engine may be able to read independently.

  3. Skip irrelevant partitions

    The benefit depends on whether the query can avoid reading partitions it does not need. A query that scans most partitions may gain little.

  4. Check the actual result

    Compare the scanning work and runtime after the change. Cost savings depend on the platform and how it charges for the workload.

View the original LinkedIn post (opens in a new tab)