Computing GeoJSON Feature Deltas with geopandas

Part of the Incremental Data Processing guide.

Operative rule: reduce every feature to a single fingerprint — a hash of its canonical WKB geometry plus its sorted attributes — keyed on a stable real-world id, then diff the two snapshots on (id, hash) so added, modified, and removed features fall out in one pass.

How the Delta Computation Works

An incremental pipeline only wants to touch what changed. To know what changed between last night’s dataset and today’s, you compare two snapshots — but a naive comparison of geometry objects is both slow and unreliable. Shapely geometries do not compare cleanly on floating-point coordinates, column order varies between exports, and iterating row by row on tens of thousands of features is a bottleneck. The robust approach is to collapse each feature into two values: a stable id that identifies the real-world thing across snapshots, and a content hash that captures everything about the feature’s current state.

The content hash is computed from the geometry’s canonical Well-Known Binary (WKB) representation combined with the feature’s attributes serialised in a deterministic order. Two features with the same id are identical if and only if their hashes match. This turns the whole diff into a set operation. Using geopandas 0.14+ you build a small frame of (id, hash) for each snapshot, then do a single pandas.merge with indicator=True: ids present only on the right are added, ids present only on the left are removed, and ids present on both but with different hashes are modified. The output is a compact change set you can feed into the next stage — appending features to MBTiles or a bounding-box-scoped re-tile — instead of rebuilding the entire map. Getting the geometry hash right also depends on both snapshots sharing one coordinate system; if they do not, normalise first with sound CRS & projection management.

Production-Ready Implementation

The routine below loads two GeoJSON snapshots as GeoDataFrames, fingerprints each feature, and returns three frames: added, modified, and removed. It normalises geometry to canonical WKB via shapely, sorts attribute keys so column order never affects the hash, and rounds coordinates to a fixed grid so sub-millimetre float noise does not register as a change.

# geojson_delta.py — classify feature changes between two snapshots.
# Requires: geopandas>=0.14, shapely>=2.0, pandas.

from __future__ import annotations

import hashlib
import json
from dataclasses import dataclass
from pathlib import Path

import geopandas as gpd
import pandas as pd
from shapely import set_precision  # shapely 2.x
from shapely.geometry.base import BaseGeometry


@dataclass(frozen=True)
class Delta:
    added: gpd.GeoDataFrame
    modified: gpd.GeoDataFrame
    removed: gpd.GeoDataFrame


def _feature_hash(geom: BaseGeometry, attrs: dict, *, grid: float = 1e-7) -> str:
    """
    One fingerprint per feature: canonical WKB of a precision-reduced geometry
    plus attributes serialised with sorted keys. Same geometry + same attrs
    always yields the same digest.
    """
    snapped = set_precision(geom, grid)               # kill sub-grid float noise
    hasher = hashlib.sha256()
    hasher.update(snapped.wkb)                         # canonical binary geometry
    hasher.update(json.dumps(attrs, sort_keys=True, default=str).encode())
    return hasher.hexdigest()


def _fingerprint(gdf: gpd.GeoDataFrame, id_col: str) -> pd.DataFrame:
    """Reduce a GeoDataFrame to a (id, hash) frame keyed on a stable id."""
    attr_cols = [c for c in gdf.columns if c not in (id_col, gdf.geometry.name)]
    hashes = [
        _feature_hash(row.geometry, {c: getattr(row, c) for c in attr_cols})
        for row in gdf.itertuples(index=False)
    ]
    return pd.DataFrame({id_col: gdf[id_col].to_numpy(), "_hash": hashes})


def compute_delta(
    old: gpd.GeoDataFrame, new: gpd.GeoDataFrame, id_col: str
) -> Delta:
    """Return added / modified / removed feature sets between two snapshots."""
    if old.crs != new.crs:
        raise ValueError(f"CRS mismatch: {old.crs} vs {new.crs}; reproject first")

    fp_old = _fingerprint(old, id_col)
    fp_new = _fingerprint(new, id_col)

    merged = fp_old.merge(
        fp_new, on=id_col, how="outer",
        suffixes=("_old", "_new"), indicator=True,
    )

    added_ids = merged.loc[merged["_merge"] == "right_only", id_col]
    removed_ids = merged.loc[merged["_merge"] == "left_only", id_col]
    both = merged[merged["_merge"] == "both"]
    modified_ids = both.loc[both["_hash_old"] != both["_hash_new"], id_col]

    return Delta(
        added=new[new[id_col].isin(added_ids)].copy(),
        modified=new[new[id_col].isin(modified_ids)].copy(),
        removed=old[old[id_col].isin(removed_ids)].copy(),
    )


if __name__ == "__main__":
    old = gpd.read_file(Path("snapshots/incidents_prev.geojson"))
    new = gpd.read_file(Path("snapshots/incidents_curr.geojson"))

    delta = compute_delta(old, new, id_col="incident_id")
    print(
        f"added={len(delta.added)} "
        f"modified={len(delta.modified)} "
        f"removed={len(delta.removed)}"
    )

Alternative Variants

Vectorised hashing for large frames

itertuples is fine up to a few tens of thousands of features. Beyond that, hash the WKB column directly with a vectorised apply, which avoids per-row Python attribute access:

import geopandas as gpd
import pandas as pd

def fast_fingerprint(gdf: gpd.GeoDataFrame, id_col: str, attr_cols: list[str]) -> pd.DataFrame:
    wkb = gpd.GeoSeries(gdf.geometry).to_wkb()                    # bytes per row
    attrs = gdf[attr_cols].astype(str).agg("|".join, axis=1)     # stable string
    combined = pd.Series(wkb).astype(bytes) + attrs.str.encode("utf-8")
    hashes = combined.map(lambda b: __import__("hashlib").sha256(b).hexdigest())
    return pd.DataFrame({id_col: gdf[id_col].to_numpy(), "_hash": hashes.to_numpy()})

What each change class costs downstream

The unchanged bucket is the whole point: in a typical nightly refresh it is the overwhelming majority, and skipping it is what makes the pipeline incremental rather than a full rebuild.

Verification Steps

• Run the delta on two identical snapshots and confirm all three output frames are empty — any non-empty result means a non-deterministic hash input.
• Modify one attribute of a single feature, rerun, and confirm exactly one feature lands in modified and nothing in added or removed.
• Move one feature’s geometry by more than the precision grid and confirm it appears in modified.
• Confirm added, modified, and removed counts reconcile: len(new) == unchanged + added + modified and len(old) == unchanged + removed + modified.
• Assert both snapshots share a CRS before diffing; the routine raises on mismatch by design.

Common Errors & Fixes

Every feature reports as both added and removed

The id column is not stable across snapshots — you are keying on the row index, an export sequence number, or an auto-increment that resets each build. Switch to a value tied to the real-world feature, such as a database primary key or a domain identifier, so the same feature carries the same id in both snapshots.

Unchanged features show up as modified

Float precision or attribute serialisation differs between exports. Two GeoJSON writers can emit the same point as -74.006 and -74.0060000001. Snap geometries to a fixed precision grid with shapely.set_precision before hashing, and serialise attributes with sort_keys=True and a fixed default so key order and numeric formatting cannot vary.

CRS mismatch error raised on merge

The two snapshots are in different coordinate systems, so their WKB bytes differ even for the same location. Reproject one frame with gdf.to_crs(...) to match the other before computing the delta. Standardising both snapshots to one CRS at ingest avoids the problem entirely.

Modified features detected but their geometry looks identical

An invisible attribute changed — a timestamp column, a computed field, or a source-system audit value that updates on every export even when the feature is unchanged. Exclude volatile bookkeeping columns from the attribute set you hash, keeping only the fields that meaningfully describe the feature.

Related

• Incremental Data Processing — parent guide covering delta detection, partial rebuilds, and change-driven pipelines
• Appending Features to MBTiles Without a Full Rebuild — consumes this delta to patch only the tiles that changed
• Scheduled Map Rebuild Workflows — where the snapshot-and-diff step runs on a recurring schedule
• CRS & Projection Management — normalise both snapshots to one coordinate system before hashing geometry