Atomic Map Artifact Swaps with Versioned Object Paths

Part of the Scheduled Map Rebuild Workflows guide.

Operative rule: exactly one object in the deployment is mutable — the pointer. Everything else is written once, under a versioned path, and never touched again.

Why In-Place Publishing Produces Half-States

Publishing a rebuild means writing many objects: tiles or an archive, a style document, data files, a legend definition. Object storage gives you atomicity per object and nothing beyond that. While the upload runs, readers are served a mixture of the previous build and the current one — and the two are not interchangeable, because a tile pyramid and its style are a matched pair.

The visible symptoms are familiar: a legend that describes classes the tiles no longer use, a seam where new tiles meet old ones at a zoom boundary, features that appear twice. All of them come from the same cause, and a failure halfway through leaves the site in a state that is neither version and cannot be described.

The mixed-content window, and how to remove it Publishing in place produces a long window during which readers receive some new objects and some old ones. Publishing to a versioned path and then switching a pointer removes the window entirely: readers see the previous build until the pointer changes, then the new one. The same publish, with and without a mixed-content window in place — overwrite the live paths old MIXED — minutes of seams, ghosts and legend mismatch new a failure inside the window leaves a state that is neither build versioned — write to a new path, then move the pointer old build served throughout the upload new build the switch is one small write — there is no window to be caught in

Production-Ready Implementation

from __future__ import annotations

import json
from dataclasses import dataclass
from pathlib import Path


@dataclass(frozen=True)
class Pointer:
    """The only mutable object in the deployment."""

    build_id: str
    tiles: str
    style: str
    source_timestamp: str
    run_id: str

    def to_json(self) -> str:
        return json.dumps(self.__dict__, indent=2, sort_keys=True)


def publish_build(storage, local_dir: Path, build_id: str) -> str:
    """Upload every artifact under an immutable prefix. Slow, safe, invisible."""
    prefix = f"builds/{build_id}/"
    for file in sorted(local_dir.rglob("*")):
        if not file.is_file():
            continue
        storage.put(
            key=prefix + str(file.relative_to(local_dir)),
            body=file.read_bytes(),
            cache_control="public, max-age=31536000, immutable",
        )
    return prefix


def switch_pointer(storage, pointer: Pointer) -> None:
    """One small write. This is the moment readers change build."""
    storage.put(
        key="current.json",
        body=pointer.to_json().encode("utf-8"),
        cache_control="public, max-age=0, must-revalidate",
        content_type="application/json",
    )


def rollback(storage, previous_build_id: str) -> None:
    """Rollback is re-publishing an older pointer — no rebuild involved."""
    previous = json.loads(storage.get(f"builds/{previous_build_id}/pointer.json"))
    switch_pointer(storage, Pointer(**previous))

Writing a copy of the pointer inside each build directory is the small extra step that makes rollback trivial. The build knows what its own pointer should look like; storing it means rolling back never requires reconstructing that information from a log.

What the Client Must Do

How the page consumes the pointer The page fetches the pointer with revalidation, reads the build paths from it, and loads the tiles, style and data from those paths. Because every path comes from one pointer read, a single session never mixes builds even if a new one is published mid-session. One pointer read per session pins the whole session to one build fetch current.json revalidated, never stale read the build paths tiles, style, data, vintage load from those paths immutable, cached forever Consequences worth having ◦ a build published mid-session cannot mix into a page already loaded ◦ every artifact is immutable, so nothing ever needs purging ◦ the pointer also carries the data vintage, which the freshness badge reads

Retention and Cleanup

What to keep and what to expire The most recent five to ten builds are kept so rollback is always possible. Any build referenced by a saved view or a published report is kept indefinitely. Everything older is expired by a lifecycle rule, because storage grows linearly with build frequency. Storage grows with every build — decide the policy before it matters last 5–10 builds kept — this is what makes rollback a pointer write referenced builds kept indefinitely — a report or saved view points at them everything older expired by a lifecycle rule on the builds prefix

Rolling Back Under Pressure

The value of this arrangement is realised on the day something goes wrong, and that day is a poor time to discover the procedure has never been run. Rolling back should be a single documented command that reads a previous build’s stored pointer and republishes it — no rebuild, no manual editing of paths, no reasoning about which artifacts belong together.

Practise it. A rollback rehearsed once in normal working hours takes two minutes and confirms three things at once: that old builds are retained, that their pointers were stored, and that the person on call knows where the command lives. The same rehearsal in an incident takes twenty minutes and produces the questions that make an outage longer than it needed to be.

It is also worth deciding in advance what a rollback does not fix. Reverting the pointer restores the previous data and the previous style, but it does not undo a schema change in a downstream consumer, and it does not recall a notification that has already been sent. Naming those boundaries in the runbook keeps the rollback from being attempted as a general-purpose remedy for problems it cannot address.

Verification Steps

  • Publish a build without switching the pointer and confirm readers still see the previous one.
  • Switch the pointer and confirm a reload picks up the new build within one revalidation.
  • Roll back and confirm the map returns to the previous data with no rebuild.
  • Confirm every artifact URL under a build prefix carries an immutable cache lifetime.
  • Confirm the lifecycle rule does not touch builds referenced by saved views.

Common Errors & Fixes

Readers still see the old build long after the switch

The pointer is being cached. It is the one object that must revalidate on every load.

Rollback needs a rebuild

Old builds were deleted, or the pointer contents were never stored alongside them. Keep both.

Storage costs grow steadily

No lifecycle rule on the builds prefix. Add one, and exclude the referenced prefix explicitly.

A partially uploaded build was switched to

The switch ran before the upload completed. Make the pointer write the last statement after an upload verification step, in the same order the rebuild transaction prescribes.

Choosing the Build Identifier

The build id appears in every path, every log line and every rollback conversation, so it is worth choosing deliberately rather than reaching for a random string.

Three properties matter. It should sort chronologically, so a directory listing reads as a history rather than as a jumble. It should be unique even when two runs start in the same second, which a timestamp alone is not. And it should be traceable back to what produced it, so that an artifact found six months later can be connected to a pipeline version.

A compound identifier satisfies all three: a UTC timestamp for ordering, a short random suffix for uniqueness, and the pipeline’s commit hash recorded inside the build rather than in the name. That keeps paths readable while preserving the link to the code — and it avoids the trap of putting the commit in the path, which produces two identical-looking builds when the same commit runs twice against different data.

It is also worth deciding what the id means when a run is retried. A retry that rebuilds from the same source should generally get a new id, because its output may differ; a retry that merely re-uploads an already-built artifact should reuse the original, because the bytes are the same. Conflating the two produces a directory full of near-duplicates that nobody can tell apart later.

What the Pointer Should Carry

Beyond naming the current build, the pointer is the natural place for the small facts every reader and every operator needs. The data’s source timestamp goes here, because the freshness badge reads it. The run id goes here, so any artifact can be traced back to the record that describes how it was produced. And a short schema version goes here, so that a page from an older deploy can detect a pointer it does not understand and fail visibly rather than misreading it.

Keep it small and keep it flat. The pointer is fetched on every page load with revalidation, so every field costs something on every visit, and a nested structure invites the page to depend on parts of it that were never meant to be an interface.

Gotchas & Edge Cases

  • Object storage is eventually consistent for listings on some providers even when it is strongly consistent for reads; verify the build by fetching known keys rather than by listing the prefix.
  • A pointer written before the last artifact finishes uploading produces exactly the mixed state this pattern exists to prevent — verify, then switch.
  • Readers who loaded the page before a switch keep the old build for the life of their session, which is correct; a change that must reach them immediately needs a page-level reload prompt, not a faster pointer.
  • Deleting an old build that a saved view still references breaks that link silently. Cleanup must consult the references, not only the age.
  • Do not reuse a build id after a failed run. The partial artifacts may still be present, and a second run under the same id produces a directory that is a mixture of two attempts.
  • Keep the pointer small enough to be fetched on every page load without thought; if it is growing, that is a sign something belongs in the build directory instead.