Coastal & Marine Spatial Analysis Pipelines

Building Multi-Arch Geospatial Images

Cloud arm64 instances are cheaper per core than amd64 and a bathymetric gridding job is embarrassingly parallel, so the arithmetic is attractive. The obstacle is that a geospatial image is not portable the way a pure-Python one is: the wheels differ, the library versions available differ, and the numerical results can differ in the last bits. This page builds both architectures and proves they agree, within the containerization workflow.

Root cause: the stack below Python is not the same stack

A geospatial container is a thin Python layer over GDAL, PROJ, GEOS and their dependencies, all compiled. Three things vary by architecture:

Wheel availability. rasterio and fiona ship manylinux wheels bundling their own GDAL. Coverage for aarch64 is good now and lags for new releases, and for less common packages — pdal, laspy extras, some netCDF stacks — it can be absent entirely. A missing wheel means pip compiles from source, which needs a toolchain in the image and produces a build that links against whatever system GDAL is present rather than the bundled one.

Distribution package versions. apt on arm64 sometimes carries a different GDAL point release than on amd64 for the same distribution snapshot. A pipeline pinned to libgdal-dev rather than to a version gets two different libraries from one Dockerfile.

Floating point. Both architectures are IEEE 754, so basic arithmetic agrees. Transcendental functions do not: libm implementations differ, and a PROJ transformation involving trigonometric functions can differ in the last one or two bits between architectures. That is far below survey accuracy and it is not zero, which matters for any test asserting exact equality.

docker run --rm --platform linux/amd64 my-image python -c \
  "from pyproj import Transformer; print(repr(Transformer.from_crs('EPSG:4326','EPSG:32618',always_xy=True).transform(-75.0941,38.7870)))"
# (492013.8624549816, 4293201.155595882)

docker run --rm --platform linux/arm64 my-image python -c "$(: same)"
# (492013.8624549817, 4293201.155595882)

One unit in the last place, on the easting. Irrelevant to a survey and fatal to a test written with ==.

Only the Top Layer Is Actually Portable A three-layer stack drawn twice, once for amd64 and once for arm64, with the differences marked. The application layer of pure Python source is identical between them. The wheel layer differs: a package may have a manylinux wheel on one architecture and not the other, and where both exist they may bundle different GDAL point releases. The system layer differs in the distribution's available library versions and in the libm implementation, whose transcendental functions produce results differing in the last bits. A note records that the top layer is the only one a developer looks at and the bottom one is where the differences that matter live. linux/amd64 application Python — identical wheels: rasterio 1.4.3 (GDAL 3.9.2) manylinux_x86_64 libgdal 3.9.2 · libm (glibc x86) PROJ 9.4.1 linux/arm64 application Python — identical wheels: may be absent → source build manylinux_aarch64 libgdal 3.9.1 · libm (glibc aarch64) PROJ 9.4.1 — same, this time The layer a developer reads is the only one that is genuinely the same. Everything that can differ is below it. Which is why the check is a runtime assertion inside each image rather than a review of the Dockerfile. The dashed boxes are what a multi-arch build has to verify rather than assume.

Step-by-step build

1. Buildx with native builders

Emulated builds work and are slow: compiling GDAL bindings under QEMU is roughly an order of magnitude slower than native, which turns a five-minute build into an hour and makes CI unusable.

# One builder per architecture, on native hardware.
docker buildx create --name marine --node marine-amd64 \
  --platform linux/amd64 --driver docker-container
docker buildx create --name marine --append --node marine-arm64 \
  --platform linux/arm64 --driver docker-container \
  ssh://build@arm-runner.internal

docker buildx build --builder marine \
  --platform linux/amd64,linux/arm64 \
  --tag ghcr.io/org/marine-pipeline:2026.03 \
  --provenance=true --sbom=true \
  --push .

--provenance=true is worth the flag. It records which builder produced each architecture’s image, which is the first question asked when the two behave differently.

2. Assert the stack inside the image, per architecture

The Dockerfile check that catches drift has to run in the built image, not in the build script:

"""selfcheck.py — run at image build time; the build fails if the stack drifts."""
import platform
import sys

EXPECTED = {"gdal": "3.9.2", "proj": "9.4.1", "geos": "3.12.1"}
TOLERANCE_M = 1e-6


def main() -> int:
    from osgeo import gdal
    import pyproj
    import shapely

    actual = {
        "gdal": gdal.__version__.split("e")[0],
        "proj": pyproj.proj_version_str,
        "geos": shapely.geos_version_string.split("-")[0],
    }
    drift = {k: (v, actual[k]) for k, v in EXPECTED.items() if not actual[k].startswith(v)}
    if drift:
        print(f"FAIL {platform.machine()}: version drift {drift}", file=sys.stderr)
        return 1

    # A reference transform, compared against a value fixed in the repository.
    tf = pyproj.Transformer.from_crs("EPSG:4326", "EPSG:32618", always_xy=True)
    e, n = tf.transform(-75.0941, 38.7870)
    if abs(e - 492013.862455) > TOLERANCE_M or abs(n - 4293201.155596) > TOLERANCE_M:
        print(f"FAIL {platform.machine()}: reference transform {e:.6f}, {n:.6f}", file=sys.stderr)
        return 1

    print(f"ok {platform.machine()}: {actual}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

A micrometre tolerance rather than exact equality is the deliberate choice. It is far tighter than any survey requirement and loose enough to absorb the last-bit libm differences, so the check fails on a genuine version drift and not on the architecture itself.

3. Record both digests, not the tag

A multi-arch tag is a manifest list pointing at two images with two digests. Recording the tag records neither:

docker buildx imagetools inspect ghcr.io/org/marine-pipeline:2026.03 --format \
  '{{range .Manifest.Manifests}}{{.Platform.Architecture}} {{.Digest}}{{"\n"}}{{end}}'
# amd64 sha256:a41c...
# arm64 sha256:7fe0...

Both belong in the run manifest, alongside the architecture the job actually ran on. Without them, “we ran image 2026.03” does not identify the bytes that executed, which is the whole point of pinning by digest as pinning GDAL and PROJ versions sets out.

One Tag, Two Images, Two Digests A tag named 2026.03 points at a manifest list. The list contains two entries: an amd64 image with digest beginning a41c and an arm64 image with digest beginning 7fe0. Two job runs are shown below, one scheduled onto an amd64 node and one onto an arm64 node; each pulls a different image from the same tag. A run manifest beside each records the digest that actually executed together with the architecture. A note states that recording only the tag identifies neither image, and that on a mixed-architecture cluster the same job name can run different bytes on different days depending on where the scheduler placed it. tag :2026.03 manifest list amd64 → a41c… arm64 → 7fe0… job on amd64 node job on arm64 node manifest: a41c… manifest: 7fe0… Recording the tag identifies neither image. On a mixed cluster the same job name runs different bytes depending on where the scheduler placed it that day. Which is fine — the two are verified equivalent — and only verifiable if both digests were written down. The architecture belongs in the manifest too: it is the first thing to check when results differ.

Verification and acceptance test 

import subprocess

import pytest

IMAGE = "ghcr.io/org/marine-pipeline:2026.03"
REFERENCE = (492013.862455, 4293201.155596)


@pytest.mark.parametrize("arch", ["linux/amd64", "linux/arm64"])
def test_reference_transform_agrees_across_architectures(arch):
    out = subprocess.run(
        ["docker", "run", "--rm", "--platform", arch, IMAGE, "python", "/selfcheck.py"],
        capture_output=True, text=True, check=False)
    assert out.returncode == 0, f"{arch}: {out.stderr.strip()}"


@pytest.mark.parametrize("arch", ["linux/amd64", "linux/arm64"])
def test_proj_grids_are_present(arch):
    out = subprocess.run(
        ["docker", "run", "--rm", "--platform", arch, IMAGE,
         "sh", "-c", "ls /usr/share/proj/*.tif | wc -l"],
        capture_output=True, text=True, check=True)
    assert int(out.stdout.strip()) > 0, f"{arch}: no PROJ transformation grids in the image"

The grid check is separate from the version check on purpose: a rebuild that changes the base image can drop the grids while every library version stays identical, and that failure changes coordinates by about a metre with no other symptom.

Is the second architecture worth building at all?

Multi-arch is not free: two builders, two sets of wheels to verify, two digests to track, and a CI matrix that doubles. The decision deserves the same arithmetic as any other, and for a marine pipeline it usually turns on three numbers.

The price difference. Typically twenty to forty per cent per vCPU-hour in favour of arm64 on the major clouds, before any spot discount. For a pipeline whose compute bill is dominated by a nightly gridding job, that is the headline.

Per-core performance on this workload. Geospatial work is memory-bandwidth-bound as often as it is compute-bound, and the two architectures differ in cache and memory behaviour. The honest figure is measured on the actual pipeline, not on a benchmark: a job that is thirty per cent cheaper per core and twenty per cent slower per core has saved twelve per cent, not thirty.

The verification cost, amortised. The self-check above is written once. The CI matrix runs on every build. For a pipeline deployed weekly the overhead is negligible; for one deployed a few times a year it can exceed the saving.

def arm_saving(price_ratio: float, throughput_ratio: float,
               ci_minutes_extra: float, builds_per_month: int,
               compute_hours_per_month: float, ci_cost_per_min: float,
               compute_cost_per_hour: float) -> dict[str, float]:
    """Net monthly saving from adding an arm64 target. Negative means don't."""
    compute_saving = compute_hours_per_month * compute_cost_per_hour * (
        1.0 - price_ratio / throughput_ratio)
    ci_cost = builds_per_month * ci_minutes_extra * ci_cost_per_min
    return {"compute_saving": compute_saving, "ci_cost": ci_cost,
            "net": compute_saving - ci_cost}

Dividing the price ratio by the throughput ratio is the step people skip. A twenty-five per cent cheaper core that delivers ninety per cent of the throughput saves about seventeen per cent, not twenty-five, and on a workload where the gap is larger the saving can vanish entirely.

Price per Core Is Half the Calculation A chart of net saving as a percentage against arm64 per-core throughput expressed as a fraction of amd64. At equal throughput the saving equals the full price advantage of twenty-five per cent. As relative throughput falls the saving declines roughly linearly, reaching zero at seventy-five per cent throughput where the cheaper core is exactly offset by needing more of them, and going negative below that. Two measured workloads are marked: a gridding job at ninety-four per cent relative throughput saving about twenty per cent, and a PROJ-heavy reprojection job at seventy-eight per cent saving only about four per cent. A note says the second workload is not worth a second build target. arm64 per-core throughput, relative to amd64 net saving 0% gridding: 94% → saves 20% PROJ-heavy: 78% → saves 4% 0.700.780.881.00 Measure the throughput on the real pipeline. The price list is the easy half and the misleading one.

Edge cases and gotchas

  • --platform on docker run uses emulation silently. Running an arm64 image on an amd64 host works through QEMU and is roughly ten times slower. A CI job that “tests both architectures” on one runner is testing emulation, which shares the libm of neither.
  • The same wheel tag can bundle different GDAL builds. A manylinux wheel bundles its own GDAL, and the bundled version can differ between architectures for the same package version. The runtime assertion catches it; reading requirements.txt does not.
  • Mixed-architecture node pools need a nodeSelector or a tolerance for both. A Kubernetes Job that assumes one architecture will schedule onto the other and either fail to pull or run the image you did not test. Constrain the pool, or verify both, and prefer the second — the cost advantage of arm64 disappears if half the fleet cannot use it.

Up: Containerizing Marine Geospatial Pipelines with Docker