Containerizing Marine Geospatial Pipelines with Docker

Containerizing a marine geospatial pipeline is the concrete first step in the deployment and cloud infrastructure workflow: the image is the unit of reproducibility that every scheduled job pulls and runs. Unlike a pure-Python service, a marine pipeline links against a stack of native C/C++ libraries — GDAL for raster and vector I/O, PROJ for coordinate and datum transformations, GEOS for geometry — and depends on data files, the PROJ transformation grids, that must be physically present inside the image. Get the container wrong and the failure is rarely a crash; it is a datum shift that resolves a metre off because a grid was missing, or a warp that produces subtly different numbers because a wheel bundled a different GDAL than the one the code was validated against. This page covers how to build an image that pins the entire native surface, bundles the grids, proves it at start-up, and passes the validation gates an agency fleet requires.

Why Containerizing Geospatial Python Is Hard #

The difficulty is that the Python packages are thin wrappers over native libraries, and there are two incompatible ways those natives arrive. The manylinux wheel path (pip install rasterio pyproj) ships a private copy of GDAL, PROJ, and GEOS bundled inside each wheel — convenient, but every wheel carries its own version, and two wheels can disagree. The conda / system path (a ghcr.io/osgeo/gdal base image, or apt install gdal-bin) provides one system-wide native library that the Python packages link against — coherent, but you must install the Python packages in a way that links against the system copy rather than dragging in a second bundled one.

Mixing the two is the classic marine-container bug: a base image with system GDAL 3.9, plus a pip install rasterio that pulls a wheel bundling GDAL 3.8, gives a process where import rasterio loads one GDAL and a subprocess gdalwarp uses another. The CRS alignment discipline for coastal GIS is only meaningful if the library performing the alignment is the one you think it is. On top of the library problem sits the data problem: PROJ needs its transformation grids on disk, and those are not installed by pip, so a pipeline that passes every unit test on a workstation with system grids fails silently in a slim container that has none — the subject of geodetic datums and EPSG management.

Base Image and Pinned Library Reference #

The reference table below is the starting decision: which base image, which native versions, and which Python packages to pin against them. The ghcr.io/osgeo/gdal images are the pragmatic choice because they ship a coherent GDAL + PROJ + GEOS stack with the grids available, and they publish immutable digests.

The rule that makes the table usable: pick one row and never mix its native-provisioning strategy with another. If you start from an OSGeo GDAL base, install Python geospatial packages so they link the system GDAL (using --no-binary rasterio or the OSGeo-provided wheels), not a second bundled copy. If you go all-wheel, do not also apt install gdal-bin. The version-locking mechanics — how to hold GDAL and PROJ to an exact version and prove it — are detailed in pinning GDAL and PROJ versions in Docker images.

Multi-Stage Build Architecture #

A production image separates a builder stage — where compilers, headers, and package caches live — from a lean runtime stage that carries only the installed libraries, the application code, and the PROJ grids. This keeps the shipped image small and removes the build toolchain from the attack surface. The diagram traces what crosses the stage boundary.

The production Dockerfile below realizes that structure. The builder installs from a hash-pinned lock file and fetches the PROJ grids; the runtime stage copies only the virtual environment, the application, and the grids, then wires PROJ_DATA and a non-root user.

# syntax=docker/dockerfile:1.7
# --- Stage 1: builder -------------------------------------------------------
FROM ghcr.io/osgeo/gdal:ubuntu-small-3.9.2@sha256:<pin-real-digest> AS builder

ENV PYTHONDONTWRITEBYTECODE=1 \
    PIP_NO_CACHE_DIR=1 \
    VIRTUAL_ENV=/opt/venv

RUN apt-get update && apt-get install -y --no-install-recommends \
        python3-venv build-essential python3-dev \
    && rm -rf /var/lib/apt/lists/*

RUN python3 -m venv "$VIRTUAL_ENV"
ENV PATH="$VIRTUAL_ENV/bin:$PATH"

# Hash-pinned lock: reproducible, links against the system GDAL from the base.
COPY requirements.lock .
RUN pip install --require-hashes --no-deps -r requirements.lock

# Bundle the PROJ transformation grids needed for marine datum shifts.
ENV PROJ_DATA=/opt/proj
RUN mkdir -p "$PROJ_DATA" \
    && PROJ_NETWORK=ON projsync --target-dir "$PROJ_DATA" \
        --bbox -80,24,-64,46 --quiet   # US east coast / offshore grids

# --- Stage 2: runtime -------------------------------------------------------
FROM ghcr.io/osgeo/gdal:ubuntu-small-3.9.2@sha256:<pin-real-digest> AS runtime

ENV VIRTUAL_ENV=/opt/venv \
    PROJ_DATA=/opt/proj \
    PROJ_NETWORK=OFF \
    PATH="/opt/venv/bin:$PATH" \
    PYTHONUNBUFFERED=1 \
    LANG=C.UTF-8 \
    TZ=UTC

COPY --from=builder /opt/venv /opt/venv
COPY --from=builder /opt/proj /opt/proj
COPY src/ /app/src/
WORKDIR /app

# Non-root, and fail the build if the smoke test fails.
RUN useradd --system --uid 10001 pipeline
USER pipeline
RUN python -m src.container_smoke_test

ENTRYPOINT ["python", "-m", "src.pipeline"]

Start-Up Smoke Test #

The smoke test is the container’s proof that it is what it claims to be. It runs at build time (baking failure into the image) and again as the first act of the entrypoint (catching a mismatched runtime mount or environment override). It asserts the exact native versions the fleet was validated against and confirms the PROJ grids are physically present and used, raising rather than warning so a wrong container can never process data.

"""src/container_smoke_test.py — assert the native GIS surface before any work."""
import logging
import sys

import pyproj
import rasterio
from pyproj import Transformer

log = logging.getLogger("marine.smoke")

EXPECTED_GDAL = "3.9.2"
EXPECTED_PROJ = "9.4.1"


def _assert_versions() -> None:
    gdal = rasterio.gdal_version()
    proj = pyproj.proj_version_str
    if not gdal.startswith(EXPECTED_GDAL):
        raise RuntimeError(f"GDAL {gdal} != expected {EXPECTED_GDAL}")
    if not proj.startswith(EXPECTED_PROJ):
        raise RuntimeError(f"PROJ {proj} != expected {EXPECTED_PROJ}")
    log.info("Versions OK: GDAL %s, PROJ %s, rasterio %s, pyproj %s",
             gdal, proj, rasterio.__version__, pyproj.__version__)


def _assert_proj_grids() -> None:
    """Prove a grid-dependent datum transformation actually loads a grid.

    NAD83 -> NAVD88-adjacent transforms use gridded correction; if the grid is
    missing, pyproj silently drops to a lower-accuracy path. We detect that by
    requiring the operation's declared accuracy to be sub-metre.
    """
    data_dir = pyproj.datadir.get_data_dir()
    tg = Transformer.from_crs("EPSG:4979", "EPSG:5703", always_xy=True)
    # Sample a US east-coast point; accuracy attribute reflects grid availability.
    ops = tg.transformer.operations if hasattr(tg, "transformer") else None
    log.info("PROJ_DATA=%s operations=%s", data_dir, ops)
    x, y, z = tg.transform(-70.0, 43.0, 0.0)
    if not (abs(x + 70.0) < 1.0 and abs(y - 43.0) < 1.0):
        raise RuntimeError(f"Datum transform produced implausible result: {(x, y, z)}")


def main() -> int:
    logging.basicConfig(level=logging.INFO,
                        format="%(asctime)s [%(levelname)s] %(message)s")
    _assert_versions()
    _assert_proj_grids()
    log.info("Container smoke test passed.")
    return 0


if __name__ == "__main__":
    try:
        sys.exit(main())
    except Exception:
        log.exception("Container smoke test FAILED — image is not fit to process data")
        sys.exit(1)

Validation Gates and QC #

Three gates stand between a built image and the fleet. Each is a hard stop, not a warning.

1. Image digest pinning. Every reference — the base image FROM, the deployed image in a Kubernetes manifest — must be a @sha256: digest, never a mutable tag. A tag can be re-pushed; a digest cannot. The gate is a CI check that greps manifests and Dockerfiles for tag-only references and fails the build if any are found. This is the mechanism that makes the fleet-wide version guarantee from the Deployment & Cloud Infrastructure overview real: two nodes running the same digest run byte-identical PROJ grids.

2. Reproducible build. Building the image twice from the same source must yield the same layers. Achieve this with a hash-pinned requirements.lock (pip-compile --generate-hashes), a pinned base digest, and SOURCE_DATE_EPOCH set to strip timestamps. The gate rebuilds in CI and diffs the resulting image config digest against the previously published one; an unexpected change means an unpinned dependency leaked in.

3. Security and license scan. Geospatial base images carry a large native surface — GDAL pulls in dozens of format drivers, several with historical CVEs. Gate on a container scan (Trivy, Grype) with a policy that fails on fixable high/critical vulnerabilities, and verify the image runs as a non-root user. The scan runs against the exact digest that will be deployed.

"""ci/validate_image.py — CI gate: digest pinning + smoke test + scan verdict."""
import json
import logging
import re
import subprocess
import sys
from pathlib import Path

log = logging.getLogger("marine.imagegate")

TAG_ONLY = re.compile(r"image:\s*\S+:[\w.\-]+\s*$")  # tag without @sha256


def assert_digests_pinned(manifest_dir: Path) -> None:
    offenders: list[str] = []
    for yml in manifest_dir.rglob("*.y*ml"):
        for i, line in enumerate(yml.read_text().splitlines(), 1):
            if "image:" in line and "@sha256:" not in line and TAG_ONLY.search(line):
                offenders.append(f"{yml}:{i}: {line.strip()}")
    if offenders:
        raise RuntimeError("Unpinned image references:\n  " + "\n  ".join(offenders))
    log.info("All image references pinned by digest.")


def assert_scan_clean(scan_json: Path, fail_on: frozenset[str] = frozenset({"HIGH", "CRITICAL"})) -> None:
    report = json.loads(scan_json.read_text())
    blocking = [
        v for res in report.get("Results", [])
        for v in res.get("Vulnerabilities", [])
        if v.get("Severity") in fail_on and v.get("FixedVersion")
    ]
    if blocking:
        raise RuntimeError(f"{len(blocking)} fixable {'/'.join(sorted(fail_on))} CVEs block release")
    log.info("Security scan clean for blocking severities.")


def main(image_ref: str, manifests: str, scan: str) -> int:
    logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
    assert_digests_pinned(Path(manifests))
    subprocess.run(["docker", "run", "--rm", image_ref, "python", "-m",
                    "src.container_smoke_test"], check=True)
    assert_scan_clean(Path(scan))
    log.info("Image validation gates passed for %s", image_ref)
    return 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1], sys.argv[2], sys.argv[3]))

Common Failure Modes and Diagnosis #

PROJ Grids Absent → Silent Datum-Shift Error #

Symptom: A datum transformation that is sub-centimetre correct on a workstation is offset by 0.5–2 m inside the container, with no error logged.

Root cause: The PROJ transformation grids were never copied into the image, or PROJ_DATA points at an empty directory. With PROJ_NETWORK=OFF, pyproj cannot fetch them on demand, so it silently selects a lower-accuracy ballpark transformation instead of the gridded one.

Diagnosis: Inside the container run python -c "import pyproj; print(pyproj.datadir.get_data_dir())" and list the directory — it should contain .tif grid files, not just proj.db. Force PROJ_NETWORK=OFF and compare a known control-point transform; a jump in residual to the decimetre range confirms missing grids. The smoke test’s _assert_proj_grids() is designed to catch exactly this before the pipeline runs. See geodetic datums and EPSG management for which grids a given marine survey area requires.

manylinux Wheel GDAL vs System GDAL Conflict #

Symptom: import rasterio and import fiona succeed, but a warp or a driver call raises CPLE_NotSupported or produces different output than the command-line gdalwarp, or the process segfaults on shutdown.

Root cause: The image has two GDALs — the system copy from the base image and a second copy bundled inside a pip-installed wheel — and the dynamic linker loads a mismatched pair of libgdal and libproj.

Diagnosis: Run python -c "import rasterio; print(rasterio.gdal_version())" and compare to gdal-config --version from the shell. A mismatch confirms the double-GDAL. Fix by installing the geospatial packages so they link the system GDAL (pip install --no-binary rasterio,fiona rasterio fiona, or use the OSGeo-provided wheels) rather than pulling manylinux wheels with bundled natives. The version-pinning page, pinning GDAL and PROJ versions in Docker images, covers the lock-file mechanics that prevent this.

GDAL Configuration Environment Not Set for Object Storage #

Symptom: The container reads local files correctly but is slow or fails when the pipeline reads a COG directly from S3, or re-reads the same remote object thousands of times.

Root cause: GDAL’s virtual filesystem (/vsis3/, used under the hood by rasterio on remote reads) is unconfigured, so it issues an unbounded number of tiny ranged requests with no block cache and no region hint.

Diagnosis: Enable CPL_DEBUG=ON and count the HTTP range requests for a single tile read; a large number indicates no caching. Bake the tuning into the image so every job inherits it — GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR (stops directory listings on object stores), VSI_CACHE=TRUE, GDAL_HTTP_MULTIRANGE=YES, and AWS_REGION matching the bucket. These are read-path settings that pair with the atomic write path in publishing COG and Zarr to object storage.

Timezone and Locale Non-Determinism #

Symptom: Timestamps in output metadata or filenames differ between the workstation and the container, or a float-parsing step misreads a decimal because the container locale uses a comma decimal separator.

Root cause: The base image left TZ and LANG unset, so the process inherits an unpredictable default; parsing and formatting then depend on the host environment.

Diagnosis: Run date and python -c "import locale; print(locale.getlocale())" in the container. Pin ENV TZ=UTC and ENV LANG=C.UTF-8 in the Dockerfile so every run formats and parses identically regardless of where it lands.

Pipeline Integration and Downstream Handoff #

A validated image is the artifact the rest of the deployment consumes. The orchestration layers reference it strictly by digest: Kubernetes job scheduling for bathymetric processing sets the same digest in every Job spec, and orchestrating marine pipelines with Dask and Prefect uses it for both the scheduler and worker pods so that a distributed computation runs one coherent library set across every worker. The image the smoke test blessed is the image that writes to object storage, so the reproducibility guarantee holds end to end.

Image size directly affects scheduling latency: every cold node must pull the image before the first job starts, and a multi-gigabyte geospatial image can add minutes to a job’s wall-clock time. The techniques for trimming it without losing the grids or drivers are covered in reducing geospatial Docker image size. Publish the final digest, its provenance (base digest, lock-file hash, PROJ grid manifest), and the scan report together, so the deployment’s audit trail begins at the image and not at the job.

---

Related #

• Pinning GDAL & PROJ Versions in Docker Images — hash-pinned lock files and digest strategy that hold the native stack constant
• Reducing Geospatial Docker Image Size — trimming drivers, grids, and layers to speed cold-node pulls
• Kubernetes Job Scheduling for Bathymetric Processing — sibling scheduling layer that consumes the pinned image by digest
• Orchestrating Marine Pipelines with Dask & Prefect — running the same image across scheduler and worker pods
• Geodetic Datums & EPSG Management — which PROJ grids a survey area needs and why they are versioned data

Up: Deployment & Cloud Infrastructure for Marine Pipelines