Defining custom pyproj transformation pipelines

A custom pyproj pipeline gives you deterministic control over exactly how coordinates move between reference systems: which datum shift is applied, which grid file backs it, and in what order the steps execute. The default Transformer.from_crs is convenient but opaque — it silently selects the best operation whose grids happen to be installed, so the same code can yield sub-centimetre accuracy on one machine and metre-scale error on another. When positional accuracy is contractual or your results must be reproducible across a fleet of workers, you pin the operation yourself with Transformer.from_pipeline or an explicit +proj=pipeline string.

Why this matters

Between two coordinate reference systems there is rarely a single transformation. Moving from WGS84 to a national grid can go through a seven-parameter Helmert shift, an NTv2 grid interpolation, or a time-dependent plate-motion model, and these differ from one another by anywhere from a few centimetres to several metres. PROJ 9 exposes all of them as ranked candidate operations. The convenience constructors hide that ranking, which is fine for a one-off map but dangerous in a pipeline where the choice of reference system and its transformation propagates into every distance, join, and speed calculation downstream.

The failure mode is subtle: from_crs prefers the most accurate operation that it can actually run. If the required NTv2 grid is not present in the PROJ data directory, it falls back to a coarser Helmert transform without raising an error. Your continuous integration container, which lacks the optional grids, then produces coordinates that disagree with your workstation by a metre. Every rule of thumb in CRS transformation best practices for movement data assumes you know which operation ran — pinning the pipeline is how you guarantee it.

Core pipeline

There are four steps: enumerate the candidate operations, pin exactly one, build the transformer with always_xy=True, and validate the round trip in a metric projected CRS.

Custom pyproj Transformation Pipeline Four sequential steps: Enumerate candidate operations, Pin one operation by area-of-interest or code, Build transformer with always_xy, and Validate the round trip in metres. 1. Enumerate candidates TransformerGroup 2. Pin one operation by AOI / code 3. Build transformer always_xy=True 4. Validate round trip residual in metres
  1. Enumerate candidate operations. Use TransformerGroup to list every operation PROJ can build between the source and target CRS. Each carries an accuracy estimate (in metres), a human-readable name, and an is_available flag telling you whether its grids are installed.
  2. Pin exactly one operation. Select by area of interest, by accuracy, or by authority code so the transform is reproducible. This turns an implicit choice into an explicit, reviewable line of code.
  3. Build the transformer. Construct with Transformer.from_pipeline for a raw +proj=pipeline string, or Transformer.from_crs(..., always_xy=True) when you pin via area-of-interest. Always set always_xy=True.
  4. Validate the round trip. Transform forward and back, then measure the residual in the metric projected CRS. A correct pipeline round-trips to well under a millimetre.

Production-ready Python implementation

The module below enumerates candidate operations, pins one deterministically, and exposes a reusable transformer. It handles the real edge cases: no operation found, the requested operation unavailable because its grid is missing, and empty coordinate arrays.

PYTHON
from __future__ import annotations

from dataclasses import dataclass

import numpy as np
from pyproj import Transformer
from pyproj.transformer import AreaOfInterest, TransformerGroup


@dataclass(frozen=True)
class OperationChoice:
    """A single candidate transformation between two CRS."""
    index: int
    name: str
    accuracy_m: float | None
    available: bool


def list_candidate_operations(
    src_crs: str,
    dst_crs: str,
    aoi: AreaOfInterest | None = None,
) -> list[OperationChoice]:
    """
    Enumerate every datum-shift operation PROJ can build between two CRS.

    Parameters
    ----------
    src_crs, dst_crs : str
        CRS identifiers, e.g. "EPSG:4326", "EPSG:25832".
    aoi : AreaOfInterest, optional
        Bounding box used to rank operations by regional validity.

    Returns
    -------
    list[OperationChoice]
        One entry per candidate, in PROJ's ranked order (best first).

    Raises
    ------
    ValueError
        If PROJ can build no operation between the two CRS.
    """
    group = TransformerGroup(src_crs, dst_crs, area_of_interest=aoi)
    if not group.transformers:
        raise ValueError(f"No transformation found: {src_crs} -> {dst_crs}")

    choices: list[OperationChoice] = []
    for i, t in enumerate(group.transformers):
        op = t.operations[0] if t.operations else None
        acc = getattr(op, "accuracy", None) if op else None
        choices.append(
            OperationChoice(
                index=i,
                name=t.description,
                accuracy_m=float(acc) if acc not in (None, "") else None,
                available=t.is_available,
            )
        )
    return choices


def build_pinned_transformer(
    src_crs: str,
    dst_crs: str,
    bbox: tuple[float, float, float, float] | None = None,
) -> Transformer:
    """
    Build a reproducible Transformer, pinning the best *available* operation.

    bbox is (west, south, east, north) in degrees. Supplying it lets PROJ
    prefer a regional grid over a global low-accuracy fallback.

    always_xy=True forces longitude-latitude (easting-northing) order so
    coordinate arrays never silently swap between CRS definitions.

    Raises
    ------
    ValueError
        If no *available* operation exists (e.g. required grid not installed).
    """
    aoi = AreaOfInterest(*bbox) if bbox is not None else None
    candidates = list_candidate_operations(src_crs, dst_crs, aoi)

    available = [c for c in candidates if c.available]
    if not available:
        raise ValueError(
            f"No installed operation for {src_crs} -> {dst_crs}. "
            f"Install the required PROJ transformation grids."
        )

    # Pin the top-ranked *available* candidate. from_crs with the same AOI
    # reconstructs exactly this operation deterministically.
    return Transformer.from_crs(
        src_crs, dst_crs, always_xy=True, area_of_interest=aoi
    )


def build_pipeline_transformer(pipeline: str) -> Transformer:
    """
    Build a Transformer from an explicit +proj=pipeline string.

    Use this when you must pin an exact multi-step chain, e.g. a named
    NTv2 grid, regardless of which grids PROJ would otherwise rank first.
    """
    if "+proj=pipeline" not in pipeline:
        raise ValueError("pipeline must be a '+proj=pipeline' string")
    return Transformer.from_pipeline(pipeline)


def transform_arrays(
    transformer: Transformer,
    lon: np.ndarray,
    lat: np.ndarray,
) -> tuple[np.ndarray, np.ndarray]:
    """
    Vectorized reprojection of longitude/latitude arrays (lon, lat order).

    Returns projected (x, y) in the target CRS units — metres for a UTM
    or national metric grid. Never loop per row; pass whole arrays.
    """
    lon = np.asarray(lon, dtype=float)
    lat = np.asarray(lat, dtype=float)
    if lon.shape != lat.shape:
        raise ValueError("lon and lat must share the same shape")
    if lon.size == 0:
        return lon.copy(), lat.copy()  # empty in, empty out
    x, y = transformer.transform(lon, lat)
    return np.asarray(x), np.asarray(y)

An explicit +proj=pipeline string makes the datum shift auditable. This one converts WGS84 geographic coordinates to UTM zone 32N (EPSG:32632) through an inverse-unit-conversion, a Helmert step, and the Transverse Mercator projection — every stage visible and version-controllable:

TEXT
+proj=pipeline
  +step +proj=unitconvert +xy_in=deg +xy_out=rad
  +step +proj=utm +zone=32 +ellps=WGS84

Validation block

Round-trip validation is the single most valuable check. Transform forward to the metric grid, then inverse back to WGS84, and measure the residual in the projected metric CRS — never in degrees, where a fixed angular error means different ground distances at different latitudes.

PYTHON
def validate_round_trip(
    fwd: Transformer,
    inv: Transformer,
    lon: np.ndarray,
    lat: np.ndarray,
    tol_m: float = 1e-3,
) -> dict[str, float]:
    """
    Reproject forward then inverse and assert the residual is negligible.

    The residual is measured in the metric projected CRS (metres), NOT in
    degrees: a fixed degree error is a different ground distance at every
    latitude, so degree-space tolerances are meaningless.

    Raises AssertionError if the maximum residual exceeds tol_m.
    """
    if lon.size == 0:
        return {"max_residual_m": 0.0, "n_points": 0}

    # Forward: WGS84 -> metric grid (x, y in metres)
    x, y = transform_arrays(fwd, lon, lat)
    # Inverse: metric grid -> WGS84 (degrees)
    lon_rt, lat_rt = inv.transform(x, y)
    # Re-project the round-tripped degrees back to the metric grid so the
    # residual is a true ground distance in metres.
    x_rt, y_rt = transform_arrays(fwd, lon_rt, lat_rt)

    residual_m = np.sqrt((x - x_rt) ** 2 + (y - y_rt) ** 2)
    max_res = float(np.max(residual_m))
    assert max_res <= tol_m, (
        f"Round-trip residual {max_res:.6f} m exceeds tolerance {tol_m} m — "
        f"the forward and inverse operations are not consistent."
    )
    return {"max_residual_m": max_res, "n_points": int(lon.size)}

Common mistakes and gotchas

  • Trusting from_crs to be reproducible across machines. It selects the best installed operation, so a missing NTv2 or geoid grid silently downgrades accuracy. Enumerate with TransformerGroup, check is_available, and pin the operation. Log the chosen operation name so the run is auditable.

  • Forgetting always_xy=True. EPSG defines EPSG:4326 as latitude-first, but mobility data is almost always stored longitude-first. Without always_xy, the transform swaps axes and your points land in the ocean. Set it on every transformer, forward and inverse.

  • Measuring round-trip error in degrees. A residual of 1e-6 degrees is roughly 0.1 m near the equator but a different distance near the poles. Always compute the residual in the metric projected CRS, exactly as the validation block does, before trusting any spatial join that matches trajectories to zones.

  • Omitting the area of interest. Without a bounding box, PROJ cannot prefer the regional grid that is precise over your study area, and a global fallback may outrank it. Pass the data’s bounding box so the ranking reflects where your points actually are.

  • Looping per row to reproject. Transformer.transform is vectorized in C and accepts NumPy arrays directly. Iterating over rows to call it once per point is orders of magnitude slower and has no accuracy benefit. Pass the whole longitude and latitude arrays in a single call.

Back to Coordinate Reference System Mapping