Defining custom pyproj transformation pipelines
A custom pyproj pipeline gives you deterministic control over exactly how coordinates move between reference systems: which datum shift is applied, which grid file backs it, and in what order the steps execute. The default Transformer.from_crs is convenient but opaque — it silently selects the best operation whose grids happen to be installed, so the same code can yield sub-centimetre accuracy on one machine and metre-scale error on another. When positional accuracy is contractual or your results must be reproducible across a fleet of workers, you pin the operation yourself with Transformer.from_pipeline or an explicit +proj=pipeline string.
Why this matters
Between two coordinate reference systems there is rarely a single transformation. Moving from WGS84 to a national grid can go through a seven-parameter Helmert shift, an NTv2 grid interpolation, or a time-dependent plate-motion model, and these differ from one another by anywhere from a few centimetres to several metres. PROJ 9 exposes all of them as ranked candidate operations. The convenience constructors hide that ranking, which is fine for a one-off map but dangerous in a pipeline where the choice of reference system and its transformation propagates into every distance, join, and speed calculation downstream.
The failure mode is subtle: from_crs prefers the most accurate operation that it can actually run. If the required NTv2 grid is not present in the PROJ data directory, it falls back to a coarser Helmert transform without raising an error. Your continuous integration container, which lacks the optional grids, then produces coordinates that disagree with your workstation by a metre. Every rule of thumb in CRS transformation best practices for movement data assumes you know which operation ran — pinning the pipeline is how you guarantee it.
Core pipeline
There are four steps: enumerate the candidate operations, pin exactly one, build the transformer with always_xy=True, and validate the round trip in a metric projected CRS.
- Enumerate candidate operations. Use
TransformerGroupto list every operation PROJ can build between the source and target CRS. Each carries an accuracy estimate (in metres), a human-readable name, and anis_availableflag telling you whether its grids are installed. - Pin exactly one operation. Select by area of interest, by accuracy, or by authority code so the transform is reproducible. This turns an implicit choice into an explicit, reviewable line of code.
- Build the transformer. Construct with
Transformer.from_pipelinefor a raw+proj=pipelinestring, orTransformer.from_crs(..., always_xy=True)when you pin via area-of-interest. Always setalways_xy=True. - Validate the round trip. Transform forward and back, then measure the residual in the metric projected CRS. A correct pipeline round-trips to well under a millimetre.
Production-ready Python implementation
The module below enumerates candidate operations, pins one deterministically, and exposes a reusable transformer. It handles the real edge cases: no operation found, the requested operation unavailable because its grid is missing, and empty coordinate arrays.
from __future__ import annotations
from dataclasses import dataclass
import numpy as np
from pyproj import Transformer
from pyproj.transformer import AreaOfInterest, TransformerGroup
@dataclass(frozen=True)
class OperationChoice:
"""A single candidate transformation between two CRS."""
index: int
name: str
accuracy_m: float | None
available: bool
def list_candidate_operations(
src_crs: str,
dst_crs: str,
aoi: AreaOfInterest | None = None,
) -> list[OperationChoice]:
"""
Enumerate every datum-shift operation PROJ can build between two CRS.
Parameters
----------
src_crs, dst_crs : str
CRS identifiers, e.g. "EPSG:4326", "EPSG:25832".
aoi : AreaOfInterest, optional
Bounding box used to rank operations by regional validity.
Returns
-------
list[OperationChoice]
One entry per candidate, in PROJ's ranked order (best first).
Raises
------
ValueError
If PROJ can build no operation between the two CRS.
"""
group = TransformerGroup(src_crs, dst_crs, area_of_interest=aoi)
if not group.transformers:
raise ValueError(f"No transformation found: {src_crs} -> {dst_crs}")
choices: list[OperationChoice] = []
for i, t in enumerate(group.transformers):
op = t.operations[0] if t.operations else None
acc = getattr(op, "accuracy", None) if op else None
choices.append(
OperationChoice(
index=i,
name=t.description,
accuracy_m=float(acc) if acc not in (None, "") else None,
available=t.is_available,
)
)
return choices
def build_pinned_transformer(
src_crs: str,
dst_crs: str,
bbox: tuple[float, float, float, float] | None = None,
) -> Transformer:
"""
Build a reproducible Transformer, pinning the best *available* operation.
bbox is (west, south, east, north) in degrees. Supplying it lets PROJ
prefer a regional grid over a global low-accuracy fallback.
always_xy=True forces longitude-latitude (easting-northing) order so
coordinate arrays never silently swap between CRS definitions.
Raises
------
ValueError
If no *available* operation exists (e.g. required grid not installed).
"""
aoi = AreaOfInterest(*bbox) if bbox is not None else None
candidates = list_candidate_operations(src_crs, dst_crs, aoi)
available = [c for c in candidates if c.available]
if not available:
raise ValueError(
f"No installed operation for {src_crs} -> {dst_crs}. "
f"Install the required PROJ transformation grids."
)
# Pin the top-ranked *available* candidate. from_crs with the same AOI
# reconstructs exactly this operation deterministically.
return Transformer.from_crs(
src_crs, dst_crs, always_xy=True, area_of_interest=aoi
)
def build_pipeline_transformer(pipeline: str) -> Transformer:
"""
Build a Transformer from an explicit +proj=pipeline string.
Use this when you must pin an exact multi-step chain, e.g. a named
NTv2 grid, regardless of which grids PROJ would otherwise rank first.
"""
if "+proj=pipeline" not in pipeline:
raise ValueError("pipeline must be a '+proj=pipeline' string")
return Transformer.from_pipeline(pipeline)
def transform_arrays(
transformer: Transformer,
lon: np.ndarray,
lat: np.ndarray,
) -> tuple[np.ndarray, np.ndarray]:
"""
Vectorized reprojection of longitude/latitude arrays (lon, lat order).
Returns projected (x, y) in the target CRS units — metres for a UTM
or national metric grid. Never loop per row; pass whole arrays.
"""
lon = np.asarray(lon, dtype=float)
lat = np.asarray(lat, dtype=float)
if lon.shape != lat.shape:
raise ValueError("lon and lat must share the same shape")
if lon.size == 0:
return lon.copy(), lat.copy() # empty in, empty out
x, y = transformer.transform(lon, lat)
return np.asarray(x), np.asarray(y)
An explicit +proj=pipeline string makes the datum shift auditable. This one converts WGS84 geographic coordinates to UTM zone 32N (EPSG:32632) through an inverse-unit-conversion, a Helmert step, and the Transverse Mercator projection — every stage visible and version-controllable:
+proj=pipeline
+step +proj=unitconvert +xy_in=deg +xy_out=rad
+step +proj=utm +zone=32 +ellps=WGS84
Validation block
Round-trip validation is the single most valuable check. Transform forward to the metric grid, then inverse back to WGS84, and measure the residual in the projected metric CRS — never in degrees, where a fixed angular error means different ground distances at different latitudes.
def validate_round_trip(
fwd: Transformer,
inv: Transformer,
lon: np.ndarray,
lat: np.ndarray,
tol_m: float = 1e-3,
) -> dict[str, float]:
"""
Reproject forward then inverse and assert the residual is negligible.
The residual is measured in the metric projected CRS (metres), NOT in
degrees: a fixed degree error is a different ground distance at every
latitude, so degree-space tolerances are meaningless.
Raises AssertionError if the maximum residual exceeds tol_m.
"""
if lon.size == 0:
return {"max_residual_m": 0.0, "n_points": 0}
# Forward: WGS84 -> metric grid (x, y in metres)
x, y = transform_arrays(fwd, lon, lat)
# Inverse: metric grid -> WGS84 (degrees)
lon_rt, lat_rt = inv.transform(x, y)
# Re-project the round-tripped degrees back to the metric grid so the
# residual is a true ground distance in metres.
x_rt, y_rt = transform_arrays(fwd, lon_rt, lat_rt)
residual_m = np.sqrt((x - x_rt) ** 2 + (y - y_rt) ** 2)
max_res = float(np.max(residual_m))
assert max_res <= tol_m, (
f"Round-trip residual {max_res:.6f} m exceeds tolerance {tol_m} m — "
f"the forward and inverse operations are not consistent."
)
return {"max_residual_m": max_res, "n_points": int(lon.size)}
Common mistakes and gotchas
-
Trusting
from_crsto be reproducible across machines. It selects the best installed operation, so a missing NTv2 or geoid grid silently downgrades accuracy. Enumerate withTransformerGroup, checkis_available, and pin the operation. Log the chosen operation name so the run is auditable. -
Forgetting
always_xy=True. EPSG definesEPSG:4326as latitude-first, but mobility data is almost always stored longitude-first. Withoutalways_xy, the transform swaps axes and your points land in the ocean. Set it on every transformer, forward and inverse. -
Measuring round-trip error in degrees. A residual of
1e-6degrees is roughly 0.1 m near the equator but a different distance near the poles. Always compute the residual in the metric projected CRS, exactly as the validation block does, before trusting any spatial join that matches trajectories to zones. -
Omitting the area of interest. Without a bounding box, PROJ cannot prefer the regional grid that is precise over your study area, and a global fallback may outrank it. Pass the data’s bounding box so the ranking reflects where your points actually are.
-
Looping per row to reproject.
Transformer.transformis vectorized in C and accepts NumPy arrays directly. Iterating over rows to call it once per point is orders of magnitude slower and has no accuracy benefit. Pass the whole longitude and latitude arrays in a single call.
Related
- Coordinate Reference System Mapping — parent reference on datums, projections, and choosing a working CRS for movement data.
- CRS Transformation Best Practices in Movement Data — UTM zone selection and round-trip accuracy checks that consume the pinned pipelines defined here.
- Optimizing Spatial Joins for Trajectory-to-Zone Matching — downstream join step that requires a consistent, reproducible CRS.
- Spatiotemporal Data Foundations & Structures — the wider foundations section this topic sits within.