How two frames find each other

Rung one · correspondences known

The lucky case has a closed form

Suppose an oracle hands you paired points: $p_i$ in the new frame is the same physical point as $q_i$ in the old frame. Registration is then a least-squares problem over a rotation and a translation,

and — unusually for anything involving rotations — it needs no iteration at all. Center both clouds on their centroids $\bar p,\bar q$, build the cross-covariance $H=\sum_i (p_i-\bar p)(q_i-\bar q)^{\!\top}$, and the optimal rotation falls out of an SVD (Arun, Umeyama). In the plane the whole SVD collapses to a single angle,

with $\tilde p_i, \tilde q_i$ the centered points. Drag and twist the orange constellation below as far off as you like, then press Solve: one evaluation of these formulas lands it exactly, every time, from anywhere. No initialization, no local minima. Hold on to how good this feels — the next two rungs are about losing it.

The residual never reaches zero — it converges to the level implied by the noise baked into the measurements (per-axis σ = 0.14 u in the default noisy scene, 0.045 u in the clean one; 2-D residuals run about √2·σ, less the small share the three fitted parameters absorb). A registration that consistently fits far below its sensor noise is a sign of overfitting somewhere else in the pipeline. And switch the scene to 20% outliers, then Solve: the closed form dutifully appeases the junk and abandons the honest points — least squares' 0% breakdown point, one rung early. The next rung is the answer.

Rung two · correspondences corrupted

Real matchers lie: RANSAC

In practice nobody hands you correspondences; a feature matcher proposes them, and some fraction are simply wrong — repetitive texture, occlusion, a moving pedestrian. Least squares has zero tolerance for this: its breakdown point is 0%, so a single gross outlier can drag the closed-form fit arbitrarily far. The gray ghost below is the closed-form solve applied to all matches. Push the outlier slider and watch it wander off.

RANSAC inverts the logic: instead of letting every match vote on one fit, it lets many small fits compete for votes. Draw the minimal sample — two matches fix a planar rigid motion — solve exactly, and count how many other matches agree within a threshold $\varepsilon$. Keep the hypothesis with the largest consensus, then re-run the closed-form fit on its inliers. The number of draws needed to hit one all-inlier sample with confidence $p$ is

with $w$ the inlier ratio and $s=2$ the sample size — which is why minimal samples matter: $N$ explodes exponentially in $s$. The demo re-estimates $w$ from the best consensus so far and shrinks $N$ adaptively as it runs.

Things worth trying. At 45% outliers the naive fit is garbage while RANSAC recovers the motion in a handful of draws. Push outliers to 80% and $N$ climbs into the hundreds — consensus still wins, it just needs patience. Now widen ε: rejected matches start counting as inliers, and the "consensus" quietly absorbs corruption. The threshold is doing the epistemology.

Rung three · correspondences unknown

No correspondences at all: ICP

A lidar scan has no feature descriptors — just anonymous points along the walls. Now the problem is circular: knowing the transform would reveal the correspondences, and knowing the correspondences would give the transform in closed form. ICP (Besl & McKay) breaks the circle by alternation: guess correspondences by nearest neighbor, solve rung one's closed form on that guess, move the cloud, guess again.

For vanilla ICP — every point matched against the full reference cloud — neither half-step can increase the error, so the iteration always converges, though only to a local minimum picked by the initialization. The closed form's from-anywhere guarantee is gone: what you get now depends entirely on where you start. The orange scan below also covers only part of the room, so points hanging past the overlap have no true partner; the demo therefore gates far-away matches and trims the worst quarter each iteration to keep those from voting. That robustness is bought by giving up even the monotone-descent guarantee — the RMS strip can occasionally tick upward as the match set changes membership.

Things worth trying. The default scene now stacks σ = 0.10 noise with a fifth of junk, and from the default pose Run ends in honest failure: ICP declares convergence in a dozen-odd iterations at a pose some 35° off, its trimmed consensus happily built on outliers. Remember that opening state — rung three and a half repairs part of it, and the race later in this note shows what surviving it actually takes. Flip to high noise for the classic picture: ICP settles in about forty iterations — watch the RMS strip decay — and parks visibly off the truth, the pose-error stat reading about 4° and 0.13 u. Keep both numbers for the next rung. Now set the rotation past ±90° and run again: it converges just as confidently onto the wrong wall and stops, pose error high, perfectly satisfied. That is a local minimum, not a bug. On clean the bias disappears — the forty iterations of crabbing do not. And try unticking trim worst 25% on high noise: the discarded quarter is honest data, and the rank-cut itself is most of the bias — rung three and a half picks this thread up. The next rung's residuals are the cure for the crabbing.

Rung three and a half · a point meets a wall

Point-to-plane, GICP & robust kernels: matches with structure

Vanilla ICP's residual pretends the reference is a bag of isolated points. But it is a wall, sampled at arbitrary positions: the point your nearest neighbor should have matched almost never exists in the other scan, so point-to-point drags every pair all the way together and crabs sideways along every wall. Chen & Medioni's point-to-plane residual fixes the model instead of the matches: estimate a surface normal $n_j$ at each reference point (PCA over its neighbors — the blue ticks in the demo below, drawn once its residual selector is set to point-to-plane), and penalize only the error component across the surface,

so sliding along a wall costs nothing. The price is the closed form: each iteration now linearizes the rotation and solves a tiny 3×3 Gauss–Newton system. GICP (Segal et al.) then completes the thought symmetrically: give every point in both clouds a covariance disc — thin across its estimated surface, long along it; the blue and orange slivers the demo draws in GICP mode — and weigh each matched pair by the pair's combined uncertainty,

Point-to-point (spherical covariances) and point-to-plane ($C^A\!\to\!0$, $C^B$ a surface disc) drop out as special cases. Notice what just happened: every point became a little Gaussian, and matching became Gaussian against Gaussian. Hold that thought — rung four is one step away.

In practice the split between the two upgrades is clean. Point-to-plane already cures the crabbing: on structured scenes it captures most of the win, needs only one normal per reference point, and its scalar residual is cheap — which is why it remains the default polish stage in lidar pipelines. But it hands structure to one cloud only; the moving points stay structureless dots. GICP — plane-to-plane, in Segal et al.'s own naming — earns its extra eigendecompositions when the two scans are noisy peers rather than a clean map plus a fresh scan: the Mahalanobis weight discounts exactly the directions in which both surfaces are uncertain, and a shaky normal fattens its own disc, automatically softening that pair's vote instead of trusting a bad estimate at full strength. Watch the iteration counts in the demo below — the two share a basin; GICP's better-conditioned steps simply land a few iterations sooner.

One more upgrade completes the practical kit: robust kernels — the "robust" in robust ICP. Trimming is a hard vote: a pair is either in or out, decided by rank. M-estimators soften it: replace the squared loss with one that stops growing for large residuals, and solve by iteratively reweighted least squares — exactly the same solvers as above, with each pair scaled by a weight computed from its current residual $r$ and a robust scale $s$ (here $1.4826\cdot\mathrm{MAD}$ of the match distances):

Huber (convex, safe) merely tapers a far pair's influence; Tukey's redescending biweight drives it to exactly zero — an outlier eventually stops voting at all. Modern "fast and robust ICP" (Zhang et al.) is this recipe industrialized: a Welsch kernel inside the same alternation, plus Anderson acceleration to claw back the iterations robustness costs. The kernel composes with every residual above. Below, the same scans and the same start; pick a residual and a kernel — point-to-point, point-to-plane, and GICP are all in the residual selector — and compare iteration counts and pose errors.

Things worth trying. The default scene now opens on the hardest case — σ = 0.10 noise plus 20% junk, from the far default start. Press Run and every residual loses: with trim 25% all three finish tens of degrees off, the rank-cut consensus latched onto junk; switch robustness to Huber and point-to-plane recovers to about a degree while the other two still wander. That is this section's closing lesson delivered up front: kernels and trimming protect the fit inside the basin, they do not widen it — drag the scan near the truth and the very same scene converges for every residual under every kernel, point-to-plane and GICP around 1–2° with point-to-point a little behind. Flip to high noise for the cleaner story the residuals were built for: point-to-point grinds for about forty iterations and still parks visibly short of the truth — the pose-error stat gives it away — while point-to-plane and GICP land on it in under twenty, their surface models averaging the noise out along the walls; there, point-to-point's parked bias is largely trimming's doing — under pure noise the "worst quarter" is honest data, and cutting it by rank tilts the fit; Huber (or even none: pure noise has nothing to reject) makes the bias all but disappear. On clean, point-to-point reaches the truth too, but the counts keep talking: about forty iterations of crabbing against point-to-plane's thirteen and GICP's nine. Striped density barely bothers any of them here (trimming plus structure absorb it; the race later in this note shows where that stops being true). And on 20% outliers alone, from a close start, none lets the junk visibly drag GICP, trim 25% mostly fixes it, and Huber/Tukey do better still — a weight computed from the residual rejects junk more surgically than cutting a fixed quarter by rank. Robustness is a tool with a price, not a free upgrade.

Rung four · correspondences dissolved

NDT: trade points for sufficient statistics

Every rung so far still trafficked in correspondences — an oracle's, a matcher's, or nearest-neighbor guesses remade each iteration, each paid for with a search against the raw reference cloud. The Normal Distributions Transform (Biber & Straßer, 2003) dissolves them. Its observation: what the reference actually contributes to registration is local surface shape, and a Gaussian per cell captures exactly that. Subdivide the plane into square cells of side $h$; every cell $c$ holding at least three points gets the mean and covariance of its points,

The covariance is the local geometry: a cell on a straight wall gets a needle-shaped Gaussian — long along the wall, thin as the sensor noise across it — while a cell containing a corner gets a round blob. One degenerate case needs care: perfectly collinear points make $\Sigma_c$ singular, so the smaller eigenvalue is clamped to $0.001\,\lambda_{\max}$ before inverting (the demo does this too). The result is a piecewise-defined density that answers, for any location, "how likely is a surface point here?":

Below is a lidar scan of a room with a pillar, and its transform. Hover the cells to inspect each Gaussian: walls collapse into Gaussians exactly as thin as the sensor noise — generous in the default noisy-plus-junk scene, where a few stray cells also appear off the walls entirely; flip it to clean and they sharpen into needles — while corners become blobs. Then press Nudge grid a few times: the same points, binned with a shifted grid, give visibly different Gaussians. That discretization sensitivity is not cosmetic — it is the reason for a trick in step two.

Things worth trying. Shrink the cell size to 0.6: more, sharper Gaussians — a finer sculpture of the room, but each built from fewer points, and more cells dropped entirely. Push it to 2.4 and corners, walls, and the pillar melt together into fat blobs. Toggle the density field and look along a single wall: the density jumps at every cell border, because each location is explained by exactly one Gaussian. NDT's fidelity–smoothness dial is this one slider, and you will feel it again in both demos below.

Rung four · score a pose

Registration becomes function evaluation

To register a new scan, transform its points $x_i$ by a candidate pose $p=(t_x,t_y,\varphi)$ with $T_p(x)=R(\varphi)\,x+t$, look up the one cell each transformed point lands in — an $O(1)$ cell lookup, where ICP does a nearest-neighbor search — and sum the density under the scan:

where $(\mu_{ig},\Sigma_{ig})$ is the Gaussian of the cell of grid $g$ that $T_p(x_i)$ falls into. The inner sum is Biber's answer to the discretization sensitivity you just saw: keep four copies of the grid, each shifted by half a cell horizontally, vertically, and both, and let every point be scored by all four cells it lands in. No single cell border is a border of all four grids, so the worst seams cancel.

The heatmap below is this score over candidate translations, rotation held at the slider value — dark is high. The green ring marks the true pose. Click anywhere to drop a start, and a translation-only Newton iteration (the next section's machinery, with $\varphi$ frozen) runs on the actual score function and draws its path.

Things worth trying. The default scene stacks σ = 0.10 noise with 20% junk, and on this landscape the noise does all the talking — the junk only fills a few cells the ring never visits — so what you see is softened everywhere: wide gentle ridges and an easy stroll to the peak. And yet the path stat rarely reads zero, because with that noise in both scans the score's maximum genuinely sits a few hundredths of a unit off the true pose. That residual offset is the estimator's own noise bias, not a solver failure. Now flip the scene to clean at cell size 1.2 with a single grid: the landscape sharpens into a razor — thin dark ridges over wide flat shelves, terraced by cell borders. Click a start two cells from the ring and Newton stalls immediately — on a shelf the gradient is numerically zero, and no line search can fix a direction that does not exist. Turn on the four-grid sum: the terraces soften and the basin visibly widens — clean or noisy, it converts a good share of stalling starts into converging ones. Push cell size to 2.4: the peak broadens and clicks a full unit out stroll home, at the price of a blunter optimum — though the outermost corners of this map still win, because a straight wall's Gaussian stays only as thick as the sensor noise across it no matter how large the cell, so even a coarse grid has finite reach. Drag the rotation slider off zero and watch the whole mountain range decay: translation cannot repair a wrong rotation. This picture — sharp-but-narrow versus wide-but-blunt, set by one slider and by the sensor itself — is the entire art of tuning NDT.

Rung four · climb

Newton's method, Hessian included

Here is what the Gaussians buy that ICP never gets: the score is an analytic function of the pose, so its gradient and Hessian come in closed form. Only the rotation makes $T_p$ nonlinear, and in the plane its derivatives are two short vectors:

and the translation derivatives are constant. Writing $f=-s$ (minimize the negative score), $s_i=\exp(-\tfrac12\tilde q_i^{\;\top}\Sigma_i^{-1}\tilde q_i)$ for each point–cell pair, and suppressing the sum over the four grids, the derivatives are

a handful of dot products per point. Each iteration solves $(H+\lambda I)\,\Delta p=-g$, with $\lambda$ raised just enough to make $H$ positive definite when the surface is locally saddle-shaped, then backtracks along $\Delta p$ until the score improves. (The demo also caps a single step at half a unit and 0.2 rad — an uncapped Newton jump can leap into a better-scoring wrong basin and strand there.) Inside the basin this homes in within a couple dozen capped steps — watch the score strip. The orange scan covers only part of the room. Drag and twist it, then run.

Things worth trying. From the default pose on the default scene, plain Run now fails honestly: noise-fattened Gaussians plus 20% junk leave the 1.2-unit basin just short of the start, and the solver freezes a short walk from the truth while calling itself converged — compare the score stat against the truth score; a stall like this scores a fraction of it, which is exactly how a real system flags a failed registration. Press Coarse → fine: the same start is first pulled in by a 2.4-unit grid's wide basin, then polished at your slider's resolution — the standard NDT recipe, and on this scene not a refinement but a necessity. Flip to high noise alone and plain Run lands in about fifteen iterations — worth doing once to feel exactly how much difficulty the junk adds. It is not magic, though: twist the scan past ±40° and even the coarse grid hands you a confidently wrong pose. At cell size 0.6 the basin is a razor (drag the scan a few pixels and Run fails); at 2.4 it forgives sloppy starts but the final pose error stat reads visibly worse. One slider, both failure modes. The scene selector reshapes the transform itself: high noise fattens every wall Gaussian and — counterintuitively — widens the basin (24/40 hard starts converge under noise against 15/40 clean, in the study behind this demo) at the price of a blunter, slightly biased peak; outliers alone barely dent it, because junk points just make a few extra cells the true alignment never visits — but stacked on noise they shave the basin's edge, which is precisely what the default start now sits just outside of.

The wider field · the contestants

Three answers to one question: what stands in for the matches?

The rungs above climbed one ladder — ever more structure per match, until the matches dissolved. The wider field contains whole other ladders. MMD-Reg (Crane et al., ICML 2026) benchmarks three families of registration methods — hard-correspondence ICP variants, soft-correspondence probabilistic methods, and its own correspondence-free objective — under noise, skewed sampling density, and outliers. The rest of this note is a faithful 2-D miniature of that comparison, run live on one shared problem, so you can watch where and why the families part ways.

Every local registration method minimizes some misalignment measure between a transformed source cloud and a target cloud. Since true correspondences are unknown, each family substitutes something for them — and that substitution is the method. Hard-correspondence methods commit to one nearest neighbor per point and re-decide each iteration. Soft-correspondence methods let every source point fractionally match every target point, weighted by a probabilistic model. Correspondence-free MMD-Reg matches no points at all: it compares the two clouds as distributions, through the distance between their kernel mean embeddings.

The last two rows are the unfamiliar ones, and each gets its own section before the race: the soft family's mixture machinery first, then MMD-Reg's mean embeddings. The race then runs all of them — with RANSAC, GICP, and NDT from the ladder as controls, and two data-association transplants from target tracking as guests — so you know exactly what each panel is spending. A fourth answer — generate the registered scene outright and read the poses off afterwards — arrived with flow matching in late 2025, and waits at the end of the race's reading. And the note closes by putting the three classical families' champions through their loops stage by stage on one scene, ending in a single design table.

The wider field · the soft family

Soft assignment: registration as mixture fitting

The founding move of the soft family is to stop asking which target point each source point matches and ask instead: if the moving cloud were a probability density, how likely would the fixed cloud be? Coherent Point Drift (Myronenko & Song, TPAMI 2010) makes the construction concrete. Put a Gaussian of width $\sigma$ on every transformed source point $\mathcal T(y_m)$, so the moving cloud becomes a mixture of $M$ equal-weight components; add one flat uniform component with weight $w$ to own the junk; then choose the transform that maximizes the likelihood of the $N$ target points $x_n\in\mathbb R^{d}$ as a sample from that mixture. Expectation–maximization (EM) solves it, and since the whole family lives inside that loop, it is worth spelling out once. The unknown correspondences are the latent variables. The E-step (expectation) freezes the transform and computes, for every target point, the posterior probability that each mixture component generated it — not a decision about who matches whom, but a distribution over the possibilities. The M-step (maximization) freezes those probabilities — the responsibilities — and re-fits the transform that best explains the points weighted by them. Alternate until nothing moves; each round is guaranteed not to decrease the likelihood, which is EM's whole contract. And the E-step's output is precisely the object the family is named for — a soft assignment over every pair:

Read the denominator as two competing explanations: either some Gaussian claims the target point, or the flat term does — and a point far from everything quietly hands its mass to the outlier bin and stops voting. The M-step is rung one wearing weights: with responsibilities fixed, the best rotation comes from an SVD of the responsibility-weighted cross-covariance $\hat X^{\!\top}P^{\!\top}\hat Y$, det-corrected exactly like Kabsch — Myronenko & Song point out this exact weighted multidimensional solution had not been derived before them (earlier attempts dropped terms). Then the detail that defines the method: $\sigma^2$ gets its own closed-form update from the same likelihood. It starts wide — every pair sees every pair, the basin is broad — and tightens as the fit improves, sharpening the assignment automatically. The family's ancestors, Gold & Rangarajan's Softassign and Chui & Rangarajan's RPM, ran the same soften-then-sharpen loop with an explicit temperature schedule and Sinkhorn normalization; CPD's argument is that maximum likelihood already knows the schedule — "it is preferable to estimate $\sigma^2$ instead of using deterministic annealing." And in the limit the family closes over the ladder below it: as $\sigma\to 0$ the responsibilities collapse onto the nearest neighbor, so ICP is this family at zero temperature, with the outlier bin as its gate.

One aside on the name, because this note stays rigid: the "coherent" refers to CPD's non-rigid variant, where the transform becomes a smoothly regularized displacement field and nearby points are made to drift together — a story set aside here. In the rigid case the loop above is the whole algorithm: responsibilities, weighted SVD, $\sigma^2$ update, repeat. What remains of the scale question is the E-step's all-$MN$-pairs sum, and the paper's own remedy — a fast Gauss transform — already made it effectively linear (36k-point bunny: 3.5 hours naive, 51 seconds with FGT), a decade before that became table stakes.

FilterReg (Gao & Tedrake, CVPR 2019) is the same probabilistic sentence spoken nine years later, re-engineered end to end — and each change is a decision about where the compute goes. First it flips the mixture: the Gaussians sit on the fixed cloud and the moving cloud plays the data, so the sums run over points that never move and the spatial index is built once — where CPD's centroids drag their index along every EM iteration. Second, it notices that the E-step is a Gaussian blur: everything the M-step needs are two filtered moments of the fixed cloud — around each moving point, the Gaussian-weighted mass $M^0$ and mean $M^1$ — so a permutohedral-lattice filter (splat, blur, slice: the machinery of fast bilateral image filtering) delivers the E-step in linear time, with another 8× from a GPU. Third, it abandons the closed form CPD was proud of: the M-step is Gauss–Newton on a twist $\zeta\in\mathfrak{se}(3)$, which for a single rigid body merely approximates what the SVD solves exactly. The trade is generality — the same per-point Jacobian extends unchanged past the single rigid body (articulated chains, deformable node graphs; outside this note's rigid scope) — and, staying rigid, it buys one concrete thing: a point-to-plane residual drops straight into EM via filtered normals, which no closed form accommodates. On $\sigma$ it is agnostic: fix it and keep the index truly static, or run CPD's closed-form update and converge in fewer sweeps.

Measured on the same 3 500-point bunny, 50° off, the reversal buys two orders of magnitude — 38 ms against reference CPD's 6.1 s, with robust ICP between them at 218 ms (CPD's own FGT variant closes part of that gap; all of it paper-specific 2019 hardware, not a law of nature). The fair reading is not that CPD was slow but that the family's costs turned out to be engineering, not physics: the robustness lives in the mixture-plus-uniform model, which both share — the quadratic E-step and the bespoke M-step were negotiable. One branch of the family took a different exit entirely: GMMReg (Jian & Vemuri) fits mixtures to both clouds and minimizes the closed-form $L_2$ distance between the two densities — no EM, no assignments even soft ones — and SVR sparsifies those mixtures with support vectors. Compare the clouds as distributions and skip correspondence altogether: that is the next section's move, made with kernels instead of fitted mixtures. (The full three-way ledger — CPD against FilterReg against MMD-Reg, row by row — closes the note.)

Where this runs below. In the race, the green panel is this section at demo scale: Gaussian responsibilities over all pairs, the uniform outlier bin, a responsibility-weighted Kabsch M-step. One honest simplification — its σ follows an RPM-style fixed schedule (×0.90 per iteration) rather than CPD's closed-form update; at demo size the two behave alike, and the naive quadratic E-step is affordable precisely because there are only a few hundred points.

The wider field · the new objective

What MMD-Reg actually minimizes

GMMReg ended the last section comparing two fitted densities; MMD-Reg keeps the density view and drops the fitting — no mixture is estimated, the clouds enter directly. Maximum Mean Discrepancy embeds a distribution $P$ into a reproducing kernel Hilbert space by averaging the kernel's feature map, $\mu_P=\mathbb{E}_{x\sim P}[k(x,\cdot)]$, and measures the distance between two distributions as the distance between their embeddings:

For a characteristic kernel this is zero only when $P=Q$, so it compares whole densities, not just a few moments. Estimated on point clouds it needs every pairwise kernel evaluation — quadratic cost. The paper's trick is Rahimi–Recht random Fourier features: sample $D$ frequencies $\omega_k\sim\mathcal N(0,\ell^{-2}I)$ and map each point to

so the squared MMD becomes the squared distance between two feature means — one pass over each cloud, linear in the number of points. Registration is then the nonlinear least-squares problem — here $X$ is the moving cloud and $Y$ the fixed one, the reverse of CPD's lettering —

handed to Levenberg–Marquardt. Two knobs matter. The kernel scale $\ell$ sets what the objective can see: large $\ell$ gives a wide, smooth, forgiving basin; small $\ell$ gives a sharp, accurate, narrow one — so standalone MMD-Reg runs a decreasing $\ell$-schedule, coarse to fine (the paper uses $\ell$ from 4.0 down to 0.25 on outdoor LiDAR). The feature count $D$ sets how faithfully the random surrogate approximates the true MMD: small $D$ is fast but noisy, large $D$ smooth but slower. And because the optimum $\theta^\star$ is defined implicitly by $\nabla F=0$, its derivative with respect to the input clouds exists via the implicit function theorem — that is what lets Neural MMD-Reg backpropagate through the solver and learn an initializer plus a per-problem kernel scale.

The wider field · the race

Same scans, same start — nine solvers

Two partially-overlapping scans of the same room, misaligned by the sliders' rotation and offset. Each panel runs its own solver on identical data. Alongside the paper's four contestants, three rungs from the ladder above race as controls: RANSAC samples two-point congruences over the whole map and polishes with rung one's closed form — as a global method it never reads the init sliders at all; GICP is the hard family's covariance-weighted upgrade; and NDT runs rung four's damped Newton on a four-grid transform of the target, with its standard coarse-to-fine cell schedule. Two more panels are transplants from a neighboring field: registration's correspondence problem is a data-association problem, and target tracking has its own answers. BP computes association marginals by belief propagation on the bipartite match graph — every source point may claim at most one target point and vice versa, a mutual-exclusion constraint the CPD-style E-step simply does not have — and PMBM keeps a small weighted mixture of one-to-one global assignment hypotheses and, to feed the shared M-step, collapses them to per-pair weights — strictly a TOMB/MOMB-style marginalization; the real filter's defining trait is that it carries the mixture forward instead. Both are grafted onto the green panel's exact scaffold — same annealed σ, same 0.12 miss mass, same weighted-Kabsch M-step — so whatever separates them from the soft assigner is the association rule alone. (The two are compared on their home turf in the BP vs PMBM note; the PMBM here is a flavor, not the filter — ranked hypotheses by randomized greedy sweeps, no Bernoulli bookkeeping.) The stress settings recreate the paper's PCPNet conditions: high noise perturbs every point, gradient density smoothly over-samples one end of the room, striped density alternates dense and sparse bands, and outliers replaces a fifth of each cloud with uniform junk — and the default, noise + outliers, stacks the first and last into the meanest of the lot. Watch who cares about which.

The stats under each panel are the field's standard evaluation pair — RRE, the rotation error against ground truth, and RTE, the translation error — plus iterations spent. But a single run is an anecdote: the paper's protocol, like every registration benchmark's, is distributions — medians and success rates over many draws. The button below reruns all nine solvers over 25 seeded scenes at whatever setting and start the controls show, and fills the table with median RRE, median RTE, the success rate at a 5° threshold, and median iterations-to-stop; at the defaults it reproduces the fignote's success counts exactly. One honest asterisk: RRE and RTE exist here because the scene is synthetic — we own the truth. A deployed system does not, and must judge a match from the inside: residual RMS and inlier fraction for the ICP family and RANSAC, the mass in the outlier bin for soft assignment, a score threshold for NDT (the registration demo above flags a stall by comparing against the truth score — a luxury of simulation; deployed NDT settles for thresholds and consistency checks), and the size of the residual objective for MMD. Every one of those internal signals can be fooled — a confident local minimum scores well on all of them — which is why loop-closure pipelines verify registrations independently before trusting them. And the code is not sealed inside this page: every solver here, the scene generator, and this exact benchmark protocol live as readable modules in the companion repository — clone it and node benchmark.js reproduces the table below.

Things worth trying (each behavior below is what this demo actually measures across seeds, not a promise for every draw). The default setting stacks noise on 20% junk, and one Run usually sorts the families on sight: across 25 seeded scenes at the 45° start, point-to-point finishes under 5° of rotation error on 11, point-to-plane on 14, the soft assigner on 16 — when it lands it lands cleanly, median about 1°; when it loses, the junk has captured its mixture — and MMD on all 25. The ladder's controls read differently. GICP matches the soft family's sturdiness — 19 of 25, median 1.4° — its combined covariances discounting exactly the junk-against-wall votes that capture plain ICP (the paper's harsher 3-D benchmark, Figure 4, is less kind to it). NDT lands only 6 of 25, and it is not the junk's fault but the rotation's: its basin ends near ±40°, and from a ±15° init it lands 21 of 25 with coarse-to-fine doing the rest; slide init offset to 3 u instead and it dies outright — even a 2.4-cell coarse grid has finite reach. And RANSAC is the panel the sliders cannot touch: rotation and offset anywhere, identical answer — median ≈2.6°, rough but basin-safe, though on a few seeds it flips onto the room's near-symmetry, which is why a global stage's answer wants verification, not trust. That three-way split is the outro's production recipe drawn live: a global stage that ignores the init, local stages that polish inside its basin. The two tracking transplants then split the association question itself open. BP is the best pure upgrade on the board: the green panel's exact scaffold, but its exclusivity marginals hold 20 of 25 on the default against the soft assigner's 16, and 21 on 20% outliers — junk finds it harder to share a match an honest point has already claimed. PMBM is the specialist: on the default it collapses (1 of 25 — hard one-to-one hypotheses amputate the long-range diffusion a cold 45° start needs, the locality that tracking's prediction step and gating normally supply), but flip to striped density and the hypothesis mixture is the only association rule left standing on the soft scaffold — 21 of 25 at median 0.16°, against zero for both the soft assigner and BP — because a duplicated point cannot vote twice inside a one-to-one global hypothesis, and BP's softer exclusivity-in-expectation turns out not to be enough. Association rules have regimes. Flip the setting to clean at the default 45° start and the four paper contestants land — hard correspondences are unbeatable when they are right. Switch to striped density: point-to-point and the soft assigner usually get captured by the dense bands — a duplicated point is another vote, whether the vote is hard or Gaussian-weighted — while point-to-plane slides along the walls and MMD's distributional view mostly shrugs (BP falls with the soft assigner here; PMBM stands, as above). On 20% outliers the finishing order tends to follow the paper's Figure 4: the hard-correspondence panels latch onto junk most readily (point-to-point most often across seeds, though on some draws point-to-plane is the casualty), soft assignment is sturdier, MMD degrades most gracefully. On gradient density, watch MMD's RTE instead of its RRE: the rotation is fine but the translation acquires a systematic bias, because with partial overlap the density-weighted feature mean of the source shifts relative to the target's — an honest limitation of mean-embedding objectives (the paper's benchmark uses fully overlapping supports, and Neural MMD-Reg learns per-point overlap weights precisely to cancel this). Finally push the init to ±60°: nearly everyone dies — amusingly, vanilla point-to-point holds the widest basin on clean data — because every panel but RANSAC's is a local method, which is exactly why the paper pairs MMD-Reg with coarse kernel scales or a learned initializer, and why real pipelines put a global stage first.

The wider field · the landscape

Why smoothness is the whole argument

This heatmap is the objective itself, evaluated over candidate translations (rotation held at the truth): dark is low. Click anywhere to drop a start point and the matching optimizer runs — Levenberg–Marquardt on the MMD surface, translation-only gated ICP on the nearest-neighbor surface. Toggle between the two objectives and move the sliders: this is the difference the paper is built on, drawn as terrain.

Things worth trying. At ℓ ≈ 1.2 the MMD bowl is wide and smooth — LM strolls in from anywhere on the map. Shrink ℓ to 0.3: the minimum sharpens (better final accuracy) but the basin narrows and side ripples appear — starts from far away now stall, which is exactly why the paper anneals ℓ instead of picking one value. Drop D to 4 and resample ω a few times: the surface visibly wobbles between draws — the surrogate is stochastic, and more features buy fidelity. Then switch to the nearest-neighbor objective: same scene, but the terrain is kinked and terraced — every crease is a set of correspondences flipping. A discrete assignment cannot be differentiated, and an optimizer on this surface inherits every plateau; smoothing it away is precisely what the soft-assignment family and MMD-Reg, in different ways, are for.

The wider field · reading the race

What actually separates the families

ICP trusts its matches; that is its speed and its fragility. When data is clean and the start is close, hard nearest neighbors are the right answer and nothing beats their precision per unit of compute — on KITTI driving scans the paper honestly reports that multi-scale point-to-plane ICP remains ahead of MMD-Reg. But each point votes through its single match: duplicated points in over-sampled regions vote twice, outliers vote for garbage, and noise reshuffles who matches whom. In the paper's stress tests that is the whole story — ICP and GICP fail fastest as outliers grow (Figure 4), and non-uniform density skews them systematically. Point-to-plane's normal-projected residual buys a faster, better-conditioned basin on structured scenes, not robustness.

The probabilistic family buys robustness with model mass. Every pair contributes with a probability and the uniform bin absorbs the junk that captures the hard panels — the stress settings barely reach the green panel. The tax is the one the soft-family section priced: a quadratic E-step (in the MMD-Reg paper's measurements, CPD is prohibitively slow beyond a few thousand points), and escape routes that each buy their speed with machinery and hyperparameters. The green panel runs the naive E-step because at a few hundred points it can afford to; FilterReg exists because real clouds cannot.

MMD-Reg removes the matches instead of softening them. No assignment exists anywhere in the objective — only two feature-mean vectors of fixed size $2D$, so the per-iteration cost is linear in points and near-constant on a GPU. Individual outliers dilute a mean rather than hijack a match, which the violet panel shows as graceful degradation. And the objective's smoothness is not a convenience but the point: it makes registration a differentiable layer. That is what Neural MMD-Reg exploits — a set transformer proposes an initialization and kernel scale, the LM solve refines it, and gradients flow through the solution itself, reaching 0.006° rotation error on ModelNet40 — roughly five times below the best prior learned method in the paper's Table 4 and orders of magnitude below most earlier learned pipelines. (In that clean, unsupervised setting the network predicts only the initialization; the learned kernel scale and per-point overlap weights belong to the supervised partial-overlap variant.) The costs are equally plain: the surrogate is stochastic in $D$, the objective is non-convex (hence the ℓ-schedule or a learned initializer), and a fixed random-feature budget can blur very fine geometry that a converged point-to-plane ICP would nail.

The newest exit removes the optimizer itself. Register Any Point (Pan et al., 2025 — the same Bonn lab as KISS-ICP) casts registration as conditional generation by flow matching: a transformer-parameterized velocity field, trained on 141k multi-view samples from 17 datasets across four continents, transports noisy points along near-straight paths to the registered scene, and ten Euler steps replace the whole descent — no initial guess and no basin, because nothing is being descended at test time. Rigidity is not baked into the model but enforced at inference: each integration step projects every view back onto its own $SE(3)$ orbit, several samples are drawn and the most rigid one kept, and the final per-view poses are read off with the Kabsch algorithm — rung one's closed form, reappearing as the last line of the field's newest method. One network, zero-shot across sensors and scene scales, edges the correspondence pipelines pairwise (95.6% registration recall on 3DMatch, past GeoTransformer and BUFFER-X) and runs away from them multi-view (13.7° mean rotation error on ScanNet scenes against 47.6° for the best matching-plus-pose-graph pipeline). Its immediate ancestor, Rectified Point Flow (NeurIPS 2025 spotlight, by an overlapping team), set the template months earlier — one conditional generative problem covering pairwise registration and shape assembly; RAP is that idea scaled, and the near-straight rectified paths are what make ten integration steps enough where diffusion-style samplers walk much longer. The price is the generative bargain, stated plainly: correspondence knowledge amortized into weights — a hundred thousand scenes of supervision standing in for the E-step nobody runs at inference. (A trained two-dimensional miniature of exactly this recipe runs in the finale below.)

The families are converging on the same recipe from different directions. Multi-scale ICP anneals correspondence distances; the soft-assignment family lets its mixture width shrink — RPM by schedule, CPD by maximum likelihood; MMD-Reg anneals kernel scale. Everyone, it turns out, needs a coarse story before a fine one — the disagreement is about what should carry the alignment signal in between: matched points, weighted pairs, or distributions. Or, in the generative exit, none of the above: a learned flow that simply carries the points home.

The finale · one scene, three memories

What each method keeps — step by step

The ladder has been climbed, the race run, the terrain mapped. What remains is to put the three families' champions side by side and ask the bookkeeping question the whole note has been circling: between two sweeps, what does each method remember about "who matches whom"? CPD keeps every soft pair — the full $M\times N$ responsibility matrix. FilterReg compresses each row of that matrix before it is ever stored: one filtered target $\bar y_i$ and one confidence $\alpha_i$ per moving point. MMD-Reg refuses to keep correspondences at all — each cloud is averaged into a fixed-size feature signature, and all that survives is a single residual between the two signatures. Every soft pair; one soft target per point; one residual for both clouds. And the field's newest arrival answers by refusing the premise: a flow-matching network keeps nothing at run time — every correspondence it will ever use is amortized into its weights.

Below, each method walks its own loop stage by stage — press next or auto under a panel, or click any stage chip directly. CPD's eight beats: the problem, the mixture, the E-step web, then the closed-form M-step split into its three moves — center, rotate, move + scale — then the $\sigma^2$ tighten, then sweeps to convergence. FilterReg's four: density, filter, query, twist. MMD-Reg's four: encode, average, compare, optimize. The fourth panel is the generative exit run for real — a genuinely trained miniature of RAP's recipe (condition, flow, rigidify, read pose) whose velocity network learned this page's scene generator offline; the small print below says exactly what that means. The scene opens noisy with planted junk in both clouds; set the starting misalignment yourself with the initial-rotation and initial-shift sliders. And the scale switch stays the sharpest separator: CPD's "rigid" model is actually a similarity with a uniform scale $s$, so it recovers a 15% shrink exactly (watch its $s$ readout), while FilterReg and MMD-Reg are rotation-plus-translation by design and park with an honest size mismatch.

What to look for. Watch the counts in each caption: CPD carries a responsibility matrix of ~23 000 entries between sweeps, FilterReg carries one target per moving point, MMD-Reg carries a single 96-number residual — the whole trichotomy in three integers. Watch the junk: CPD's uniform bin owns the planted points in the fixed cloud (its data), FilterReg's confidence gate $\alpha_i$ zeroes out the planted points in the moving cloud (its data — the mixture flip swaps which cloud is "data," so the same junk is handled from opposite sides), and MMD-Reg, which has no outlier bin at all, folds all of it into both signatures — its residual floors above zero and the fit drags a touch toward the junk, exactly the price its paper's neural variant pays learned per-point weights to remove. Crank the initial rotation past ±90° and watch all three fall into the same trap — a confident wrong basin; none of these objectives is global, and each paper says so. Honest small print: everything is planar; CPD runs its full similarity EM ($R$, $s$, $t$, closed-form $\sigma^2$), with the M-step's three closed-form moves shown as separate beats of one sweep; FilterReg runs the sparse-grid filter E-step with a coarse-to-fine σ schedule ending at its slider and one Gauss–Newton twist step per sweep; MMD-Reg runs 48 random frequencies, Levenberg–Marquardt, and an ℓ-schedule ending at its slider; both mixtures share a uniform outlier density of 1/area of the demo box. The fourth panel's velocity network is real, not scripted: a 15,458-parameter MLP (an $\hat x$-prediction parameterization of conditional flow matching, with fixed random-wave signatures of both clouds standing in for RAP's transformer tokens), trained offline for a few minutes of CPU on this page's own scene generator across every rotation, shift, noise and junk setting — and on nothing else. That is the amortization bargain made visible: on the families it trained on it lands within a degree or two of the truth from any start; flip moving cloud 15% smaller — a setting it never saw — and it falls apart while CPD's closed form shrugs. Its κ slider reproduces RAP's ablation (even two steps nearly suffice — rectified paths are nearly straight), and rigidity forcing is RAP's test-time trick: fit a pose to the predicted endpoint each step and keep the rigid version, which tightens the tail errors.

That is the note's closing image. The speed ledger was priced in the soft-family section and the failure modes were drawn by the race; what the walkthrough adds is the part neither can show — that these are not four speeds of one idea but four different answers to what a correspondence is: a matrix, a target, a residual — or a trained network's weights. The ladder climbed from matches known to matches dissolved; the wider field asks a different question — how much memory of the matches your problem can afford — and every method on this page is an honest answer to it.