Beyond the Gaussian Belief

This is a teaching companion, not the authors’ implementation or a reproduction of their robotics experiments. Read the first half to understand the probability representation, then follow one prediction and one update before trying the complete scalar filter. Green curves are the richer representation; coral curves are a target or reference, except in the sensor lab where coral denotes a Gaussian sensor approximation. Every lab identifies its finite integration interval and displays numerical diagnostics. Source: Sangli Teng, Harry Zhang, David Jin, Ashkan Jasour, Ram Vasudevan, Maani Ghaffari, and Luca Carlone, arXiv:2506.00838v1, June 1, 2025.

Derive the fourth moment by writing each mixture component as plus or minus a plus s times a standard normal variable. Odd terms vanish by symmetry. A centered Gaussian with the same variance has fourth moment three times the variance squared, giving a difference of 2a^4. This original example explains why more moments can distinguish distributions that a mean-and-covariance description cannot. It does not assert that every fourth-order maximum-entropy fit recovers an arbitrary mixture exactly.

The paper’s MED coefficients, predicted moments, and MAP point are distinct objects. Optimizing the coefficients is a convex parameter-estimation problem. Searching for a mode is generally a nonconvex polynomial problem, handled through convex relaxations. A correct implementation must not confuse the moment sequence representing a probability density with the relaxation moments used to optimize its energy.

For several variables, alpha is a multi-index and x to alpha is the product of coordinate powers. The paper permits a more general polynomial residual h(y,x)=v and polynomial equality-constrained states. This deck begins with additive measurement noise because its likelihood has an unambiguous unit Jacobian. Every expectation requires that the relevant moment exists. In the labs the integration measure is approximated with trapezoidal weights, not with an unweighted array of density heights.

Entropy is not invariant to an arbitrary change of coordinates unless the reference measure is transformed consistently. Consequently the support and coordinates cannot be left implicit. On a finite grid, an interior feasible moment target has a well-behaved maximum-entropy solution. On unbounded continuous domains, existence, integrability, and limiting arguments need additional conditions. The paper’s asymptotic claim concerns moment-determinate distributions, not every conceivable heavy-tailed law.

Vary the entropy Lagrangian with respect to p and solve the pointwise stationarity equation. Absorb the constant multiplier into the partition function. On the real line, a positive quadratic coefficient is needed for the quadratic family to integrate. A positive quartic leading term gives integrable tails even when the quadratic term is negative. On a bounded interval, the corresponding order-two family is generally a truncated exponential-quadratic density, not literally the unrestricted Gaussian.

The paper writes an integral of the unnormalized exponential plus the multiplier-moment inner product. Minimizing over its constant coefficient yields this normalized log-partition dual, up to an additive constant. Differentiating log Z once gives minus the feature expectation, and differentiating again yields feature covariance. The browser solver uses theta=-lambda and scaled Legendre features, so its gradient is E[phi]-m. This is the same convex problem in another polynomial basis, not a different belief family.

Differentiating E gives 4x(x^2-a^2), so the stationary points are zero and plus/minus a. At zero the second derivative is negative when a is positive; at plus/minus a it is positive. Thus the state energy has two minima while the maximum-entropy coefficient-fitting dual remains convex. No contradiction is involved. This distinction prevents the misleading claim that the entire posterior is log-concave merely because its coefficients were obtained by a convex solver.

The target mixture is renormalized on the lab interval before its moments are measured. The solver fits a genuine finite-quadrature maximum-entropy model by optimizing log-sum-exp, not a hand-designed curve. It uses all polynomial features through the selected order, including odd features. Increasing the order enlarges the exponential family and, for an accurately solved feasible problem with the same target and support, cannot increase the optimal forward KL divergence. This does not imply monotonic point-estimation accuracy.

The default is a symmetric mixture with separation 1.5 and component standard deviation 0.48. Target and model are compared on a fixed quadrature grid. A low residual certifies only approximate satisfaction of the selected grid moments. It is not a statement about unmeasured moments, tails outside the interval, or global filtering error.

The lifted relation is exact algebra when all required moments exist and are supplied accurately. Exact propagation relative to the current fitted belief is not the same as exact propagation of the unknown true posterior. Reconstruction introduces a finite-order approximation, and numerical integration may introduce another one. A finite vector of stored moments alone need not close under nonlinear dynamics; the fitted density supplies the higher moments requested by the next step.

Expand the square of aX+b(X^2-1)+W. Independence and E[W]=0 remove its cross terms. The fourth input moment already appears in the second output moment. The implementation generalizes the expansion by multiplying polynomial coefficient arrays and applying the binomial theorem for independent additive noise. It uses input moments through 2r even when b=0, where that bound is conservative. The coral reference separately evaluates the pushforward density using the uniform-noise kernel.

The input is a truncated, slightly asymmetric two-component mixture on [-2.5,2.5]. The uniform noise support and slider range keep the exact pushforward inside [-3,4]. The reference uses deterministic quadrature over the input state and the exact uniform kernel; its edges have small grid artifacts. The green fit uses independently propagated raw moments, transformed into the solver’s Legendre basis.

These formulas describe a posterior density, not just a MAP estimate. Multiplying exponentials adds their energies. Constants independent of x affect the normalizer but not the location of a mode. If the sensor energy has degree r_v and g has degree d, the resulting state energy may have degree r_v times d; this need not equal the prediction reconstruction order. The complete scalar lab has a quartic posterior likelihood even when prediction is reconstructed using only order two.

This is a modeling qualification, not a claim that all residual equations in robotics are invalid. For equal-dimensional invertible changes of observation coordinates, the usual change-of-variables determinant is required. Noninvertible maps and dimension-changing constraints require an explicit observation measure and more care. In the paper’s planar localization rearrangement, multiplication by a proper rotation has determinant one. The additive sensor examples in this deck deliberately avoid this ambiguity.

The symbol s here sets the quartic energy scale; it is not the standard deviation of v. The lab calculates the sensor variance by quadrature before constructing the Gaussian baseline. Observations are a deliberately fixed sequence around alternating modes for a static state x=0.6; they are not claimed to be random draws from the sensor density. Both methods receive exactly the same observations and prior. A grid argmax is displayed only as an approximate mode, never as an SDP-certified estimate.

The normalizer is recomputed after each batch of observations. The gray curve is the original prior, green is the quartic sensor posterior, and coral is the Gaussian sensor approximation posterior. Blue marks the known synthetic truth. The same truth is not supplied to either inference calculation. The integration interval is [-4,4] with 501 points.

The update does not require a Kalman gain. The analogy with a Kalman filter is the recursive predict–correct structure, not a promise that nonlinear filtering is a finite closed-form recursion. The current exponential-polynomial density can be integrated to obtain more moments than were originally constrained at prediction. Those extra moments come from the chosen maximum-entropy representative, and therefore include a closure assumption.

The synthetic trajectory begins at +1.2, but the estimator receives only the broad zero-mean prior. A seeded generator produces the process and measurement noise. With zero control and an even initial belief, every transition and likelihood preserves reflection symmetry, so no correct filter should claim to know the sign. A nonzero known control changes this identifiability. At every step the MEM path integrates its posterior moments, propagates through the affine transition, reconstructs a maximum-entropy prediction, then multiplies by the quartic likelihood induced by y-x^2.

This is a genuine scalar implementation of the predict–reconstruct–correct architecture, but not the authors’ SE(2) solver. The comparison uses 321 points on [-4,4]. It reports L1 discrepancy between quadrature masses, the last prediction’s moment-fit residual, and the largest raw mass truncation discrepancy in the grid reference. Order denotes prediction reconstruction order, not the degree of the post-measurement energy.

Expand E[(x-a)^2 | y] into posterior variance plus (E[x|y]-a)^2 to obtain the mean’s squared-error optimality. MAP is tied to density relative to the chosen coordinates and measure. In the symmetric filter lab there may be equally plausible positive and negative modes; the grid argmax may return the first by indexing convention. That is not evidence for the negative sign. The paper’s point-extraction step specifically targets a posterior mode.

The paper discusses a rank-one solution as a sufficient extraction certificate. More general moment hierarchies use flatness conditions to extract finite collections of minimizers; a problem with multiple global minima need not have a unique rank-one moment solution. A certificate concerns optimization of the represented posterior energy. It does not certify that a finite-order reconstructed posterior equals the true filtering distribution, nor that a MAP point is minimum-MSE.

Expand both quadratic forms and collect the coefficients of x transpose x and x. Constants independent of x are absorbed into the normalizer. This derivation assumes positive-definite covariance matrices and Euclidean coordinates. The familiar exact Kalman closure relies on Gaussian initialization and Gaussian noise as well as linear models; a Gaussian approximation to genuinely non-Gaussian noise does not become exact just because the model is linear.

This slide connects the scalar multimodal sensor intuition to the paper’s SE(2) example. The paper uses equal landmark probabilities in its maximum-entropy association model and also evaluates a mismatched association frequency scenario. A moment representation does not magically infer arbitrary unknown association probabilities; the assumed probabilities and noise moments remain modeling inputs. The browser labs do not implement this constrained multidimensional localization experiment.

No performance numbers in these labs are presented as reproductions of the authors’ results. The model dimensions, sensor distributions, quadrature, optimizers, and synthetic sequences differ. The paper’s own small-sample discussion warns that multimodality can make a point estimate’s empirical error worse even when the represented distribution is richer. A useful reading question is whether the result concerns density approximation, MAP objective value, estimation error, or runtime: these are different axes.

The word arbitrary signals departure from a Gaussian-only noise assumption, not the removal of existence and integrability conditions. Higher-order moments grow quickly and may be unstable when estimated from limited data. Increasing r can improve expressiveness while making estimation and numerical conditioning harder. Polynomial state constraints require a suitable measure and consistency relations among moments. Numerical quadrature replaces a difficult symbolic marginalization; it does not eliminate integration from the method.

The model source is model.mjs and the tests are test.mjs. The build generates a static SVG for each lab from exactly the same functions used by the interactive controls. This makes printed slides honest computed snapshots instead of decorative illustrations. The filter reference reports raw prediction mass loss on the finite interval; the other labs use fixed normalized intervals explicitly stated in their controls. Browser tests check the native Bento document, the four iframe integrations, keyboard navigation, plot changes, MathJax rendering, and narrow-screen laboratory layout.

Reference checked against arXiv version 1. The slides’ scalar examples, numerical code, and explanatory derivations are original teaching material built around the paper’s ideas. No author-provided code is embedded. The actual Bento runtime and site-shared live-slide adapter are retained from the existing Random Thoughts presentation format, with their original license notices.