Monte Carlo, made visible
Explore Gibbs sampling—and the six other dominant methods—one step at a time. All seven run live in your browser: four chain samplers in the lab, and the weighted and particle methods in a second lab below. Every demo is the genuine algorithm, not an animation of one.
Rebuilt as a single self-contained note from the original Sampling Playground: same correlated-Gaussian target, same four chains with identical parameters — plus live demos for the three population methods the original only described.
Condition, then draw
These four are the chain samplers. The other three of the seven — importance, rejection, and SMC — are population methods and run in their own lab below ↘.
Freeze one coordinate, draw the other from its exact conditional distribution, then swap. Every proposal is accepted.
Measured on this page's seeds (ρ = 0.88, all 72 steps): Gibbs accepts every draw by construction, but its lag-1-autocorrelation ESS proxy is only 4 of 73 visible points — the coordinate-wise zig-zag pays for the free acceptances. Random-walk Metropolis–Hastings accepts 42% of its σ = 0.72 proposals and does no better (ESS proxy 4). HMC accepts 99% of its 9-leapfrog-step trajectories at ε = 0.16 and reaches an ESS proxy of 28 — the gradient buys distant moves that survive the accept test. Slice sampling accepts everything and lands at an ESS proxy of 6: it adapts its scale, but still moves one coordinate at a time. A separately seeded 20,000-move Gibbs run recovers the target's moments — variances within 0.01 of 1, correlation within 0.004 of 0.88 — so the chains are sampling the right thing, slowly or quickly. The ESS proxy uses only the lag-1 autocorrelation of x₁; treat it as a mixing hint, not a certified effective sample size.
01
The central idea
The cloud of visited points becomes an empirical approximation of the target. The methods differ mainly in how they choose the next point—and therefore in what they require, how quickly they mix, and where they struggle.
02
The sampler atlas
These are not seven interchangeable buttons. They belong to different families, make different assumptions, and sometimes return fundamentally different objects: a chain, exact independent draws, weighted samples, or a particle population.
x′ ∼ p(x | y), y′ ∼ p(y | x′)
Replace one variable at a time using an exact conditional draw.
accept with min(1, π(x′)q(x|x′) / π(x)q(x′|x))
Correct almost any proposal mechanism with an acceptance probability.
H(x, p) = −log π(x) + ½pᵀp
Use gradients and auxiliary momentum to propose distant, informed moves.
u ∼ Uniform(0, π(x)); x′ ∼ {x : π(x) ≥ u}
Turn density evaluation into uniform sampling under the density curve.
wᵢ ∝ π(xᵢ) / q(xᵢ), xᵢ ∼ q
Draw from an easier proposal and repair the mismatch with weights.
accept if u ≤ π(x) / Mq(x)
Keep proposal draws in proportion to how well they match the target.
propagate → weight → resample
Move a population of weighted particles through time or intermediate targets.
Draw, then repair
Measured on this page's seeds. The static target is the bimodal mixture 0.55·N(−1.35, 0.60²) + 0.45·N(1.15, 0.45²) with proposal q = N(0, 1.5²). Rejection sampling under the numerically tightened envelope (M = 2.36) accepts 42.6% of 4,000 attempts, matching the 1/M = 42.4% the envelope predicts — and every kept point is an exact independent draw. Importance sampling keeps all 600 draws, but its Kish effective sample size is 388 (65% of nominal), and its estimate of E[x] sits at −0.10 against the true −0.23 — about 1.9 standard errors off, consistent with the ~0.07 standard error that a 388-draw effective size implies. The SMC panel filters the state-space model xₜ = 0.92·xₜ₋₁ + 0.45·ε with observations y = x + 0.5·ν using 44 particles; systematic resampling triggers whenever the weight ESS drops below N/2, which happens on 28 of 60 filtering steps, each time keeping on average only 45% distinct particles — the impoverishment the atlas card warns about, visible when duplicate particles stack at the same position. The weighted-particle mean still tracks the hidden state with RMS error 0.37 over the full 60-step run, under the 0.5 observation noise.
03
Choose by structure
Start from the mathematical access you have—not from which sampler sounds most sophisticated.
A 100-dimensional continuous posterior is differentiable.
Gradients prevent random-walk behavior, and NUTS adapts the trajectory length automatically.
Always verify with diagnostics: trace plots, multiple chains, effective sample size, and—when applicable—weight degeneracy.
| Method | Needs | Returns | Signature strength | Typical failure |
|---|---|---|---|---|
| Gibbs | Sampleable conditionals | Correlated chain | No rejection or step-size tuning | Slow zig-zagging |
| Metropolis–Hastings | Density ratios | Correlated chain | Works with almost any proposal | Random walk or rejections |
| HMC / NUTS | Density + gradients | Correlated chain | Efficient long moves in high dimension | Bad geometry or divergences |
| Slice | Density evaluations | Correlated chain | Adapts its own local scale | Expensive interval search |
| Importance | Proposal density + target | Weighted independent draws | Parallel and reusable | Weight collapse |
| Rejection | Bounded envelope Mq ≥ π | Exact independent draws | Conceptually clean and exact | Vanishing acceptance |
| SMC | Proposal + incremental weights | Weighted particle population | Online and multimodal | Particle impoverishment |
04
Never stop at “it ran”
Early draws may still remember the arbitrary starting point. Discarding them can remove initialization bias—but cannot repair a chain that never mixed.
A well-mixing chain explores relevant regions quickly. Slow wandering, sticky runs, or separated chains warn that the sample is incomplete.
Effective sample size translates correlated or weighted draws into the approximate number of independent draws carrying the same information.
Run multiple chains. Agreement, stable traces, and diagnostics such as R̂ near one support—not prove—that they reached the same target.
The particle population that SMC pushes through time is the same object the EO-MTT note wrestles with when it maintains uncertainty over measurement partitions, and the weighted hypothesis mixtures in the BP vs PMBM note are exactly the "keep several weighted explanations alive" move that importance weighting formalizes — PMBM's ranked global hypotheses are a discrete cousin of a particle population. The annealed soft-assignment EM in the frame-registration note shares SMC's tempering trick: walk a sequence of easier intermediate objectives toward the hard one instead of attacking it cold. This page was rebuilt from the original Sampling Playground; the four chain samplers here use the same algorithms and tuning constants, re-implemented as commented vanilla JavaScript you can read in this file's source.