✳ Visual SLAM · the whole process

One loop of SLAM

A robot drives one lap around a room it has never seen, building its map and estimating its own position at the same time — each from the other. Step through the lap to watch frame-to-frame registration, tracking, drift, loop closure, and the two repairs that follow. The small nonlinear optimizations run live in your browser.

Further reading: ORB-SLAM · ORB-SLAM3 · official implementation · visual inspiration: @sincethestudy

Scope of the demo. The pinhole reprojection residuals, keyframe window, pose graph, and dense finite-difference Levenberg–Marquardt solves are genuine. The front end is deliberately easier than visual SLAM: the registration inset receives metric planar landmark coordinates with known IDs, and loop detection plus its relative transform are supplied from the simulated truth. It therefore teaches the optimization structure, not a deployable camera front end.

estimated path & keyframes landmark estimate anchors, bolted down loop gap ground truth, revealed at the end

frames 0 keyframes 0 landmarks 0 reprojection error – true drift ?

What this is

This is the third piece of a series. The first taught PnP — freeze the map, move one camera until its view matches. The second taught bundle adjustment — adjust camera poses and landmarks jointly to reduce reprojection error. This page combines those ideas with keyframes, local refinement, loop constraints, pose-graph correction, and a final joint refinement.

Step 2 estimates a planar relative motion by aligning consecutive metric landmark constellations with known correspondences. That is closer to range-scan registration than monocular visual odometry, which must infer motion and scene geometry from image features and has scale and observability issues. Reprojection error is one useful internal residual, not the only measurement or quality signal available to a robot. It can stay small while the trajectory drifts because camera poses and map points can deform together; revisiting old visual information adds a long-range constraint that exposes this inconsistency.

Systems in the ORB-SLAM family separate tracking, local mapping, and loop closing, and use sparse nonlinear optimization. This page uses dense numerical Jacobians and also adds didactic relative-pose factors, an oracle loop transform, and pose priors in its final solve. Those choices make the sequence easy to see, but the exact factors and scheduling are implementation choices rather than a universal SLAM recipe.