Forward physics + backward sensitivity
Ordinary ray tracing predicts what a scene produces. Differentiable ray tracing also tells us how that prediction would change if a supported scene parameter moved—so an optimizer can work backward from a target observation.
01 / The central idea
The word “differentiable” refers to derivatives of a chosen output or loss with respect to supported inputs. It does not mean that rays bend because of calculus, and it does not mean every discrete visibility decision is automatically smooth.
ordinary ray tracing
R(θ; ξ)
θ holds scene and system parameters. ξ
represents sampling choices in a Monte Carlo renderer. The result
might be pixel radiance, a path response, or a spatial field. With
Monte Carlo sampling, R(θ; ξ) is a sampled estimate;
its expectation over ξ is the underlying prediction.
differentiable ray tracing
D(R(θ; ξ), y*)θ − η ∇θℒA differentiable renderer supplies the derivative path needed for forward- or reverse-mode automatic differentiation, sometimes with specialized estimators for visibility boundaries.
Vertex positions, object pose, receiver/camera pose, or other continuously parameterized geometry.
Supported reflectance, roughness, permittivity, scattering, emission, or attenuation parameters.
Source intensity, lighting, antenna pattern, array geometry, sensor pose, or sampling configuration.
One scene / four pixel-forming operations
The camera, light, orange object, and floor below are an invented conceptual arrangement. Switch families and reveal the stages that turn their representation into a pixel. These are explanatory 2-D abstractions—not four production renderers or a quality benchmark.
surface representation
Triangles are projected into screen space. Coverage and depth select a surface sample before interpolation and shading.
Illustrative only: the overlay compares nearby conceptual states; these four teaching diagrams do not numerically evaluate a derivative.
These families overlap. Volume rendering integrates along camera rays, splatters often use rasterization-like kernels, and hybrid renderers are common. The selector compares the dominant operation used to form a pixel, not mutually exclusive scene types.
Sionna RT / PathSolver sampling
In Sionna RT, samples_per_src controls SBR samples
launched from each source. It is not a waveform sampling rate, a
CIR tap count, or receiver noise. The solver uses those samples to
discover interaction sequences that may become valid propagation
paths.
More samples make ray tubes narrower and increase the chance of finding relevant interaction sequences; they do not guarantee that every path is found. One PathSolver “sample” is a connected chain of ray segments—not a row of points sampled along one ray. It can yield candidates or valid paths at multiple depths and for multiple targets.
Each source spawns samples_per_src rays from a spherical Fibonacci lattice over the sphere. Initial directions are not drawn from the transmit antenna pattern; that pattern weights fields later.
Direct LoS is tested explicitly. At a surface hit, at most one enabled interaction continues until the sample exits or reaches max_depth.
SBR proposes primitive and interaction sequences. For a specular suffix, the image method moves provisional vertices to connect the target, then checks finite surfaces, interaction validity, and occlusion.
For valid paths, Sionna computes complex channel coefficients, delay, departure/arrival angles, and Doppler information; cir(), taps(), or cfr() are later representations.
deterministic scene, sampled discovery
The physical path set is determined by the scene and endpoints. The Fibonacci launch lattice is deterministic; later interaction choices and diffuse directions make discovery stochastic, while a fixed seed makes those choices reproducible.
PathSolver ≠ RadioMapSolver
PathSolver discovers source-to-target paths. RadioMapSolver transports contributions onto map cells and additionally supports controls such as Russian roulette and gain thresholds.
differentiability boundary
Candidate discovery is discrete and detached from later differentiation. For discovered topology, supported refinement and field quantities can carry gradients. The current API notes that the fastest default symbolic loop mode does not support AD; use an AD-compatible mode such as solver.loop_mode = "evaluated".
Behavior and defaults above were checked for Sionna RT 2.0.1 against the official Sionna RT Path Solver technical report and PathSolver API.
02 / Specified geometry
In a scene you authored, the wall, source, receiver, and specular law are all given — so the hit point, reflection angle, and path length are consequences, not variables. Nothing on this diagram is optimized, and no gradient is needed to compute any of it. This is the ordinary forward solve that a ray tracer performs.
known geometry · forward solve
The wall stays fixed. The image method determines the hit point and equal incident/reflected angles for every receiver position.
forward value
The broken emitter→surface→receiver path becomes one straight line to the mirrored receiver. Its intersection with the wall gives the specular hit—no angle search is required.
derived, not free
Every quantity on this diagram — hit point, angle, length, AoD/AoA — is a deterministic function of the specified scene. There is no free variable left here, which is exactly why gradients in a radio ray tracer attach to other quantities: the electromagnetic and system parameters covered in the next section.
When these are trusted, keep them fixed in the forward model.
The reflection angle can be unmeasured and still not be an independent unknown: geometry determines it.
These are the parameters Sionna's gradients actually attach to — the exact list, with what stays non-differentiable, is in the next section.
03 / θ in Sionna RT
Sionna RT, built on Mitsuba 3 and Dr.Jit, computes gradients of paths, CIRs, and radio maps with respect to — in the documentation's own words — “most parameters of the ray tracing process, including material properties, antenna and scattering patterns, orientations, and positions of objects.” That is an explicit, finite list, and the geometry you specified is not on it.
differentiable ✓
Radio materials — each surface's RadioMaterial holds relative permittivity εr, conductivity σ, scattering coefficient S, and XPD coefficient as AD-capable Dr.Jit floats. Shipped scenes use ITURadioMaterial: ITU-R P.2040-3 named materials (concrete, brick, glass, …) whose εr = a·fb and σ = c·fd follow tabulated frequency power laws.
Antenna patterns and scattering patterns — differentiable, and replaceable with trainable parametric or neural patterns.
Positions, orientations, and array geometries of transmitters, receivers, and objects.
One switch required: the default loop_mode = "symbolic" “does not support automatic differentiation” — set the solver's loop_mode = "evaluated" before calling dr.backward.
not differentiable ✗
Mesh geometry. No official parameter list includes scene mesh vertices or a wall coordinate. The wall from section 02 stays a constant of the forward model — exactly as you would expect.
Discrete path topology. Which surfaces a path visits is decided by SBR candidate discovery plus image-method refinement — a discrete selection, outside automatic differentiation. Gradients flow through the field computation of discovered paths, not through the discovery.
RIS, currently. Sionna RT v1/v2 has no reconfigurable intelligent surfaces; the TF-based v0.19 had them, including a gradient-based optimization tutorial for phase and amplitude profiles.
If you trust every number you typed, the tracer is a pure forward model and the gradients go unused. That reading of section 02 is correct.
NVIDIA calibrated materials, scattering, and antenna patterns against channel-sounder measurements (DICHASUS, Univ. of Stuttgart): ITU-default materials missed the measured channel gain by 4.93 dB on average; gradient-calibrated materials cut it to 2.16 dB, neural materials to 1.00 dB. The learned values also absorb geometry and model error — they are the best-predicting parameters, not certified physical constants.
The Sionna RT paper's own demo: gradient ascent on a transmitter's orientation to maximize received power in a chosen map region. The v0.19 tutorial did the same for RIS phase/amplitude profiles; envisioned extensions include precoding codebooks and array geometries.
Sources: the Sionna RT introduction and radio-materials API, the developer guide's “Calibrating Material Parameters Through Gradient Descent” example, and the papers arXiv:2303.11103 (material learning, transmitter-orientation optimization) and arXiv:2311.18558 (measurement calibration; Table II error figures).
04 / The difficult part
Continuous path contribution is one part of the derivative. When an edge crosses a ray, visibility changes discretely: a path exists on one side and is blocked on the other. Naively differentiating only the currently traced path misses that boundary effect.
Within a fixed hit sequence, intersection position and path contribution can vary smoothly. The Boolean visibility indicator is constant almost everywhere and undefined at its switch, while an integrated image or measurement can require an additional boundary term.
Li and colleagues explicitly sample discontinuity edges so their Monte Carlo estimator captures derivative contributions that normal path samples almost surely miss.
Another family warps or reparameterizes samples so a moving discontinuity becomes stationary in the transformed domain, making differentiation tractable.
05 / From derivative to solution
Once a renderer can propagate the relevant derivatives, it can sit inside an optimizer. The loop is simple; choosing a faithful model, identifiable parameters, a stable estimator, and a good initialization is not.
# θ: supported scene/system parameters
θ = initialize()
for step in range(K):
prediction = trace(θ, samples)
loss = distance(prediction, target)
gradient = backward(loss, θ)
θ = optimizer.update(θ, gradient)
# Validate with held-out observations and physics.automatic differentiation modes
Propagate a chosen input perturbation alongside the forward computation; this evaluates a Jacobian–vector product.
Backpropagate an output sensitivity through the recorded computation; this evaluates a vector–Jacobian product.
The better mode depends on input/output dimensions, memory, estimator design, and the renderer—not the word “differentiable” alone.
Estimate pose, geometry, materials, or illumination from images or other measured responses.
Adjust supported transport and sensor parameters so simulated observations better match measurements — Sionna\u2019s measurement-calibrated materials cut channel-gain error from 4.9 to 1.0\u20132.2 dB.
Move sources or sensors, tune patterns, or search continuous design parameters against a task loss.
Place the renderer inside a larger differentiable program while keeping an explicit physical forward model.
what the gradient does not promise
A low loss can still come from the wrong parameters when the inverse problem is ambiguous or the forward model is incomplete.
Multiple scenes can explain similar observations; gradients can reach a local minimum.
Monte Carlo estimates can have high variance—even when a gradient estimator is unbiased.
Paths can appear, disappear, or change interaction sequence; these events need special treatment.
A ray or geometric-transport model contains only the phenomena it implements; differentiability does not turn it into a full-wave solver.
Discrete choices and renderer features are not differentiable merely because an AD library is present.
primary sources and official documentation
The demos above are analytic teaching examples. These references describe production research methods and systems.