Pullback Fisher–Rao Atlas

Figure gallery

Browse all 47 figures in paper order, with their original captions. Select an image to view it at full size.

Figures by Pierre Guilbert, licensed under CC BY 4.0. Please credit the author and paper when reusing or adapting a figure.

Figure 1; description below
Figure 1. Three univariate parametric families (first row) and iso-contours in the parameter plane (second row). Left column: exact Fisher–Rao distance around 𝒩(0,1/2)\mathcal{N}(0,1/2). Middle and right columns: Riemannian length of the Euclidean chord (an upper bound on the geodesic distance), for the linear Gibbs family ϕ=(x,x2)\phi=(x,x^{2}) and the nonlinear Gibbs family U=cos⁡(xθ1+x2θ2)U=\cos(x\theta_{1}+x^{2}\theta_{2}).
Figure 2; description below
Figure 2. The Fréchet differential of f(x,y)=(x+0.35sin⁡y,y+0.35sin⁡x)f(x,y) = (x + 0.35\sin y,\ y + 0.35\sin x) at the point aa. Left: the Cartesian grid of the source plane, the base point aa, the two vectors uu and vv, and the parallelogram they span. Right: the same grid transported by ff, which curves it, together with the images f(a+u)f(a+u), f(a+v)f(a+v) and f(a+u+v)f(a+u+v) of the three other vertices. Superimposed at f(a)f(a) is the parallelogram generated by dfa(u)df_{a}(u) and dfa(v)df_{a}(v): the linear map dfadf_{a} preserves the parallelogram structure, and its diagonal is dfa(u+v)=dfa(u)+dfa(v)df_{a}(u+v) = df_{a}(u) + df_{a}(v). The dashed segments measure the distance between the linear model and the true images; they are the remainder o(‖h‖)o(\lVert h \rVert), and they shrink faster than ‖h‖\lVert h \rVert as uu and vv are shortened.
Figure 3; description below
Figure 3. Two overlapping local charts. Here ℳ\mathcal{M} is the sphere, p0p_{0} its south pole, and p1p_{1} its east pole. The chart φ\varphi (red) maps a neighbourhood of p0p_{0} onto the lower plane; the chart ψ\psi (blue) maps a neighbourhood of p1p_{1} onto the plane on the right. The two grids, pushed onto ℳ\mathcal{M} by the inverse maps, intersect. In the blue plane, the deformed red grid is the image of this overlap under ψ∘φ−1\psi \circ \varphi^{-1}: the transition map remains smooth.
Figure 4; description below
Figure 4. Intrinsic definition of a tangent vector as an equivalence class of curves. On the manifold ℳ\mathcal{M}, the curves γ1\gamma_{1} (black) and γ2\gamma_{2} (orange) both pass through pp with the same velocity but different curvature. In each chart their images remain distinct, yet they share a common tangent arrow at the chart image of pp: vφ=(φ∘γ1)′(0)=(φ∘γ2)′(0)v_{\varphi}=(\varphi\circ\gamma_{1})'(0)=(\varphi\circ\gamma_{2})'(0) and vψ=(ψ∘γ1)′(0)=(ψ∘γ2)′(0)v_{\psi}=(\psi\circ\gamma_{1})'(0)=(\psi\circ\gamma_{2})'(0). The transition map ψ∘φ−1\psi\circ\varphi^{-1} relates the two representatives by vψ=d(ψ∘φ−1)φ(p)vφv_{\psi}=d(\psi\circ\varphi^{-1})_{\varphi(p)}\,v_{\varphi}, which is precisely the chart-independence of the equivalence relation γ1∼pγ2\gamma_{1}\sim_{p}\gamma_{2}.
Figure 5; description below
Figure 5. Effect of the embedding ψ\psi on the metric. On the left, the simplex 𝒫+\mathcal{P}_{+} of densities (p1,p2,p3)(p_{1},p_{2},p_{3}) with p1+p2+p3=1p_{1}+p_{2}+p_{3}=1; on the right, its image under ψ:p↦2p\psi : p \mapsto 2\sqrt{p}, the positive orthant of the sphere of radius 22. Both surfaces are coloured by the Fisher–Rao distance to the uniform distribution p⋆=(13,13,13)p_{\star}=(\frac{1}{3},\frac{1}{3},\frac{1}{3}), marked in white, together with its iso-distance curves. On the right, these are the geodesic circles of the sphere; on the left, their images under ψ−1\psi^{-1}. These curves become denser near the boundary of the simplex: one of the coefficients pip_{i} approaches 00, and the metric u⊤diag(1/p)uu^{\top}\mathrm{diag}(1/p)\,u diverges.
Figure 6; description below
Figure 6. Two interpretations of a displacement in 𝒫+(ℝ)\mathcal{P}_{+}(\mathbb{R}). On the left, Fisher–Rao: the same dipole uu, with ∫u=0\int u=0, has a low cost at the centre of p=𝒩(1,1)p=\mathcal{N}(1,1) and a high cost in the tail, where pp is small. On the right, Wasserstein: pp is translated first by a small displacement and then by a large one; the cost is the length of the transport arrows.
Figure 7; description below
Figure 7. Graph of φ(u,v)=(u,v,u2+v2)\varphi(u,v)=(u,v,u^{2}+v^{2}) in ℝ3\mathbb{R}^{3}, coloured by a numerical approximation of the Riemannian distance to the point (1,1,2)(1,1,2) (geodesic shooting, Runge–Kutta 4), and the same function on the parameter plane (u,v)(u,v). The iso-contours on the graph can be read again in the chart: this is the pullback. The arrow ξ\xi in the plane is mapped to dφ(ξ)d\varphi(\xi); by definition, (φ*⟨⋅,⋅⟩)(ξ,ξ)=⟨dφ(ξ),dφ(ξ)⟩\bigl(\varphi^{\ast}\left\langle \cdot,\cdot \right\rangle\bigr)(\xi,\xi)=\left\langle d\varphi(\xi),\, d\varphi(\xi) \right\rangle.
Figure 8; description below
Figure 8. Three paths between the same two points under the square-root embedding p↦2pp\mapsto 2\sqrt{p}. The grey surface is the positive orthant of the sphere of radius 22, and the black curve is a parametric subfamily. Left: the red path remains within the subfamily. Centre: the blue Fisher–Rao geodesic may leave the subfamily but remains on the sphere. Right: the green straight chord leaves the sphere and lies in the ambient L2L^{2} space of square-root coordinates. This last path represents the chordal, or Hellinger, geometry and should not be confused with the ordinary L2L^{2} straight line between the original densities.
Figure 9; description below
Figure 9. Geometric interpretation of the natural gradient in the two-dimensional example. Top left: the Euclidean unit circle and the Euclidean gradient on the graph of ff. Top right: the same graph with the Riemannian unit ellipse dθ⊤Gdθ=1d\theta^\top G\,d\theta=1 and the Riemannian gradient. Bottom left: the coordinates y=A(θ−θ0)y=A(\theta-\theta_0), with A=G1/2A=G^{1/2}, straighten the metric locally and turn the ellipse into a Euclidean unit circle. Bottom right: direct comparison of the Euclidean and Riemannian gradients in the original parameter coordinates.
Figure 10; description below
Figure 10. Three centred univariate Gaussians (μ=0\mu = 0), for σ∈{0,5,1,2}\sigma \in \{0{,}5,\ 1,\ 2\}. The standard deviation spreads the density and lowers its peak.
Figure 11; description below
Figure 11. Fisher–Rao distance from 𝒩(0,1)\mathcal{N}(0,1) in the parameter plane (μ,σ)(\mu, \sigma). Blue-to-red colour map and level sets; the “circles” of the metric are not Euclidean circles.
Figure 12; description below
Figure 12. The same mean shift Δμ=0,05\Delta\mu = 0{,}05. On the left, with σ=0,1\sigma = 0{,}1, the densities separate clearly (∫|p−q|dx≈0,40\int |p-q|\,dx \approx 0{,}40); on the right, with σ=1\sigma = 1, they remain almost indistinguishable (∫|p−q|dx≈0,04\int |p-q|\,dx \approx 0{,}04). The orange area marks the loss of overlap.
Figure 13; description below
Figure 13. The same increase in standard deviation Δσ=0,05\Delta\sigma = 0{,}05. On the left, from 0,10{,}1 to 0,150{,}15, the profile changes considerably; on the right, from 11 to 1,051{,}05, it barely changes. Once again, the cost depends on the current scale of σ\sigma.
Figure 14; description below
Figure 14. Fisher–Rao geodesic from 𝒩(−2,1)\mathcal{N}(-2,1) to 𝒩(2,1)\mathcal{N}(2,1). On the left, the arc in (μ,σ)(\mu, \sigma); on the right, the densities at the numbered points. We increase σ\sigma before translating μ\mu.
Figure 15; description below
Figure 15. Four multivariate Gaussians in ℝ2\mathbb{R}^{2}, centred at μ=0\mu = 0. From left to right: Σ=I\Sigma = I, elongation along x1x_{1}, elongation along x2x_{2}, then orientation θ=π/4\theta = \pi/4. The surface represents the graph of p0,Σp_{0,\Sigma}; the curves on the base plane are the level sets of the density.
Figure 16; description below
Figure 16. Metric gμ,Σg_{\mu,\Sigma} on mean displacements, at fixed Σ\Sigma. Here Σ\Sigma has angle π/4\pi/4 and principal variances (1,4)(1,\ 4). The level sets are Mahalanobis ellipses; the arrows mark the principal axes.
Figure 17; description below
Figure 17. Numerical approximation of a Fisher–Rao geodesic from 𝒩((0,0),Σ)\mathcal{N}((0,0),\Sigma) to 𝒩((2,2),Σ)\mathcal{N}((2,2),\Sigma), projected onto the (μ1,μ2)(\mu_{1},\mu_{2}) plane, with θ\theta fixed at π/4\pi/4. The mean displacement follows the line segment; it is Σ\Sigma that changes.
Figure 18; description below
Figure 18. Parameters (σ1,σ2,θ)(\sigma_{1},\sigma_{2},\theta) of Σ\Sigma along the same curve. The standard deviations move away from their endpoint values in the middle of the path (spreading), then return to them; θ\theta is constrained to π/4\pi/4.
Figure 19; description below
Figure 19. Densities along the same curve. From 11 to 77, the mean shifts while the Gaussian widens and then narrows again, as in the univariate case.
Figure 20; description below
Figure 20. Fisher–Rao geodesic in the (σ1,σ2)(\sigma_{1},\sigma_{2}) plane, with μ=0\mu = 0 and θ=π/4\theta = \pi/4 fixed. The path is a hyperbola (a straight line in logarithmic coordinates), distinct from the Euclidean line segment.
Figure 21; description below
Figure 21. Densities along the geodesic. The mean and orientation remain fixed; the principal variances exchange their values, changing from (1,4)(1,\ 4) to (4,1)(4,\ 1).
Figure 22; description below
Figure 22. Densities along the orientation geodesic. The mean and standard deviations are fixed; only θ\theta changes from π/4\pi/4 to 3π/43\pi/4.
Figure 23; description below
Figure 23. On the left, θ(t)\theta(t) along the geodesic (affine); on the right, the Riemannian speed, which is constant because gθθg_{\theta\theta} does not depend on θ\theta.
Figure 24; description below
Figure 24. Numerical approximation of a Fisher–Rao geodesic between two general bivariate Gaussians (discrete energy minimization). At each step, μ\mu, (σ1,σ2)(\sigma_{1},\sigma_{2}), and θ\theta evolve simultaneously.
Figure 25; description below
Figure 25. The same curve, projected into (σ1,σ2,θ)(\sigma_{1},\sigma_{2},\theta). The increase in θ\theta is concentrated near the diagonal σ1=σ2\sigma_{1}=\sigma_{2}, where rotation costs the least.
Figure 26; description below
Figure 26. Three centred univariate Laplace densities (μ=0\mu = 0), for σ∈{0,5,1,2}\sigma \in \{0{,}5,\ 1,\ 2\}.
Figure 27; description below
Figure 27. Univariate Laplace: the same mean shift Δμ=0,05\Delta\mu = 0{,}05. On the left, σ=0,1\sigma = 0{,}1; on the right, σ=1\sigma = 1. The orange area marks |p−q||p-q|.
Figure 28; description below
Figure 28. Univariate Laplace: the same increase Δσ=0,05\Delta\sigma = 0{,}05. The misalignment is large at small scale and small at large scale.
Figure 29; description below
Figure 29. Four elliptical Laplace densities in ℝ2\mathbb{R}^{2}, centred at μ=0\mu = 0. From left to right: Σ=I\Sigma = I, axis-aligned elongation, orientation π/4\pi/4, orientation 3π/43\pi/4.
Figure 30; description below
Figure 30. Numerical approximations of the Fisher–Rao geodesics between the same endpoint parameters, projected into (σ1,σ2,θ)(\sigma_{1},\sigma_{2},\theta). Gaussian in blue, elliptical Laplace in red.
Figure 31; description below
Figure 31. Evolution of the densities along the two numerical curves, at the same seven stages: Gaussian on the top, elliptical Laplace on the bottom. Each panel shows the values of μ\mu, (σ1,σ2)(\sigma_{1},\sigma_{2}), and θ\theta.
Figure 32; description below
Figure 32. Three linear Gibbs families in dimension one, for fixed ϕ\phi and θ\theta. From left to right: ϕ(x)=x\phi(x)=x, ϕ(x)=(x,x2)\phi(x)=(x,x^{2}), ϕ(x)=(x2,cos⁡x)\phi(x)=(x^{2},\cos x).
Figure 33; description below
Figure 33. Three linear Gibbs densities in ℝ2\mathbb{R}^{2}. From left to right: ϕ(x)=(x1,x2)\phi(x)=(x_{1},x_{2}), ϕ(x)=x12+x1x2+x22\phi(x)=x_{1}^{2}+x_{1}x_{2}+x_{2}^{2}, ϕ(x)=(cos⁡(x12+x22),sin⁡(4x1x2)+x1+x2)\phi(x)=\bigl(\cos(x_{1}^{2}+x_{2}^{2}),\,\sin(4x_{1}x_{2})+x_{1}+x_{2}\bigr).
Figure 34; description below
Figure 34. Circles of the metric in the (θ1,θ2)(\theta_{1},\theta_{2}) plane. On the left, g=Hessψg=\mathrm{Hess}\,\psi: the background shows the level sets of ψ\psi. On the right, a field with fixed eigenvalues whose axes rotate with θ2\theta_{2}; the background, ∂2g11−∂1g12\partial_{2}g_{11}-\partial_{1}g_{12}, is the Schwarz obstruction, which vanishes if and only if gg is a Hessian metric.
Figure 35; description below
Figure 35. Progressive degeneration of the Fisher–Rao metric for a linear Gibbs family. The panels show the Fisher–Rao distance from the reference parameter θ⋆\theta_{\star} for ϕα(x)=(x,hα(x)),hα(x)=(2−α)x+(α−1)x2,\phi_{\alpha}(x)=\bigl(x,h_{\alpha}(x)\bigr), \qquad h_{\alpha}(x)=(2-\alpha)x+(\alpha-1)x^{2}, as α\alpha decreases from 22 towards 11. The metric is gθ=Cov⁡θ(ϕα(X)).g_{\theta}=\operatorname{Cov}_{\theta}\!\bigl(\phi_{\alpha}(X)\bigr). For α=2\alpha=2, ϕ2(x)=(x,x2)\phi_{2}(x)=(x,x^{2}) traces a parabola: its two components are affinely independent and the metric is positive definite. As α→1\alpha\to1, the image of ϕα\phi_{\alpha} becomes increasingly close to the affine line ϕ1(x)=(x,x)\phi_{1}(x)=(x,x). One eigenvalue of Cov⁡θ(ϕα(X))\operatorname{Cov}_{\theta}(\phi_{\alpha}(X)) then becomes small: the corresponding direction in parameter space produces an almost constant variation of the energy and therefore hardly changes the density. The progressive stretching of the iso-distance contours makes this near-degeneracy visible. In the limit α=1\alpha=1, the image is contained in a line, the parametrization becomes redundant, and the Fisher–Rao metric loses rank.
Figure 36; description below
Figure 36. A step of the same Euclidean length ε\varepsilon along each eigen-direction of the metric at θ⋆\theta_{\star}, for ϕ(x)=(x,h1.25(x))\phi(x)=(x,h_{1.25}(x)). The middle panel shows the reference density pθ⋆p_{\theta_{\star}}; the side panels show the displaced density pθ⋆+εvp_{\theta_{\star}+\varepsilon v} in blue together with the displacement dpθ⋆(εv)\mathrm{d}p_{\theta_{\star}}(\varepsilon v) in red. On the left, the step is taken along the direction of the smallest eigenvalue: the displacement is weak, the density barely changes. On the right, the same step is taken along the direction of the largest eigenvalue: the displacement is much stronger and visibly reshapes the density.
Figure 37; description below
Figure 37. Numerical approximations of Fisher–Rao geodesics (discrete energy) on three parameter spaces Θ⊂ℝ2\Theta\subset\mathbb{R}^{2}, the metric G(θ)=Covθ(ϕ)G(\theta)=\mathrm{Cov}_{\theta}(\phi) being evaluated by a Riemann sum on a bounded window. The first two families are defined on E⊂ℝE\subset\mathbb{R}, the third on E⊂ℝ2E\subset\mathbb{R}^{2}. Departure in blue, arrival in green; densities at the intermediate points.
Figure 38; description below
Figure 38. Two non-linear Gibbs densities. On the left, E⊂ℝE\subset\mathbb{R} and U(θ,x)=cos⁡(xθ1+x2θ2)U(\theta,x)=\cos(x\theta_{1}+x^{2}\theta_{2}); on the right, E⊂ℝ2E\subset\mathbb{R}^{2} and U(θ,x)=(x1−θ12)θ2+x22θ2U(\theta,x)=(x_{1}-\theta_{1}^{2})\theta_{2}+x_{2}^{2}\theta_{2}.
Figure 39; description below
Figure 39. Three push-forward families of a standard Gaussian base. At the top, the transport ϕθ\phi_{\theta} itself: the curves x↦ϕθ(x)x \mapsto \phi_{\theta}(x) for the two one-dimensional examples (the identity in dashes), and the image of a grid for the two-dimensional example. At the bottom, the resulting densities pθp_{\theta}, with the base qq in dashes.
Figure 40; description below
Figure 40. Eulerian velocity field induced by a parameter direction. Transport ϕθ(x)=(x1+θ1tanh⁡x2,θ2x2)\phi_{\theta}(x)=(x_{1}+\theta_{1}\tanh x_{2},\, \theta_{2}x_{2}) at θ=(0,8,1,2)\theta=(0{,}8,\,1{,}2), with dθ=e1d\theta=e_{1}. Solid: the grid transported by ϕθ\phi_{\theta}. Dashed: the same grid after θ+εdθ\theta+\varepsilon d\theta, ε=0,15\varepsilon=0{,}15. Red arrows: vdθ(y)=∂ϕ∂θ(θ,x)dθv_{d\theta}(y)=\frac{\partial\phi}{\partial\theta}(\theta,x)\,d\theta at y=ϕθ(x)y=\phi_{\theta}(x), computed from the initial point xx. A direction in Θ\Theta becomes a vector field on EE.
Figure 41; description below
Figure 41. Conservation of probability under a push-forward. Base q=𝒩(0,I2)q=\mathcal{N}(0,I_{2}) and transport ϕθ(x)=(x1+0,8tanh⁡x2,1,3x2)\phi_{\theta}(x)=(x_{1}+0{,}8\tanh x_{2},\, 1{,}3\, x_{2}), with det⁡∂ϕ∂x=1,3\det\frac{\partial\phi}{\partial x}=1{,}3. Left: density qq, region AA, and particles from the base (in red those of AA). Middle: the same particles of AA mapped by ϕθ\phi_{\theta}. Right: density pθ=(ϕθ)♯qp_{\theta}=(\phi_{\theta})_{\sharp}q, transported region Aθ=ϕθ(A)A_{\theta}=\phi_{\theta}(A), and the same particles after transport. The probability mass of the material region is unchanged.
Figure 42; description below
Figure 42. Divergence as infinitesimal local rate of volume change. The same material cell AA (solid) is deformed by y↦y+εv(y)y\mapsto y+\varepsilon v(y) (dashed), ε=0,28\varepsilon=0{,}28. Left: radial expansion v(y)=(ay1,ay2)v(y)=(a y_{1}, a y_{2}), div⁡v=2a>0\mathop{\mathrm{div}}v=2a>0. Middle: shear v(y)=(ay2,0)v(y)=(a y_{2}, 0), div⁡v=0\mathop{\mathrm{div}}v=0: the cell becomes a parallelogram, the area is unchanged. Right: radial compression v(y)=(−ay1,−ay2)v(y)=(-a y_{1}, -a y_{2}), div⁡v=−2a<0\mathop{\mathrm{div}}v=-2a<0. The numerical ratios Vol(Aε)/Vol(A)\mathrm{Vol}(A_{\varepsilon})/\mathrm{Vol}(A) match 1+εdiv⁡v1+\varepsilon\,\mathop{\mathrm{div}}v to first order.
Figure 43; description below
Figure 43. From volume change to density change, under conservation of probability. Left: a material cell at θ\theta, with local shading of pθp_{\theta}, particles for the probability mass, and arrows of vdθv_{d\theta}. Middle: local expansion div⁡vdθ>0\mathop{\mathrm{div}}v_{d\theta}>0: the same particles occupy a larger cell, the density decreases. Right: local compression div⁡vdθ<0\mathop{\mathrm{div}}v_{d\theta}<0: the same particles occupy a smaller cell, the density increases. The particle count is identical in the three panels.
Figure 44; description below
Figure 44. A nonzero velocity field invisible to Fisher–Rao. Level sets of p=𝒩(0,I2)p=\mathcal{N}(0,I_{2}) and the rotational field v(y)=(−y2,y1)v(y)=(-y_{2},\, y_{1}). The arrows are tangent to the contours: div⁡v=0\mathop{\mathrm{div}}v=0 and ⟨∇p,v⟩=0\left\langle \nabla p,\, v \right\rangle=0, so div⁡(pv)=0\mathop{\mathrm{div}}(pv)=0 and u=−div⁡(pv)=0u=-\mathop{\mathrm{div}}(pv)=0. The points move, the density does not change.
Figure 45; description below
Figure 45. Transport ϕ(x)=(x1+α(tanh⁡x1+tanh⁡(x1−c)),x2)\phi(x)=(x_{1}+\alpha(\tanh x_{1}+\tanh(x_{1}-c)),\, x_{2}) of a standard Gaussian, α=−0,42\alpha=-0{,}42, c=2,25c=2{,}25. At the top, the same density p=ϕ♯qp=\phi_{\sharp}q; the floor is div⁡v\mathop{\mathrm{div}}v on the left and div⁡(pv)\mathop{\mathrm{div}}(pv) on the right. At the bottom, the uniform grid in the coordinates of qq, then its image by ϕ\phi. The two squeezes read on the grid as clearly as each other; Fisher–Rao keeps only the one in the bulk.
Figure 46; description below
Figure 46. Numerical approximations of Fisher–Rao geodesics for two push-forward families, q=𝒩(0,1)q = \mathcal{N}(0,1) (discrete energy, metric by quadrature on the base). At the top the path in Θ⊂ℝ2\Theta \subset \mathbb{R}^{2} (in red), compared with the Euclidean chord (in dashes); at the bottom the densities pθp_{\theta} along the path, with (θ1,θ2)(\theta_{1},\theta_{2}) in the title.
Figure 47; description below
Figure 47. Numerical approximation of a Fisher–Rao geodesic for the two-dimensional transport ϕθ(x)=(x1+θ1tanh⁡x2,θ2x2)\phi_{\theta}(x) = (x_{1}+\theta_{1}\tanh x_{2},\ \theta_{2}x_{2}), base q=𝒩(0,I2)q=\mathcal{N}(0,I_{2}). At the top left the path in Θ\Theta and the Euclidean chord; at the top right the image of a grid by ϕθ\phi_{\theta} at the two endpoints, which shows the non-affine shear; at the bottom the densities along the path, with (θ1,θ2)(\theta_{1},\theta_{2}) in the title.