Notebook

3D representations, from Zwicker to Gaussian Splatting

Interactive notes to understand (and remember) how 3DGS works: every lesson has manipulable figures — move the sliders, drag things around, train a model in the browser.

How to use — serve the folder over HTTP (the demos use ES modules from CDN): python -m http.server 8010 from the root, then open http://localhost:8010/. An internet connection is required (Three.js and KaTeX come from CDN).

Lessons

00 Prerequisites: eigenvectors, eigenvalues, the Jacobian The mental images before the math: rails and trains, rolling pins, local maps, and the determinant as an area factor.
01 The primitive: the anisotropic 3D Gaussian μ, Σ = R·S·Sᵀ·Rᵀ, opacity, spherical harmonics — play with all 59 degrees of freedom.
02 'Traditional' splatting: kernels, not parameters QSplat / EWA surface splatting: the Gaussian as a reconstruction filter. Points-with-holes vs splats.
03 The forward pass: volumetric compositing Transmittance along a ray, the same equation as NeRF, Lagrangian vs Eulerian view.
04 Training: the scene is the model Analysis-by-synthesis: train 2D gaussian splatting live in the browser, with densification.
05 The EWA projection: Σ′ = J·W·Σ·Wᵀ·Jᵀ (coming) why a Gaussian stays ~Gaussian on screen.
06 Compression and streaming: SPZ, SOG, LoD (coming) quantizing a mixture; Spark 2.0's coarse-to-fine.

Experiments

The research logbook: what we tried on our data, which decisions we made and which paths we abandoned (and why). It is the empirical complement to the lessons — and the shared memory for doing research together: before repeating an attempt, check here first.

E001 Photorealistic 3DGS from our data: how much, where, and why not PSNR 28.7 along the capture path, collapse 20 cm off it; the intrinsics recovery; the depth-regularized series (E2/E3) and the scale_reg own goal.

3D Graphics

Outside the splatting thread: "classic" real-time graphics, as a tool to see the rendering pipeline. Interactive Three.js scenes where what matters is not just the result, but how the GPU gets there — opening up the G-buffer and its passes.

G01 A closed room, and the G-buffer inspector A closed environment with three windows, cube/pyramid/sphere and a flying camera (WASD + mouse); on top, a deferred renderer with a real MRT G-buffer and a panel to step through the 11 pipeline stages — with a masking divider to compare each step against the Beauty.

Demo

The best-so-far 3DGS scene, explorable in the browser: the assets come from S3, the viewer (Spark + three.js) runs entirely in the page.

▶ test-gta-demo-1 — video, input, point cloud and 3DGS aligned One timeline: the game video, the recorded player inputs, the point cloud and the E2 scene (PSNR 28.63, 1.88M gaussians) synchronized frame by frame — camera locked to the recording or free navigation.

Why this notebook

It was born out of the 3d-viewer work (Potree, the flat renderer, and the 3DGS test on the GPU box): these concepts genuinely matter for deciding how to render and stream our reconstructions. The thesis that ties all the lessons together:

Thesis — in traditional gaussian splatting the Gaussian is a reconstruction filter applied to measured geometry; in 3DGS it is a latent parameter of a generative image model, estimated by gradient descent through a differentiable renderer. Same primitive, two epistemologies: forward rendering vs inverse problem.

The manim/ folder: scene scripts to generate video versions of the animations (instructions in the folder's README).