Notebook
3D representations, from Zwicker to Gaussian Splatting
Interactive notes to understand (and remember) how 3DGS works: every lesson has manipulable figures — move the sliders, drag things around, train a model in the browser.
python -m http.server 8010 from the root, then open
http://localhost:8010/. An internet connection is required (Three.js and KaTeX come from CDN).
Lessons
- 00 Prerequisites: eigenvectors, eigenvalues, the Jacobian The mental images before the math: rails and trains, rolling pins, local maps, and the determinant as an area factor.
- 01 The primitive: the anisotropic 3D Gaussian μ, Σ = R·S·Sᵀ·Rᵀ, opacity, spherical harmonics — play with all 59 degrees of freedom.
- 02 'Traditional' splatting: kernels, not parameters QSplat / EWA surface splatting: the Gaussian as a reconstruction filter. Points-with-holes vs splats.
- 03 The forward pass: volumetric compositing Transmittance along a ray, the same equation as NeRF, Lagrangian vs Eulerian view.
- 04 Training: the scene is the model Analysis-by-synthesis: train 2D gaussian splatting live in the browser, with densification.
- 05 The EWA projection: Σ′ = J·W·Σ·Wᵀ·Jᵀ (coming) why a Gaussian stays ~Gaussian on screen.
- 06 Compression and streaming: SPZ, SOG, LoD (coming) quantizing a mixture; Spark 2.0's coarse-to-fine.
Experiments
The research logbook: what we tried on our data, which decisions we made and which paths we abandoned (and why). It is the empirical complement to the lessons — and the shared memory for doing research together: before repeating an attempt, check here first.
Monitored papers
The watchlist: recent papers that could improve our 3D reconstruction pipeline. One entry per paper — what it does, why it matters to us, and what event would change its status. Discarded papers keep their entry (with the reason), so we never re-evaluate from scratch. Mirrored by an index note in the EC Obsidian vault.
3D Graphics
Outside the splatting thread: "classic" real-time graphics, as a tool to see the rendering pipeline. Interactive Three.js scenes where what matters is not just the result, but how the GPU gets there — opening up the G-buffer and its passes.
Demo
The best-so-far 3DGS scene, explorable in the browser: the assets come from S3, the viewer (Spark + three.js) runs entirely in the page.
- ▶ test-gta-demo-1 — video, input, point cloud and 3DGS aligned One timeline: the game video, the recorded player inputs, the point cloud and the E2 scene (PSNR 28.63, 1.88M gaussians) synchronized frame by frame — camera locked to the recording or free navigation.
- ▶ data-bundle-demo — Multi-POV Sync A 105 s excerpt of a three-player GTA V lobby, wall-clock aligned: 720p video per POV, merged live transcript, raw input stream, VLM segmentation and session metadata, all running off one shared clock. The raw bundle files link straight from the page.
Data streaming
The infrastructure thread: how our dataset reaches a world-model lab's GPU without each researcher rebuilding the plumbing. Where the splatting lessons are about representing scenes, these are about moving them — the network stack, the protocols, and why "stream straight to the GPU over TCP" is the one thing you can't do.
Why this notebook
It was born out of the 3d-viewer work (Potree, the flat renderer, and the 3DGS
test on the GPU box): these concepts genuinely matter for deciding how to render and
stream our reconstructions. The thesis that ties all the lessons together:
The manim/ folder: scene scripts to
generate video versions of the animations (instructions in the folder's README).