We introduce PhysGaussian, a new method that seamlessly integrates physically grounded Newtonian dynamics within 3D Gaussians to achieve high-quality novel motion synthesis. Employing a customized Material Point Method (MPM), our approach enriches 3D Gaussian kernels with physically meaningful kinematic deformation and mechanical stress attributes, all evolved in line with continuum mechanics principles. A defining characteristic of our method is the seamless integration between physical simulation and visual rendering: both components utilize the same 3D Gaussian kernels as their discrete representations. This negates the necessity for triangle/tetrahedron meshing, marching cubes, ''cage meshes,'' or any other geometry embedding, highlighting the principle of ''what you see is what you simulate (WS2).''
Our method demonstrates exceptional versatility across a wide variety of materials--including elastic entities, plastic metals, non-Newtonian fluids, and granular materials--showcasing its strong capabilities in creating diverse visual content with novel viewpoints and movements.
Our approach consists of two phases: 3D Gaussian splatting reconstruction and physics-integrated novel motion synthesis. The first phase is mainly adopted from the original GS framework, which reconstructs a set of 3D Gaussian kernels renderable by point splatting process, from a set of multi-view images, associated camera information, and an optional initial sparse surface point cloud constructed by COLMAP. We additionally regularize this reconstruction process with an anisotropic loss term if needed to avoid over-skinny Gaussian kernels. The second phase directly views the Gaussian ellipsoids as the simulatable particle discretization of the scene for MPM simulation. Enabled by MPM solver for continuum mechanics and our novel kinematics for Gaussian kernels and spherical harmonics, the scene can undergo physics-aware deformations and maintain photo-realistic rendering quality simultaneously and seamlessly. For better physics compliance, we also perform Gaussian kernel internal filling to capture the internal deformation of the object as needed.
We introduce a continuum mechanics-based strategy tailored for 3D Gaussian kernels.
Our method also supports flexible control of dynamics via material parameters. A larger Young's modulus \(E\) (from right to left) indicates higher stiffness while a larger poission ratio \(\nu\) (from top to bottom) leads to better volume preservation.
We show the versatility of our approach across a wide range of materials, including elastic entities, plastic metals, non-Newtonian fluids, and granular materials.
@article{xie2023physgaussian,
title={PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics},
author={Xie, Tianyi and Zong, Zeshun and Qiu, Yuxing and Li, Xuan and Feng, Yutao and Yang, Yin and Jiang, Chenfanfu},
journal={arXiv preprint arXiv:2311.12198},
year={2023},
}