Render directly in 3DGS (Gaussian Splatting)
What if the proper math was able to create and animate the point cloud that forms the basis of the Gaussian Splatting and the parameters of the splats, directly from the model etc. as a Rendering Process?
A final render would be fully walkable in real time. A new game engine is conceived.
7 Comments
There may be someone that is currently working on it, but I don't know who.
twitter.com/softyoda/status/1705244159538315600
Also, I think it should be either a separated add-on or another branch, but this techno is still experimental and more research may arise from this new tech leading to novel implementation (see Aras unity implementation that bring lot of optimization)
At first glance, I was excited but now I'm not sure. It looks like just a 3DGS Viewer for UE5 etc., not a Model → 3DGS renderer that would make image (frames) rendering and 3DGS training unnecessary.
Oh, I didn't understood your proposal this way... But how would you convert a model to 3DGS (that have spherical harmonics and so on) without making the raytraced render ?
Do you have any idea how you technically envision this ? Are there any papers that already did what you say ?
Hello! I interpret this idea as including a new 3d data type in blender that integrates with geometry nodes, EEVEE render, maybe eventually Cycles, and is treated as its own type - this would likely include renderer updates and certainly core Blender updates. We could potentially get to some early prototypes with point clouds with attributes -
Hope to see this implemented sometime soon!
There is this workaround from Olli Huttunen at youtube.com/watch
Very promising!
While it doesn't really involve 3DGS directly into blender, having a ways to export fbx path (or collection of camera) as colmap (or kapture) format could be useful. (but could be a script/addon as it's not a core blender feature)
I think the real proposal to make is to get 3DGS render engine with eevee for example, or a new engine, we could easily combine using real time compositor.