Deep Image output for Cycles (part 2 with explanation )
My previous submission about a deep image output request for VFX work was down-voted a few times and I thought I'd make a new proposal explaining the benefit / need for Visual Effects Pipelines.
Deep Image output (normally .exr or .exr 2.0 format) has been an industry standard in major VFX and animation studios for the past 10 years or more. This is a compositing tool that artists can use for multiple reasons and functionality. The extra deep render is usually a larger file size (based on the DEEP sampling from the renderer) and is read into a compositing package (usually nuke) as a separate read node that you combine with an RGBA render to "deepenate" it.
Once the RGBA color data is matched with the corresponding DEEP data, the compositing functionality is enhanced considerably! some examples:
-
Volumetric compositing. When using a deep compositing pipeline, you can composite a volumetric render with a separate non-volumetric render and the images will be merged with the transparency in tact (depth and density of the volume render). Imagine a cloud render with an aerial mountain render combined without holdouts.
-
Multiple teams, departments, and render engines combined without rendering extra matte passes or holdouts in the render. (an example I worked on was ILM was rendering black panther and spider man, and at Weta we were rendering the other characters and environment. Using deep we could seamlessly merge all of the renders without advanced compositing, hold outs, and mattes...)
-
With a Deep pass you also get 32 bit accurate depth data useful for zdepth, defocus, and pworld data in one pass.
-
As an example, If you are compositing a characters foot walking in grass you don't need to worry about holding the feet out with a matte that needs up to match up with a separate grass scatter render. When you deep combine in nuke it will combine both renders seamlessly holding out the grass in the proper worldspace. I hope that makes sense.
-
Robust compositing pipelines like Weta can use Deep as an object and or material picker similar to cryptomatte without rendering and object ID or Matte pass.
I have been working at major VFX studios trying to get Blender / Cycles involved in the pipeline but this is one of the main reasons I get shot down. The volumetric renders from Blender/Cycles are absolutely beautiful and incredibly fast... but without a Deep render to go along with them, most modern compositing teams will not allow it in their input process.
If you are wondering why more VFX studios aren't using Blender and Cycles this is one of the main reasons why.
I believe that Deep is an open tool that should be accessible to any team of developers. Maybe I'm wrong though! I think most external renderers for Blender have this output (renderman, octane, etc...) but cycles is one of the only modern engines without the option.
Instead of downvoting this much needed feature, I'd be happy to have a civil debate whether its worth the effort or not.
thanks so much
16 Comments
In addition to this, it would be nice to have more passes and/or be able to use refraction and reflection as masks. I was on a project where the comp team needed the refraction and reflection masking the object behind. I was able to reproduce it with a very painful setup with view layers and a very large node tree in the compositor VS another render engine(RS) that was a single checkbox called "reflect/refract IDs".
I totally agree. I would suggest starting a new idea/proposal for this. Also I think the aov main rgba should just the the “combined” aov… not sure why the rgba channel is just black on the multi channel exr?
This requires more develop power on cycles, I don't think the current blender developer team has enough time on this
Well... my plugin outputs exr with rgba naming
Deep EXR support will push blender to another level for sure. As well as mipmap/tx support. Currently I made a plugin, which can generate a modified depth channel for Deep From Image node in deep composite. github.com/RolandVyens/Industrial-AOV-Connector
But this can only be used with single layer of geometry, cannot be used with volume
Thank you Mike for posting this one. I completely agree. +1
Currently blender's z path its so confusing and hard to use. I think they really need to upgrade these relevant features
Unforturnately, blender seems not to be passionated about high-end VFX that really show off software capability, mip-map/tx texture full support has been on the road map for 5 years, but we still can't see any progress. projects.blender.org/blender/blender/issues/68917
I developed my blender branch that have native deepexr support and light material pass aovs. you can come to my branch here: github.com/RolandVyens/industrial-cg-platform
Hi, I'm working on it right now. I've already handled deep surfaces and semi-transparent surfaces (Transparency, DOF, Motion Blur), and I'm currently battling with volumes (the hard part).
I had hoped to solve that a week ago, but no luck yet. I still hope to make the first release within the next two weeks for CPU, but as I said, volumes are tricky to manage (no official documentation, need retro engineering).
After this first release, I'll work on CUDA and OptiX implementation.
Here are some pictures of where I am right now:
ibb.co/088pxLr
ibb.co/ycMx0D1k
ibb.co/4ZC224ZD
Hey, I have managed to solve the deep exr completely, I worked on this since January, are you interested in joining my branch's development? Here's my repo: github.com/RolandVyens/industrial-cg-platform
Good to know ;) I'll release my build today or tomorrow. I think we got the same idea :)
I made also a deep cache data to memory management, an automatic one, but I have a special panel for deep with parameters to optimise it.
My build also include a new zdepth that support volume, a new raymarcher for deep volume (Arnold like) and for motion blur and DOF, I used the same approach than Renderman (volume slab). And my roadmap include also LPE and more stuff. I will check your build too ;)
I'll put here the link to my github release soon, when I'll publish it (only the build for now, source code later after cleanup and roadmap finished).
We can talk together too to maybe make something together and propose an update at the Blender Foundation.
I start my build two month ago. Nobody talked about your build somewhere ? I saw no information about it.
An other note, it's a beta 0.5 for my release so my github will not be really wonderfull right now, I still have the old Splash screen ;)
wow, thanks for sharing your approach matthieu😺 When I vibe my industrial cg platform deepexr, I pulled an open source renderer called moonray (by dreamworks) which just open sourced last year, and It has robust and mature deep rendering method both for cpu and gpu, I let claude and codex learned their approach for deepexr, and port to cycles as well as compositor (roughly takes 3 months for both cpu and gpu optix, as well as volume debugging).
for the deepexr data rate control, I copied the arnold interface which are threshold controls.
The design for mine is based on my current job's experience, we use katana for lighting (I'm katana lighting TD), for our workflow, we only render one alpha deepexr, and do deep recolor in nuke. so for the compositor deep output, if I plug in rgba, it will output full depth sampled deepexr, if I plug single alpha channel, it will only output a, Z, Zback.
The current remaining issue for my deepexr is volume rendering memory consumption. for now the renderer honestly samples all deep datas at each shading point, but the moonray uses a smart weighted average deep data storage method, which is good enough before any artifacts happen. My current temp solution is auto-tiling with a adjustable deep memory budget.
I see LPE in your to-do list. My build has not implement full ball LPE system yet, beacuse it is another complex develop package. now it's only a fixed splitted material lobes for each light group.
I only shared my blender branch in my discord channel when I developing, and I didn't mention anywhere else because it hadn't meet my expectations and didn't fully pass my test before
Few times a year I google something like "blender deep render", every time with high hopes and always have to disappoint. This feature would make my life so much easier.