← All ideas
  • Published 30 Jan 2020

environment node for the compositor

I propose an environment image node for the compositor so you can quickly try different backgrounds. Useful for scenes whose lighting doesn't have to match the hdri image, such as indoor scenes where the hdri is visible through the window.

Perhaps if each rendered pixel could store a dictionary of the hdri pixels that contributed to it's final colour (key), and the amount that each hdri pixel contributed (value), then maybe the entire lighting/reflections etc could be recalculated in composition. A bit like corona renderer's interactive light mix (but for hdri). The dictionary should probably be stored on the hard drive though, as it'd probably be huge.

0%

Development Status: Open

Upvote 0 Downvote 1
3di
30 Jan 2020

9 Comments

jason

You may need to give a bit more detail about how this differs from simply rendering with an alpha channel and compositing your scene over a different background.

3di Author

* @Jason van Gumster * Obviously you'd only do this if the scene isn't primarily lit by the world hdri, otherwise the lighting and enviornment would look out of place. Unless the second half of my proposal is doable.

For tripod animations (camera rotates in one place), the comp node could calculate rotation based on camera angle at that frame, the rotation value in the comp node, and the frame at which the comp node rotation was set.

3di Author

* @Jason van Gumster * it means you can change and rotate any hdri in comp without having to render it from the 3d view.

jason

* @3d illusions * You're going to have to detail how camera data gets into the Compositor. As far as I know, the compositor has no awareness of any cameras. It mixes images and that's it.

3di Author

* @Jason van Gumster *

bpy.data.objects["Camera"].rotation_euler

jason

* @3d illusions * How does that work for sequences of images or render layers that are sourced from other scenes?

Look, I know how to access the data in Python. That's not my question. How do you propose that data find its way to the Compositor in a way that the user can control it (and accounts for these other use cases I've mentioned)?

3di Author

* @Jason van Gumster * The blender interface is python based, this means that nodes etc are able to access most data via what's called an API. The API is basically pythons way of using blender's C/C++ functions and the blend file data.

With this in mind, because the comp node would be written in python, it's able to programatically take user input and make calculations based on other data in the entire blend file at each frame. In this case the user would render a still image with transparent background, go into the compositor, create a environment node, and rotate the HDRI to taste. The python code would then create a list to hold the rotation vector entered, the active camera's name, the active camera's euler rotation, and the frame number used for this still image.

Now that that information is stored, for each frame that's rendered, the compositor node would get the active camera's name and rotation, and then calculate the new rotation of the comp hdri based on that. If the camera is not the same one stored in the comp nodes list, then the new active camera's rotation would be compared to that of the one stored in the comp nodes list, the offset calculated (between the new cam and the old one at the same frame stored in the list (not current)), and then that taken into consideration when setting the hdri rotation.

For different scenes you would generally be using the same camera angle as the current scene (otherwise things wouldnt line up in comp), but you would always be able to use multiple hdri nodes per node graph (one per render layers node), and of course you would also be able to keyframe the hdri comp nodes rotation, which would result in a new list of relevant data for that frame.

For different view layers, what do you see as an issue, or has the above cleared that up?

jason

* @3d illusions * Although the interface is written in Python, the Compositor is not. Note that there are no Python nodes where you can plug in any arbitrary data from elsewhere in Blender and expect the Compositor to know what to do with it. This is why I'm asking you these questions (which clearly need a mock-up).

That said, maybe a mock-up isn't necessary. The workflow you're looking for, can already be achieved in Blender with linked scenes and the Compositor:

  1. Have one scene, let's call it the Master . This is where you do your 3D work and your primary render (transparent background).
  2. Create a new blank scene with copied render settings. Let's call this one New Background .
  3. Go back to your Master scene, select your camera, and link it to the New Background scene. This ensures that you get the same camera movement in both scenes.
  4. In New Background , set up your World material with whatever HDRI or sky texture you want. This is also where you can rotate the HDRI as you'd like.
  5. Then, in the Compositor (I prefer to do this from another new scene, but you don't have to), pull in Render Layers from both Master and New Background . Then mix them to taste.

So what you're looking for exists already in Blender.

3di Author

* @Jason van Gumster * Thanks Jason, that's the approach i'm already taking. I would say though that there are indeed compositor nodes which access other parts of Blender, the mask node, keyying screen, stabilize 2d, plane track deform, undistortion nodes would be a few examples.

Anyway, regardless of whether the nodes are coded in C, C++, or python. It is possible to access any data from anywhere within the blend file, especially in the case of C and C++ (sometimes the API doesnt expose certain things to python)

You could of course always use a handler to manually update any portion of the interface. In the below script I wrote the other day, I'm changing which view layer will render in a completely different scene.

def change_view_layer(vl): C = bpy.context #set active layer in this scene.

vl = C.scene.view_layers.get(vl)
C.window.view_layer = vl


scenes_to_update = {'Scene','shadow catcher'}

for scene in scenes_to_update:

    print('\n\n')
    #set active view layer in shadow catgcher scene
    for vls in bpy.data.scenes[scene].view_layers:

        if vls.name == vl:
            vls.use = True
            print(vls.name,'set to active in scene',scene)
        else:
            #print(vls.name,'disabled for this frame')
            vls.use = False


def my_handler(scene): f = scene.frame_current

frame_to_viewlayer = {'5m' : 3 , '3m' : 5,'4m' : 1}

try:
    for key,value in frame_to_viewlayer.items():
        if value == f:
            change_view_layer(str(key))
            print('view layer changed to',key)
except Exception as e:
    print(e)




def register(): bpy.app.handlers.frame_change_post.append(my_handler)

If I wanted to update any value in any comp node then it's just a case of something like this:

bpy.data.scenes["Scene"].node_tree.nodes["Composite"].inputs[1].default_value = 0.2