VSE: "Compositor Send," effect strip proposal
With the VSE now being improved upon and talks about finally improving the compositor as well, I figured now was as good of a time as any for a VSE proposal.
My proposal is to add a new effect strip called "Compositor Send," or just 'Composite." The function of this strip is to send all the video below it (like an adjustment layer) to the compositor as a "video in," node. That video can then be manipulated in the compositor, before being sent back to the VSE to be displayed.
Alternatively, a checkbox in the video clip properties could be added which would just send the clip to the compositor in the same way. The same checkbox could be added to the standard adjustment layer and it would function the same as above, giving more flexibility and avoiding a cluttered timeline.
A small dropdown box would need to be added to the compositor to switch between "VSE strip," and "Scene composite." The former would show the node-tree of the selected clip, and the latter being the traditional compositor.
Not only would this allow a lot more flexibility in using the VSE for visual effects, but it would also unify the effects system since all (or at least most) effects would go through the compositor, making features and performance improvements carry across both systems.
2 Comments
I like this idea a lot except for the fact that I find it too restrictive and only imagined for the compositor. This should be imagined more broadly in order not to be locked in the compositor for future development and miss out on all the possible 3D effects we could get otherwise.
The best solution IMO would be to create 2 new elements: a new datablock and a new node to access that new datablock. I'll call them "Sequencer Strip datablock" and "Sequencer Strip node".
Let me explain:
Any VSE strip or metastrip could have a button that activates that strip as a datablock. Then, in any node editor (be it Material, Geometry Nodes or Compositor), you could use the new "Sequencer Strip" node, in which you would have a dropdown menu to select one of the strip datablocks you activated (requiring activation is only to prevent getting a long list of all the strips in your edit). This new node would behave like a "Virtual" Image Sequence node, rendering frames from the VSE strip with all its retiming information, cuts (for metastrips) and what have you. Basically it would render the VSE as if all other strips were muted.
Then we don't need to add a dedicated interface in node editors as you proposed, since we can now access strips from anywhere in any node editor using that new node. Or rather, we are now faced with a new choice:
Either we set a development milestone here and users can just put a Scene Strip on top of the original strip, which would result in long lists of Scenes and cluttered VSE timelines but would be less ambitious for developers
Or, expanding on your more user-friendly idea of a seamless back-to-back workflow, we create a new "Strip Sub-Scene" type, which is basically a scene (with 3D and compositor), except it would not be listed with the main scenes (and wouldn't itself have a VSE, of course). Rather, it would be a child of the scene it was created in, with exclusive access to the strip datablock it was created for. This sub-scene could then render directly back into the original VSE strip (which should then have a checkbox to switch between original output and sub-scene output).
However, this 2nd apporach, while being more user-friendly and less cluttery, seems so ambitious it may take many years before devs would have the time to consider it, especially given the (understandable) pace of progress of the VSE.
The first approach seems a lot more feasible in a reasonable time frame and would not prevent future development of a more user-friendly solution. It could just open the door to complex 2D and 3D effects in the VSE, no longer requiring intermediate render exports that fill our hard drives and take longer to re-edit after the fact.
This fork: blender.community/c/graphicall/hpbbbc (there are also builds for Mac and Linux) adds a sequencer channel node. If you select channel 0 it'll input all channels. For returning to the VSE you'll need to have the node editor in a different scene, and add a scene strip using these settings: devtalk.blender.org/t/design-proposal-scene-strip-properties/15415 This stuff is experimental. The current limitations in speed and workflow are ex. the lack of GPU in both compositor and sequencer and the fact that the sequencer can't reference to compositor output from the same scene as the sequence is in. youtube.com/watch