NPR nodes proposal for Cycles 2.8x

Feature request proposal: set of Non-Photo-Realistic nodes for cycles

The following proposal consists of a set of tools aimed at giving Cycles additional Non-Photo-Realitic (NPR for short), in this document the proposal will be explained (and compared to existing tools and another proposal) and in some extend demonstrated with some working proof of concept node setups.

A working proof of concept node setup can be downloaded here (UPDATED!!!!):

blendswap.com/blends/view/91247

1. Difference with NPR Uber Shader (Wasili Novratidis and Marius Oberholster)

Before starting, I think it’s necessary to explain why and how this proposal is different to the one made by Marius Oberholster and Wasillis Novratidis, both in focus and usage.

blender.community/c/rightclickselect/11bbbc/npr-in-2-8

Marius proposal mockup: My proposal Mockup:

Their proposal is about a single Ubershader NPR node containing everything with heavy focus on realtime use with Eevee.

In the other hand, my proposal has 2 different elements (minimal setup): a Flat shader to render flat colors in an integrated way with the rest of available shaders. And a texture node where the shading should be stored, like a shadow map buffer, this way further customization of the NPR look should be possible by the flexibility of image nodes and converters already available for Cycles. Maybe this “shading texture node” could be also be used with geometry modifiers (for example: emitting particles only on projected shadows falling on certain objects).

2. Reason d’être: NPR and photorealism integration

So, why this proposal? What’s wrong with Cycles and NPR? Simple: Until now (Blender 2.79) you either choose NPR only in old Blender Internal Renderer (BI for short), or realism with Cycles and to some extend BI too.

( blendernews.org/xe/Feature_Articles/1274 )

And if for some reason you want to make a piece of work with both kind of element together (for example “Who framed Rogger Rabit”), you could only use BI. Compositing is very useful tool, but in many cases can make things unnecessarily difficult (for example the cartoon character goes behind a refractive/reflective object).

"Who framed Rogger Rabbit", a classic movie with mixed reality.

This is what happens when you try to customise the Toon shader to look cellshaded, the environment breaks.

Example of mixed artistic element made in Cycles.

There also the concepts of “Versatility - Ease of Use - Artistic Freedom – Speed “.

o Issues with Toon BSDF vs true NPR

Toon BSDF is not bad but hard to use and doesn't give the exact results you may want.

The NPR tools in Cycles are too limited (only Toon BSDF Shader), and they are require many tweaks depending on the look you want to achieve, and by then other shaders tend to break. Worse: even if this look is achieved, there is no proper control on the color themselves. And there is the “Terminator Bug” too. In this case the “Toon Shader” proves not being versatile, it looks easy to use but has hidden complexity, it’s kind of fast compared with other shaders maybe, but it certainly doesn’t provide artistic freedom.

I think the versatility part is well explained, the ease of use is debatable, speed part is quite technical, so let’s dig into artistic freedom:

• Color Theory

While Color Theory is common for all kind of graphic related art (even realistic rendering), for drawings, painting, and thus NPR rendering it is vital: artist NEED to choose their colors, either as fixed color palettes, picture or movement composition based or enhanced with color, etc., even if it is not physically accurate. I really like this crash course on color theory, in this image color harmony, made by this artist. trisketched.deviantart.com/art/HOW-TO-MAKE-YOUR-ART-LOOK-NICE-Color-Harmony-423507240 In BI it is possible to do that in many different ways (diffuse, specular, color ramps, mixing nodes, normal manipulations, even shadows, etc.) per object/material. Cycles’s Toon shader? Not so much, especially if trying to do it within a realistic looking environment.

• Another issue of Toon shaders in general: “Shading Clamp”

Not a specific problem of Cycles, but 3D Toon shaders in general, most of them act in a realistic/logical of “light just add up”, where if you have more than 1 light affecting an surface, a new more lighted up color will pop up, which makes sense. But in Cartoons and Japanese animation (Anime for short), they use Cellshading which actually simplifies light into a concept I’ll be calling “Shade Clamping”, where all kind of light sources in a scene (direct light, indirect lights, environment lights, etc.) are concentrated and represented in only 2 or 3 colors, this can be done in 3D by using a Color Ramp Converter and force clamp the result.

I admit this technique and concept that only works when all light sources are of the same color (in general white), but again this is an example of things possible in BI but not in Cycles. The following proposal enables this.

3. Proposal

As mentioned earlier, the proposal is comprised of several elements:

o Flat Shader

This shader purpose is to get a desired color to the final image in a way that no physical interference can modify, unless this interference’s position occludes the object from the camera (example refraction, volumetric material, mist, etc.) or it’s a secondary representation of this color (example: reflections). An additional feature is the way this shader interacts with the surrounding objects/shaders, the 2 cases that come to mind are: this shaders acts as a “Diffuse shader” when ligh bounces off of it, and the second case is that light simple doesn’t bounce off it (like a black body). Here is an example of a working proof of concept; the only missing feature is a way to ascertain the input color and the rendered color are exactly the same, but this is pretty close.

o Inverted Hull Shader

This shader has a very limited and specific use and its very tricky conceptually speaking, and that is for the “Inverted Hull outline technique”, an old modeling trick to get a 3D outline of a 3D object to make cartoons, it is similar to Freestyle, but can work in realtime giving similar results, also this technique works in reflections, refractions and transparencies as shown in the proof of concept, although there are 2 examples, mostly because of some lighting issues depending on how these nodes are arranged. The ideal case scenario is that the shader can be reflected as shown in picture A, but without affecting the lighting of the inner object as in B in case the inner object has a standard shader (for example Diffuse BSDF, if the inner object has Glossy and Glass shaders it should ignore the outliner object).

o Classic OpenGL Shading Texture

The most important part of this proposal, but also the most complicated, and the one with most incomplete proof of concept, here are many unresolved questions and my lack of understanding may make me talk about things that can’t be actually done, beware. In the picture I show 1 of 2 possible versions:

• Single node • Multiple Nodes •---- Diffuse •---- Specular •---- Shadow and AO/SSAO

The picture has a single huge node that may be more practical and personally like it more, this one could be divided in diffuse, specular and shadows, and while it may go better along with the nodal concept of cycles, it could lead to unnecessary extra work for artists.

The idea is that the bulk of the NPR rendering and shader design happen here, in the texture and color nodes level of the material, in order to get access to all mix and blend modes available. It’s because of this that most nodes in and out of this proposal are yellow. Because of the above, the light sources would be limited to vector like lights, like the ones of old BI (Sun, Point, Hemi, Spot), thus mesh lights (Emission Shader) would be entirely discarded in this stage at least. But since all of this is texture based, it can be later plugged into another shader besides “Flat Shader” if the artist wants to mix things up. I have 2 theories of implementation: The first one is by making a small renderer within Cycles; this is the reason I call this node “Classic OpenGL shading Texture Node” (needs better name, I know). So far, something like this is what both my proposal and the other one have achieved to some degree with the proof of concept setups, they can simulate Diffuse and Specular shading along NormalMaps, but limited are to a single white light source.

di3d.cl/wp-content/uploads/2018/02/03-3-proof-of-concept-classic-gl-texture.jpg

There are some issues with the light source angle (driven by a empty driver) but in general works. What is missing is an implementation of Shadows and AO. But I think it could be implemented in a similar way of shadow maps, at the very least I know that it is possible to project bitmaps as shown in this node setup:

blender.stackexchange.com/questions/70717/projecting-texture-onto-surface-from-point

It seems a function to detect if a single ray had made a collision twice is missing. The second implementation is to “simply bake in realtime the viewport graphics” (the content in the GPU that shows in the 3D viewport) into the texture, at least as an idea sounds simple, not sure how that would happen code-wise (sounds kinda hackish?). The reason I think this may be feasible is because this graphical information is the GPU memory as a buffer, or at least Eevee seems to still use buffer shadow maps, which could be re-used in Cycles.

Example using very low resolution shadow map buffer.

• Single Light Source – a way to make Light Groups

At the botom of the node, as you see in the mock-up picture, there is a socket in the nodes to choose a single light source, by default this node setup shall get light vectors from all light source objects (but not mesh lights), but with this we can have further control on how individual elements are being shaded, since this proposal already uses an independent light system I think this shouldn’t be a problem, and it’s a great feature too.

• AO or SSAO

This last point, while not critical it would be very beneficial, since AO is fairly used in NPR too for different reasons, here a little example as a way to get intersection outline.
This AO should be independent to Cycles’s AO, in fact even SSAO may work, its an interesting feature that should be considered, since Cycles’s AO can’t be used without compositing, since it’s a shader.

o Versus NPR Uber Shader

• Advantages of this proposal

• All shading design within texture nodes -> RGB Data worklow ->maximum artistic freedom. • After finishing the deisgning above, it can be further expanded/experimented with already available shaders besides proposed Flat Shader, highly versatile. • If it works as imagined, could give some sort of light group functionalities. • If it works as imagined, this texture node could be used with modifiers. Interesting modeling feature if you ask me. • Potencially noise-less in final render, if using Flatshader of course.

• Disadvantages of this proposal

• No access to shader data (green nodes) -> Only Object lights (Sun, Spot, Point Hemi) and not Mesh Lights (Emision shader lighting). • Slightly harder to use because of more nodes implied. • Not sure if actually possible? • Noise would pop up again if using normal shaders, but this is expected.

• Advantages of NPR UberShader

• Single Node, easier imposible. • Compatiblity with Mesh lights (EmissionShader). • Maybe easier to develop?

• Disadvantages of NPR UberShader

• Limited blending/mixing/masking possibilities would not enable artist to do much more than its design would enable them to. • If Cycles can’t work with light groups, neither will this shader. • Heavy focus only on realtime Eevee worries me, what about Cycles and the possible noise made out by realistic shading turned into NPR?

4. Showcase of actual functions (this proposal)

o What can it do now

• White single light (maybe more but hard) Classic openGL shader • Use Ramps to modify color output and design material • Compare with drawings from artist • Using “ligh” info to modify realistic shaders or together with them

o What it currently can’t do VS BI

• Automatic multi-lights • Integrated Ramp • Shadow maps • Ao / SSAO