Proper Alembic Support

This is re-post from devtalk, unsure about re-posting rules but wanted to start a discussion.

Hia,

We’re a small VFX house (feature films), using alembic (and slowly toying with USD) for most of our work. We’re primarily a Houdini place but have introduced Blender for modeling and the VSE for contextual review of shots (thanks to the cache updates!) and now we’ve been rendering with Cycles on a few shots.

We’ve hit some roadblocks/showstoppers (in both 2.79b and latest 2.8 builds) in regards to Blenders Alembic implementation.

I hope to provide some meaningful feedback from production land and propose some suggestions for how to elevate some of our current issues. I welcome any pointers on how to circumvent the problems or discussions about best practice in terms of Blender and Alembic in its current form.

Alembic is a fantastic interchange format, with support for some crucial production features, like streaming data, data-deduplication ,instancing, hierarchy, custom attributes to name a few. I want to highlight two things in particular that we’re having issues with:

Poor streaming data support

Currently Blender only streams geometry if its deforming, but not if its a packed/instanced set of geometry, then Blender unpacks and loads all of the geometry data and stores it in the Blend file.

I can have a single ABC file with a fairly hefty point cloud that I instance an object on and it will only save the instance and the point cloud in a small .ABC file. Blender will unpack and copy all of the objects around into memory, and break the file link to the .ABC file (as in I would have to re-import the entire data-set for a new version of the cache). Which makes working with cache versioning a proper PITA. The point is to stream and load the data of disk OR import and unpack.

Our issues are, file size ballooning and duplication of data. Poor version/file control.

Suggestions:Alembic render primitive, fuzzy filtering of ABC file in Alembic modifier. Let the user chose if you want to unpack the alembic into blender for editing. Or just reference the file if it’s for static reference, render or sim work.

Poor attribute support

Alembic supports arbitrary data stored on points, vertices, primitives etc as Floats, Vectors and Strings. A lot of additional data concerning caches are stored here, including things like rest positions, velocities, age and other useful data that can be used in contexts like shading and simulation work.

Blender’s ABC support is limited to a single Color channel (which is actually clipped to 0-1!) and a single UV float2 stored on vertices. Inside of Blender I can have multiple layers of vertex colors but it only reads the first one inside an alembic file. Blender supports multiple layers internally but when I export a Blender native object with multiple UV and Col maps no other application reads them. Neither Houdini nor Gaffer can read/see that data. However blender will see it once you re-import the file into Blender. I suspect Blender isn’t following the specification of the format here.

Issue: Can’t do any production rendering of complex nature since we’re severely limited by the number of data channels we can import. No motion blur on deforming meshes based on velocity attributes! (showstopper, but it CAN render motion blur if there are only hierarchal transformation from the ABC file, just not mesh deforms)

Suggestion: please remove the clipping of data in the Col/Cd attribute on import, copy the other vertex data from the ABC file into other Col vertex maps since blender already supports multiple data types on points. The latter is a hack, but could be easier to implement.

TLDR, these are stopping us from using Blender/Cycles;

Can’t stream alembic archives to the full specification (no instancing without unpacking, no file-read only)

Can’t import attributes beyond Cd and UV (which are also crippled)

With USD looming around the corner these are issues that would need to be tackled before even thinking about USD in Blender.

We’re loving using blender for VFX and are well impressed with 2.8 and the talented developers pushing the envelope and appreciate everyones efforts. I hope this post comes across as informative and not to off-putting, but ABC is an industry standard and full support would not only benefit VFX artists working in other applications but also Blender artists directly.

It’s a good format.