Mr.Tapll - A new method of linking

@scott:

• "How does this handle cases where a saved version is bad?"

If there is a version before this, then the current one is deleted and the previous one restored as the master file. If not, then with a warning. I suspect something like, "This file isn't working and may be corrupted."

• "For example, you create a rig and it was approved. However, we soon find out that there's a serious problem with the rig and fixing it will take 2 weeks, but you only have a week until the affected shots need to be approved?"

The difference between a problem and a predicament is that a problem has a solution, whereas a predicament does not.

What you’ve outlined here is a predicament.

With no working rig, and no time to fix it, even the most robust and beautifully designed production pipeline won’t provide a solution.

Your producer might look for the least ugly workaround and call that a solution, but it really isn’t, however it’s the best you can do under the circumstances.

• “Another case would be that shots 1, 2, and 3 have been approved and we don't want the rig to be touched after that, so we go and lock the rig version for those shots. But all of the other shots in production can use the latest one.”

In this example you would simply have two different rigs used in two different sets of shots. That would mean that Mr.Tapll has two rig master files (which look almost exactly the same, but actually are different. Rig master file A (for shots 1, 2 & 3) and Rig master file B (for all other shots). You then post a note for anyone opening shots 1, 2 & 3 that says: Only use Rig master file A with this shot.

• " I ask, because these are a couple common and requested cases that I've seen in production as a pipeline TD. "

Thanks for asking, Scott. I don’t think my proposal helps you at all, however I hope my answers here have. Cheers.

@matbrady1: I know one way that'll help both cases would be to back the system by a database, so we can track if a version is approved or not. Also, track the rig version that was used to create a shot. That way, if the rig is bad, we can mark it as bad, collect the affected shots, and then swap to a previous good rig and do work from there.

@lozo.vfx:

Lozo, thank you so much for your wonderfully positive response! It deserves a visual reward (see below). :)

If you are interested in seeing where this idea evolves then I think you just helped it along! I really like your idea of having a visual representation of a pipeline, particularly to see the impact of a change throughout a pipeline of linked assets. The node editor might be a terrific place to see a “Pipeline Map” of connections. Something that could be made possible if the functionality of a pipeline was designed from the linking level upwards. Even better would be the use of filters to only see the approved assets, or (what you were addressing) show only the connections of a highlighted asset.

This also mirrors what Scott mentioned (above). If a visual map of connections was possible it would allow someone to see very easily how far-reaching a change to an asset might be.

The below image has now been added to the proposal. Thanks!

@matbrady1: Absolutely wonderful it would be! Add prevision time-per-task and real-time-per-task as well as number of retakes, and you would also have a powerful bidding database building up !

@lozo.vfx:

Fantastic.

While we're both here, and while this proposal is still on the main page (but only just) can you explain more of what you mean. Does "Prevision time-per-task" and "Real-time-per-task", mean the time allocated to a task vs the time the task actually took?

And what do you mean when you say, "Retakes"? Is this a re-edit of the main movie, or is it two versions of the same shot?

@matbrady1: Sure, excuse my encrypted terminology!
Yes, your interpretation of Prevision and Real-time is exact. Time estimation vs actual time it took, and retakes do mean versions but versions that are directly linked to revisions process reviews, so how many iterations it has gone back and forth.

@Brecht Van Lommel:

Brecht, thank you for your response, especially as you’re one of the devs in the 2.8 Code Quest right now. Thank you especially for the work you’re doing over there. We’re wishing you the best of luck. :)

This proposal talks more to the idea of wholistic design. I believe the benefits of an integrated pipeline within Blender would be far-reaching, but before we can even get to that point we need to step back to the very core of where an integrated pipeline stems from and that is Linking. It is central.

I hope that when the time comes for Blender’s linking system to be upgraded that a wholistic approach can be adopted that includes its impact on a studio pipeline. Or at the very least, that your work can leave room for such adoption in the future. Many of the functions within Blender Cloud’s Attract wouldn’t need to be duplicated if a pipeline was integrated into the design from the stage of Linking upwards. Scalability would also be simpler.

I’ll stop trying to sell the idea to you now and simply say that this proposal is here for you and your team to read and discuss when the time comes. It’s the sole reason why I made it, so thanks for stopping by. :)

@S J Bennett:

S.J.Bennett, thanks for your kind words. I appreciate that. :)

As for your question: Yes and no. 

The yes part: There does exist robust, file-based linking in Maya. (They call linking, “referencing”). Their referencing is simple, and can also use a Reference Editor that's part of the program which is quite handy.

The no part: Even with Maya’s referencing it’s simply a feature of a package and not integrated into or purposefully designed into any workflow. It just exists. You can reference/link files, but there’s no overall purpose to it apart from the act of linking itself. The files are agnostic in terms of what they do and where they go. No-one owns them, and they don’t belong anywhere, or exist as part of a system of order that could benefit a production.

The important part: It seems that a robust system of linking is what separates software between studio-grade and not studio-grade, and this may be the key reason why we haven’t seen Blender be as universally adopted by the industry as what we might have expected, especially given its attractive price tag.

@matbrady1: Another big impediment to bigger studio uptake is that it's not really possible to get help outside of community channels. There's no premium helpdesk service. I suspect that's a big impediment to professional adoption too.

Anyway, back on topic. I don't think there's much doubt even from the die-hardest corners of the Blenderverse that other 3D software (including Maya) is ahead of Blender when it comes to using data in other files. The most infamous example of Blender's linking limitations is having to copy or symlink a rig file multiple times in order to include it in a scene more than once if you want the same rig doing different animations. I think there's some action on that in the next week or two ("static overrides" if I'm not mistaken).

Your proposal is a good idea, no doubt! Even just treating it as an end point, better flexibility with linking and instancing would be great to see. If Blender Animation Studio manages to sell the Agent 327 feature film, it would be superb to see a collaboration-driven system like this come out of it.

@S J Bennett:

Thanks again for a wonderful response, SJ. It's very encouraging. :)

@matbrady1: No worries! Out of curiosity, are you at a studio right now yourself, do you work in the industry.. ?

@S J Bennett:

I'm a professional concept artist, storyboard artist and 2d/3d animator who's been working in the industry on and off for ...a long time. Games, tv, movies, blahblah. I've never taken my career seriously, but it's provided me a wonderful outlet for my creativity for the entire time.

I'm not currently in a studio however because I've saved up enough money to take a year off. I'm into week 9. I have 41 weeks to go. (That's why my proposal had a few more bells and whistles than the other ones. I had the time to properly give to it). Here's an old showreel if you were interested: 2014

@Francesco Siddi: Hi Francesco! Thanks! :)

The closest practical example would be how Maya references files.

@matbrady1: I see. To get some more context, are you familiar how the linking system in Blender works?

@Francesco Siddi: Context is important. Thanks for asking about this, Francesco.

I am “somewhat” familiar with how linking works in Blender, however I’m more familiar with how linking doesn’t work in Blender. I’ll explain:

Short version: I made a complex rig which I linked to a new file to animate with. Upon linking the rig broke completely. After research I realised how to make the rig work, but it involved duplicating bones with copy restraints, and seemed like a kludge rather than a pipeline-friendly method of linking files: hence my proposal.

Long version: The problem on Stack Exchange that inspired the Mr.Tapll proposal

PS. The aim of my proposal is to convince the devs (and others) how much linking is at the very heart of a Production Pipeline (PP). Since each PP file needs to be as light as possible, and yet deal with multiple artist's work, means that linking is very literally what connects everything together, and as such deserves (in my opinion) top priority. To design a PP from the standpoint with its method of linking as its central design principle is what my proposal attempts to loosely outline.

This point (above) is so important, in fact, that I added an extra slide in my proposal to explain it. (Below) Thanks for the inspiration to clarify this. :)

I don't think file based linking instead of group based linking solves the problems you encountered. What you encountered seem to be limitations of the old proxy (override) system, and in Blender 2.8 that is being rewritten in a way that likely solves the limitations you found.

We are looking at how to unify groups and collections (the replacement for layers in 2.8) more, in a way that hopefully will allow any collection and its child collections to be linked into another file. The result of that would be effectively be the same as linking only objects from specified layers in 2.7.

@Brecht Van Lommel:

Thanks for your response Brecht. I hope you're right, and if so, then that's GREAT news! I couldn't be looking forward to 2.8 more now. :)

And, if Francesco is still reading, I didn’t realise you were also a dev on Blender Cloud’s Attract. (Well done for being the first to respond to my proposal by the way, and so kindly as well).

The more I’ve looked into Attract the more I’ve become impressed by it, and the way in which Blender’s video editing directly informs the management software and vice versa is the type of integrated design I was outlining with both my Mr.Tapll proposal and my comment to Brecht (below).

Francesco, I must say that Attract and Flamenco are aesthetically beautiful . I’m a sucker for simple & gorgeous design, so well done there.

I’m very excited by what all the devs are producing and I speak for everyone standing on the outside looking in, we’re wishing you guys all the very best of luck. Cheers. :)

@matbrady1: Thank you for the nice words! It's very good to see active discussion around this topic. You proposal outlines several principles of a good pipeline so I'm sure it will reinforce some decisions as Blender develops.

@Caetano Veyssières: - Fixed.