NLA - Cumulative Root Motion
More of an enhancement than an actual problem, the idea is that the root motion of an Action can be used to drive the motion of the model. In other words: in its current state, if a walking Action has the model move 1 meter on the x-axis then, at the end of the action, it will snap back to its position at the start of the Action. Having duplicates of this Action will result in it 'snapping back' multiple times.
If this function could be implemented, the new location of the model would be maintained. Therefore, three of the 'walk' Actions mentioned before would leave the model 3 meters away from its starting point.
An Action of a person walking, or a car driving, has to be manually lined up so the speed at which it moves is consistent with the person's steps, or the rotation of the tire. If this could be embedded in the Action, it would only have to be aligned once, and the motion could be duplicated and adjusted from then on.
To give a more specific example, it would be useful with more complex motions. A rabbit, when moving slowly, walks with a stop-start motion. It takes a step, then pauses, then steps. Even if the pause is minor, if it's missing, the rabbit appears to just be gliding. However, if every pause between steps has to be manually added every time and for every step, it would take a long time.
If the stop-start motion could be embedded into the Action, then only a couple of steps have to be animated and given motion to, and the Action's duplication in the NLE would do the rest. There can also be Actions designed to rotate the model as well as move it, or change its scale incrementally.
Of course, there are some cases where the Action should snap back, so there should be an option to enable or disable this feature for each track. I believe this feature should be 'disabled' by default.
Check comments for comments labeled "Further Reading", notes taken from related posts.
25 Comments
I think this is an interesting proposal, and it's also something I've been thinking about doing myself when I was working a lot with motion capture data.
Alignment of two animation strips is relatively easy to do by minimizing the distance metric described in Kovar & Gleicher 2002, section 3 .
The question is mostly how to integrate this with Actions in the NLA. An Action is just a collection of F-Curves, and Blender doesn't know which bones determine the global placement of the character. It would also be nice if a single action could be used multiple times, with different placements in the 3D world so that the character's motion is continuous.
How such changes would work, needs to be fleshed out here on RCS before the proposal can be given to any developer for implementation.
* @Sybren Stüvel * Hello Sybren, thanks for taking a look. I'm not a developer, just an amateur animator, so can't give much specific advice, but as to telling the software which bones define the motion, I've got a couple ideas.
One could simply be that Blender looks for a bone with a specific name. 'Root', or 'motion' perhaps. Then the user just renames the desired bone to 'root' and the software sees that and takes its motion.
Another suggestion, maybe more complicated, is there could be a setting in the menu on the right of the NLE, which allows you to eye-dropper or type the name. In this way perhaps it could even be animated to change? Though the point is to reduce the amount of animation needed, so it's not so necessary, if the root is set up properly.
If that's still a problem, a quick 'n' dirty solution could be to use an empty, make the object a child to it and animate the empty's motion. Since an empty has just one set of location and rotation data, Blender would reliably be able to know where to take the motion from.
Of course then you've got some bloat to deal with, but I think it's a relatively simple solution, maybe good for a proof-of-concept or for testing the 'accumulation' things GuiltyGhost mentions.
This probably won't help much, but here's the source the idea came from: YouTube . This software uses a bone called 'rootTransform', with motion baked into the animation.
* @Sybren Stüvel * Although, if the user could be able to animate between which bone is taken as the root of the action's motion, then this has some advantages of its own.
In the example of the rabbit's walk above, if there was the option to change which bone was the 'root' then, rather than having to manually create the the motion from the animation, via a root bone, the user would be able to just set the root between the "front foot" and "back foot" at the specific parts of the walk cycle, and take the motion directly from that.
It may need some adjustment, such as if the rabbit jumps fully into the air, as this process assumes the bone is 'locked' in one location, deriving the motion from its change in position relative to the parent of the rig. But it's a solid start, and quicker than than creating all the root motion from scratch.
[rabbit gif source: lowgif.com/ ]
I apologize for the delay. I've been talking with another animator (Brad Clark, from Rigging Dojo) about a similar idea. So the overall goal of Cumulative motion is the ability to put animations in sequence and for them to move through space. It's about solving the problem of a walk cycle resetting to the start position on the next cycle. Cumulative motion is intended to better solve: youtube.com/watch
Brad suggests to support an alignment offset as a strip property. It will apply a parent-like transform to a set of bones, usually the root bone or hips. In terms of cumulative motion, it's like baked cumulative motion. This can be used for all the situations in the OP. We can create operators to automatically calculate the alignment for a set of selected strips. Cycled strips would auto-align with itself.
What do you think @jacksavage.ztp?
* @GuiltyGhost * I actually found out just just a couple days ago about the process in the linked YT video, but slightly different . This process allows cumulative motion but still allows me to rotate the direction the character's walking in (buy placing the cursor beneath the armature and rotating it). I'm not sure if this or the other is the better way of doing it.
Anyway, as to Brad's suggestion, I'm sorry to say I didn't understand most of it. These following questions you can just ignore if they're stupid from lack of understanding. To clarify:
So this would be setting specific to a action's strip which allows you to specify “move 'this amount' every five seconds”? Like a speed value for a given axis. If this value could be animated it might be interesting, though I can't say it'd be useful often.
If my previous assumption is correct, would this then be a strictly linear offset? For the most part I think a constant, linear offset would work, in the case of walking. Just there's some specific types of animation which might not be able to use it.
Does this mean the motion has to be baked for you to get an updated preview of where the object would be?
I'm sorry... didn't really understand. It sounds good. You're saying that it'll auto-align with the action, but you said earlier it would be a setting, making it not automatic? So you must mean it's only automatic on a cycled strip? But if a strip already has a cycle modifier, then any root motion already in the cycle already loops.
Speaking of which, wouldn't a simple answer just to embed the root motion of walking into an action and then just add a cycle modifier to everything so it loops, making the motion cumulative?
Sorry for asking for these details. Don't feel any obligation to explain everything, since we've reached the limits of my current experience here and can't promise much insight even if you explain things.
I'm not sure what you mean by the baking aspect, but so long as actions (cumulative or otherwise) can still be easily manipulated without having to bake and re-bake motion to see an update of the object's position it would be good. I'd also wonder how changing direction would work, but if that's in hand, there's no need to explain to me why.
If what I've assumed is correct I think this would work. It might not work for every type of animation that would benefit from cumulative motion, but it would work for the majority animations. And perhaps I'm mostly wrong. I'm sorry there's not much constructive here.
* @jacksavage.ztp * (i'll post a video after this reply: Edit: youtube.com/watch )
Yes, it's a property per NLA strip. No, it does not specify a speed. You can think of it like adding a root bone to the character to move the entire animation with. But the problem with that solution is that NLA blending would still be broken. The alignment property applies the world-space offset such that blending works. This means you can properly blend a through-space Walk and Run.
Uhh technically it would be a linear transform- meaning the parent-child like transform behavior you see with normal bone or object parenting. So you can rotate an Walk cycle with this property.
Sorry, that was poor wording on my part. This offset is applied real-time, there is no baking. I meant "baking" as in, each NLA strip's alignment offset (alignment transform?) has the behavior of root motion (ability to translate and rotate the animation) without the cumulative part.
The core behavior is not automatic (and thus not cumulative). I meant that there can be operators to align strips (Walk into a Run) so you don't have to manually input those numbers. For cycled strips, the NLA system would have to automatically apply alignment offsets other wise a Walk cycle would still repeat back at the start in world space.
If you don't have to blend the Walk action, then yeah that works (just like the videos we've both shown). But as soon as you try to blend the Walk, it's no longer useful. In general, I think you'd need some sort of blending (Idle into Walk, Walk into Run, etc). The video will show why this case breaks.
Edit: Can you mention the cases where the strip alignment property isn't useful? Also quick videos like the one you showed are so much easier to understand than text. Previously, I had some wrong assumptions. Your solution doesn't require a 2nd nla strip. You could add a cyclic with-offset modifier to the root bone and cyclic no-offset for the rest of the bones. Then just make the walk cycle strip longer and it'll achieve the same result. You'll still run into alignment and blending issues when you want to place the walk elsewhere in world space.
* @jacksavage.ztp * Actually... for the given examples, why not use a Follow Path constraint with a cycled-with-offset animation (like you're doing in the video)? You can change the path to make the character turn. For non linear rabbit hops, it works out too if you copy the movement animation keys to the Follow Path's offset channel. Foot sliding would also be easily minimized once you get the speed right for the follow path constraint.
edit: youtube.com/watch
It's a complicated idea given that this is not a game engine where implementing root motion is relatively much simpler. Brute forcing a solution would be simple but likely extremely slow in more involved uses. The complication comes in optimizing it. I'll ignore those technical limitations and focus on the user end.
Accumulation property: Maybe there should be a "Accumulate" property for NLA tracks, strips, Actions, objects/bones, and finally the specific property (in most cases only location/rotation/scale would be used). In that order, if any previous element is untagged as Accumulative, then the property won't be either. That gives control to re-use clips without always having to be forced to work (against) with accumulative motion. The reason for specific property marking is that maybe not all transform properties should be accumulative. It also allows non transform properties to be accumulative (material animation? glowing effects? vfx in general?)
Accumulation Blending: If we have an accumulative Walk strip and a Turn strip above it, how should the Turn be blended? If the Turn is marked as Accumulative, then how should the blendmode affect the accumulation?
* @GuiltyGhost * As I said to Sybren, I'm not exactly a dev, and don't know any coding, but I'll give what advice I can think of from just thinking through logically. I'll split this into three as you did, for the reason you did.
1 ) This feature has more potential than simply making an object move around; more than I realized, too, now that you mention materials and other vfx. But in stripping it down to just this motion alone maybe it would simplify the issue enough so a solution can be created, whereafter other features can be added using the same concept. With multiple elements put together (such as settings for modifiers, if possible), it could be made to perform some functions similar to drivers, although properties would depend on the Actions, rather than on a 'parented' property.
Having an 'on/off' for every track/strip/action/etc. sounds useful, so long as it can be laid out in an easy way to understand. My guess is elements could be greyed out or made italic or something, if 'it' or a higher property is non-accumulative?
2 ) Accumulation Blending . My question is if there needs to be blending at all? Maybe I'm being small-minded, thinking only in terms of more realistic animation but (in the case of blending location and rotation transforms), either a model's animation has been made to factor in rotation, as well as location, or it hasn't.
Taking a forwards-walking animation and blending it with a turning animation... if blended so the rotation mixes with the location, it would leave the walking animation sliding across the floor, since the animation hasn't been made with rotation in mind.
Likewise, if a walking animation has been made with some kind of rotation in mind, then both the transformation and rotation are already factored into the same action; both are already a part of the root motion, and there doesn't need to be a second track to give it this.
Of course maybe there's times when the user just wants the model to slide, or have transformation which does not line up exactly with the animation, but in these cases, they can just not use this cumulative feature and animate it normally, as it's this kind of 'sliding' or 'non-animation-specific motion' this proposal is supposed to remove. Would a user need this function to bring a walking and turning animation together, since doing so would mean the animation is no longer specific to that motion being performed? (continued in [2.2])
But what about when there multiple Actions which aren't trying to animate the same things? Even in these cases, I don't see need for special blending features. From my understanding, I'd ask if the software couldn't just use whatever properties have been marked as being cumulative, and ignore everything else?
Let's say there's one Action, which has accumulative location and rotation, and another Action affecting, say, a bend modifier (also accumulative). Couldn't it just see that the loc/rot of one has been set, and the UV map of the other is set, and use them? If there's multiple instances of the same property trying to be cumulatively animated then, perhaps, whichever is put higher could be used, and other instances can be ignored.
2.2 ) I'm out of my depth for sure, but I'll just mention that trying to interpolate location and rotation together, without having the kind of per-frame motion you described, maybe would cause issues in the motion as in the video linked here. Notice it sliding to the right. It's subtle because I haven't given it much rotation, but with more speed and rotation it can become very obvious. YouTube
The sliding is caused by, as you know, interpolating a start and end location and rotation together, without actually having them 'work together'. Maybe this issue invalidates my comment above, about there not needing to be some kind of special blending. If creating many keyframes is the issue, can't it just be baked into the curves, such as the software in the linked video does?
I suppose this is a case where some kind of blending is needed. Even if the walking animation in the video is intended as being forwards-walking only, the slight rotation is barely noticeable; if it wasn't for the sliding, I doubt it could be easily seen at all. So perhaps there is some need to have self-generated keyframes blended from different transformations... unless this can just be synthesized externally, by means of animating the object itself, or an empty it's a child to.
3 ) Accumulation Cache Points . If a user sets up an animation with something moving, and realize they need it to be at a specific location at a specific time, but find it's not where it's supposed to be then... is that really a bad thing?
If they have someone stood by a tree and in five seconds want them stood next to a bench – but the walk animation only gives them time to get half way there - then it's realistic, even if it is problematic for the animator.
Forcing it to move further would just break the animation (make it slide across the floor). So the animator would either have to give the object more time, start it closer to its goal, or speed up the animation so it moves quicker and can reach the bench in time.
Either way I don't see this as an issue, more as a benefit, as it forces the transformation of the animation to be consistent, which is one of the main purposes of this feature. Of course, they can still just animate the object itself (outside of the Action), or its delta transformation, if they really need to.
This probably won't help much, but here's the source the idea came from: YouTube
* @jacksavage.ztp * For this case, I was thinking more of a discontinuous scene, where a character goes offscreen then reappears elsewhere, likely arbitrary. In this case, being able to override the character's accumulated transform prevents the need to counter animate. Especially if all your effort to counter animate the accumulative motion won't even be seen.
* @jacksavage.ztp * I should've split my points out too. Also, I don't expect you to answer the coding problems or think in terms of code implementation. But I do need you to answer the questions that involve user interactions and expectations when you use the Accumulation motion system.
* @jacksavage.ztp *
Maybe you're right and I'm over complicating things. Accumulative motion puts an inherent dependency on the animation its moving. To add another blended accumulative layer ontop would mean it's also inherently dependent on the underlying animation. When the lower layer gets modified, then the upper must be updated too. The layering system is meant to remove this sort of dependency. Since it doesn't, maybe accumulative motion shouldn't allow layering. If animators find a practical workflow where layering accumulative motion would make life easier, then we can always add support in the future.
* @jacksavage.ztp *
I'm not sure what you meant here. Did you mean the ability to create drivers that use the accumulated property? I believe that's free. The animation system is designed to evaluate before the Driver system does.
* @jacksavage.ztp *
Actually, the complication is less about what properties have accumulation and more the core idea itself: How do we support accumulative motion and keep Blender at interactive speeds.
Another complication comes in where we consider blending and viewport keyframing. But if .. we just don't support blending, then it's a lot more doable. The complication here is more of: You have a walk cycle. You're editing the Nth cycle thus the character is at an accumulated location. If you move the character in the 3D viewport and insert a keyframe on the root motion, then how do we properly insert a keyframe that leaves the character exactly where you keyed? To change one cycle is to change the rest and thus you're actually directly editing the net accumulation, not the local accumulation.
Edit: I don't think there is generally a solution for inserting a keyframe while in the 3D viewport. For transform properties, they're all relative to prior accumulations. If you have N accumulative Walk Forward cycles. Then you rotate the character 90 degrees to turn right at the end of Walk Forward (no affect on location) and try to keyframe. The problem is that now the character is walking in a circle (or square, technically) and will never get to that same location where you keyed. Effectively, your character will potentially snap to a completely different area in the world with none of the properties you just directly keyed.
* @GuiltyGhost * Thanks for the feedback, I'll look at the rest tomorrow when I'm fresh. For now I came up with something that feels like it could work.
Just have the closest keyframe before the start of the track (not after) as the initial position which the accumulative animation continues from. Keyframes outside the NLE will be overridden by the cumulative motion (and used as normal if they're not being overridden).
Then, if the object needs an arbitrary position, or needs anything else 'set', you'd just make a cut so it's taken as a new track (or shorten it and have a second one after, if needed). It makes sense it'd need a new cut, as it's breaking away from the 'baked' motion of the Action and changing to something new.
The new track then, like before, uses the closest-previous keyframe as the source to base itself off of, so putting keyframe input on the seam from the cut or before that will change its transformation, or whatever values the keyframe sets, to what is desired.
Works, but, having it ignore keyframes... I'm not sure. On the upside, it prevents a problem I was thinking about earlier, where having user-set keyframes for transformation would break the motion-consistency this feature is for. All the same, if the user wants to do that, shouldn't they have the choice to?
But then that does mean, if they want to set an arbitrary value, they would have to create extra keyframes, one frame behind where it's to change, of the same values as the keyframes at the start of the track so that the model doesn't interpolate towards the keyframed properties intended for the 'next' track. That extra step seems clunky to me,
I'd say making users have to counter-animate motion created by this feature is a must-not, it'd be awful to have to 'un animate' like that.
* @GuiltyGhost * I'm yet to actually use drivers in animation, but my understanding is that you use properties from one object to drive other properties of another, or multiple, objects. For instance, using the X and Y of an empty to change the hue and brightness of a light.
I simply meant that an object's properties, like the hue and brightness, could be changed by actions in the NLA. They could increase the values by 'one', then be looped, or mixed in with other actions that decrease it instead.
Though in this example, I feel like this gives the user less control than using a empty with drivers. It's just an example to explain what I mean: another way of incrementally altering properties, similar to how drivers can.
I'm not sure if this would be necessary, as I'm not sure if this offers any significant advantages over just using drivers, apart from being able to directly link certain properties to the particular actions of an object. Perhaps this itself has its uses.
What about footstep sound effects? Could this be used to automatically synchronize a sound, or other properties, with a specific frame of the action, which then reoccurs every time that specific frame does? Maybe this is out of the bounds of what this is capable of.
If you mean using an accumulative animation applied to an object being used as a driver, I don't see a reason why it shouldn't be able to work. I'm not sure if I answered your question; haven't got around to using drivers yet.
* @GuiltyGhost * I don't think I know enough about optimization to give a great answer, but if the issue is in the amount of keyframes that have to be processed and updated with every change, I'd ask if having it baked directly into curves without keyframes would be better?
Or, if it has to have keyframes, could they be automatically decimated before being processed, putting only the 'key' keyframes for creating the movement. In a way, it's the animation equivalent of using a vector image instead of a raster image. The user could have an 'accuracy' option perhaps, which decimates more or less keyframes/accuracy in movement, at the cost of processing expense. To put it another way, it would take stress away form processing it, as lower 'accuracies' gives it more and more bias towards simplifying the motion, ignoring subtle changes in preference for larger, linear changes.
Oh, I see the issue now, changing the root inside the action when in a later cycle. Well, the simple solution is to either lock the user from changing it if they're not at the start, or to automatically take them to the first cycle when they try to. They'd also have the option of making a cut if they want.
Actually, I had an experience on Blender earlier today which seems to already do this. I wanted to edit an action. The cycle started about 30 frames behind the "rendering range" I'd set, so I couldn't move the playhead back to see it. If I couldn't move my playhead there, I couldn't edit it.
Since that's already the case, doesn't that solve the issue of people trying to make changes to the Nth cycle? When you're tweaking, keyframes just don't appear on the Nth cycle, just the first, so aren't available to be animated.
Do you mean this could be an issue in general, or just an issue when changing keyframes inside the accumulative action itself?
It's the same compound effect as time travel to the past is portrayed as having. It would be hard for the user to get an exact result after Nth number of cycles, but it's not really meant to be used like that. It's meant for consistent changes across time - from start to end - rather than using it to get a specific outcome or value.
* @GuiltyGhost * The only time I can think of this blending being useful is a walk-forwards and turn animation together. The turn and walk would work together, allowing the turning to happen without the character's feet sliding across the floor (as in one of the YouTube links somewhere. While that would be useful, if it's much of a complication, it could just be left out. Besides, even I realize that might be very heavy to process since, as you said, it has to build the motion on a per-frame bases.
Edit: But even then, what you're left with is two conflicting animations. So the second action would just be for the root rotation without other animations, so what's really its use to be blended in when it can just be animated normally?
If it can be done, it might be useful to allow different tracks to change 'different' properties at the same time, but not the same property. But even so, since this is mostly meant for motion, it might not be needed at all.
My opinion is you've hit the nail on the head with that. It isn't exactly necessary for this function to be usable.
* @GuiltyGhost * Well, I've given suggestions towards what I've understood. If I've misunderstood parts, missed the point, or if there's more I can help out on, feel free to let me know. My inexperience at this leads me to be unsure what to direct my focus to; what's useful, what's not useful to be said or thought upon.
* @GuiltyGhost * With regards to what I can do to help: I don't mind putting time into finding answers or thinking things through, but I need to know what's useful to be thought about and what's not needed. If you've got any direction or anything I should look into, do let me know.
Further Reading
Info compiled from here .
The problem posed by this page is that any difference in foot position between actions turns into “ice skating” when transitioning between these in the NLA. This isn't relly reliant to this, but it does mention the idea of locking bones in position, by extracting its motoin.
In other words, if the foot bone was to move forwards and left then, if it was made the Locked bone, the whole model would move backward and right by the same amount to hold the foot in position, creating motion from the foot's movement.
Since this just takes the motion from a bone it doesn't exactly make it accumulative. I feel it does relate to the issue of extracting motion from the animation itself, however, as the OP states that it should “access the global position of the Locked bone and create the transform from there.”
A commenter goes on to add that something from the constraint system might be usable to create this, saying that using a bone constraint could cause “some instability in the whole system [but that] doing it as an object constraint would solve this since they are calculated before the bones.” However, if it's tied to an external object, does it then prevent the Lock from being change per-action, or from being changed multiple times within an action?
A commenter adds you should have the option to use a joint as a Lock, not just a bone as “you loose 50% of flexibility” with only bones. A further comment adds a 'coding' solution to doing this is to:
*[I think this means for each frame?]
This is all the information I found relivent in the origional post and the 6 pages of comments. If this is not useful, then let me know and I'll remove the notes and just keep the link. Likewise, if there's parts here which are unnecessary bloat, let me know, and I'll take them out.
* @jacksavage.ztp * Without a more concrete example, I don't think LockBones would fix the foot sliding problem. When you blend two actions that results in foot sliding, then pinning one action's foot position means you're ignoring the other action's foot animation. That seems like a new problem to fix.
Or, maybe you're talking more about blending a Walk with a Run. Let's say they blend together fine as is, one action directly atop the other and lined up. But if you try to blend the Run, at let's say the midpoint of the Walk, then the actions no longer line up and there is foot sliding. The solution here would be to add* NLA support to time-offset the Run animation so it lines up with the Walk's foot placement halfway through. (* you could actually just use Animated Strip Time with a cycled strip)
Alternatively, if it is acceptable to ignore the other action's foot animation, then maybe an NLA mask would be better? Bones within the mask are animated. So the Walk action mask would include the foot animation. The Run action mask would exclude it. So blending should no longer slide since nothing is blending with the feet.