asking for the community opinion: motion planning extension ?
Good morning everyone. My name is Steve Tonneau, and I am a french researcher in robotics (and also a bit in Computer Graphics).
My research topic aims at procedurally computing motions for legged characters (such as humanoid robots): Given a start and a goal posture, automatically compute a plausible motion for a character that connects the two postures, while taking into account collision avoidance constraints, as well as physics constraints.
In particular, the method is especially interesting because it can compute the contact locations that are required to produce the motion.
You can see here some examples motions that have been achieved with my methods in robotics and computer graphics: Everything is entirely computed automatically , unless otherwise stated
Additionally, I also used my method to edit some motion capture clips:
More demos, and explanations of the method, can be found here:
To be honest, I though this had little interest for the artist community, because as you can see (expect for the motion capture parts), the motions are not reaching the level of quality that can be found nowadays in games etc...
However, a friend of mine suggested that a method able to generate these motions would be of great help for animators, because it would provide them with an "initial guess" with which they could work and produce high quality motions, rather than starting from scratch their animations.
So my question is: is the artist community interested in such a method ? I'd be willing to dedicate some time to produce a blender plugin able to do so, but as it would require a lot of work I only want to do it if it makes sense.
I hope I found the good place to post, and am willing to answer to your questions.
Thanks a lot for your attention !
Steve
11 Comments
Animation automation is inevitably going to happen anyway. The only question is "which method is the most suited for CGI ?"
I'm no expert and I wouldn't want to throw down a project that could be more promising than I expected but seeing that your work is aimed at moving real robots, I don't think it's the best choice. Animators don't care whether a character would keep balance and not fall if it was in the real world. All they care about is making the motion look believable, human-looking and smooth. For that, deep learning seems to be the most suited method to me, since it would use a large number or reference from motion capture.
This neural network is the best I've seen so far.
It could be used in games but also in movie-making to either make a layout animation more detailed or even make a base for the real animation.
Hi. Actually I know quite well this work (and am a co author of Taku Komura on the motion capture based project I show in the post), but you need to understand something about neural networks: to make it work you need data. That is, you need thousands of examples of motions similar to the ones you want to generate for your character.
I am not saying that applying NN approach would not be relevant to blender, and actually I think it is. But the thing with Daniel's work and and neural network based approach is that it is not able to generalize to arbitrary terrains and characters. Look at the second video of my post. NN based approaches are not able to generate this kind of interaction. To be able to generate this kind of motion with NN you would need thousands of examples of people performing similar tasks. So for most artists this is not possible, nor desirable. If you want to create your own monster type character with a specific locomotion behavior, or if you want the character to interact in new kind of environments, then NN won't be able to provide a solution to you.
Now let me be clear, if you just want a human like character walking around, this NN based network would perform amazingly, although you won't be able to edit your own locomotion style. So I think those are different applications: on one hand a generic method for any environment, on the second hand a NN based method that works for humans and for walking only.
Coming back to my method, in fact dynamics is a way of making the motions plausible, so it is relevant in my opinion. Still, I don't see my results as final, but more as a rough sketch upon which the artist could be more productive. After all, in the end the artist has to have the final word
@pro: Thanks for all this info. I didn't imply your approach doesn't produce plausible results. It definitely does since it works in the real world, but it looks robotic and a bit strange (on a human perspective) and all the work you put in giving this extra precision to keep balance on a physical robot is kind of wasted in a field where it doesn't have to be precise to the millimeter. Also, some movements like jumps and fast accelerations (that muscles can produce better than robotic articulations) can be captured in motion capture but with robots, you'd have to get to the level of Boston Dynamics.
I definitely don't want to discourage you from doing it, I'm sure it would be awesome for at least animating CG robots. I just want to warn you that maybe not many people will use it, even as a base to work from. If the key poses (the most important poses in an animation) aren't like you want them, it's best to start from scratch.
But I'll ask some animation students and professionals to tell you what they think of the idea and whether they would use it or not.
@Caetano Veyssières:
Hi, thanks for the reply. I definitely agree with you that the motions are robotic. My question is: as an animator, would you find it easier to make a good looking animation based on a given robotic motion, or would you rather do it from scratch ?
If the former stands, then I think the method is interesting. Otherwise i agree that it is useless
@pro: Well that depends.. in how you would apply a NN, Doing work 'before' reduces complexity and speed usually a lot. EA start with a simple feedback version based on fuzzy logic. Fuzzy logic will correct itself (that's the essence of a thermostats logic). Width fuzzy logic generated data, one might start of to go to NN. Where a NN can find the most ideal solutions (where penalties can be given by movement energy usage, to get more natural walk)
There might be some interest with artists, but I'm not sure how much. If it's a lot of work I would start with a super minimal version to gauge interest. Something that would give a very basic demo of possible features and would require little dev time. Then post to BlenderArtists, BlenderNation, and the other Blender groups to get feedback.
And possibly a stupid question, but have you notified any of the Blender Robotics groups (e.g. openrobots.org ) about this project?
Hi, indeed, I was thinking of first developing a simple contact generator: place the root of your character anywhere in the environment, and ask something like 'create a contact between the hand and the table'.
Not a stupid question, I did not even know there was a blender robotics groups (which is crazy conseidring I am in the lab hosting the page :D )!
walk automation, following (or having a goal) of a spline would be quite nice to have in blender. It takes a lots of time to that manual. Besides spline path's perhaps other options would be nice as well, walk but dont hit walls, evade / flock / follow / (animal group movements). It be great if for example people walking on streets could be scripted walking in straight lines (don't hitting each other) and occasionally cross a street.
Not sure how you would code it, an addaptive neural net?, or projection of step in direction of movement ?.
Hi thanks for the answer! No my approach is not related to neural networks. It is based on exploration of the set of possible configurations and the law of physics. Neural network is probably not a good idea if you want to address any kind of character and any kind of environments (so far).
Following a spline (or any other polynom) would definitely be possible. Advanced flocking behaviors etc is definitevely not in the scope of what I plan for the moment (although it could be built upon)
@pro: Ok well physical walk behaviour is nice as wel, i though aabout neural or evolutional networks as they figure out such the behaviour themselves ( youtube.com/watch was done in blender). But it can be done directly physics as well (as a solution to balancing, which wouldnt require training, or generations.
I'm sure animators would like it, as your friend told you; some control over directions to walk, ea a walkpath would be great, but i dont know your coding skills. I'm more of a artist in Blender not a daily python coder, but as a developer for industrial machines i keep an eye on Blender and coding in general..
Regarding neural networks please take a look at my answer below. Regarding my coding skills, well I developed the framework demonstrated in the above videos, so at least I know how to do that :).