ai-augmented animation to replace mechanical parts of animation
a) is anyone working on this b) i'll totally work on this c) english is not my first language
Ok so context: there's this youtube channel called two minute papers run by an ai researcher. He demoes amazing animations of ai/ml/dl models and advancements and whatnot. youtube.com/watch this will give you a rough idea. what the videographer did with textures, i want to do with everything two minute papers showcases. So let's say I take a few elements:
-
A model that accurately performs athletic movements with anatomically appropriate muscle distribution and flexion and all those other kinematic details.
-
A model that enables an any-legged creature to maintain balance regardless of the limb count or proportionality or symmetry
-
A model of indistinguishable-from-reality honey fluid properties
-
----GAN (I forget the name) which can produce random faces seamlessly (I think there's a site like this too, it's kinda memey)
-
StyleGAN which can take the aesthetic style of starry night and apply it to your Times Square selfie
...to create a music video in which the background is transitioning between real life, on-location shots of the artist's hometown or preferred set and the StyleGAN representation of those same backgrounds; in which the artist would walk forward on legs that change size and plop sloppily and fold onto themselves as though he were melting using the honey simulator's output tied to rig points on your characters legs; in which maybe there's a close up portrait scene that has the artist's eyes and mouth communicating the lyrics but the face around those features is changing since it is the output of the faceGAN, which in blender you've defined the output to be everywhere on your character's face but the excluded eyes and mouth and scaled appropriately.
blah blah blah, I want to take the output of these models and plug them in blender as tunable parameters. Like after I create my character and rig it, I want to be able to populate it with those pre-trained muscles from that algo. Then, instead of animating my character's walk from point a to point b using keyframes, I can use that balance algo to simply define ok "maintain balance" from point a to point b. And then if you as the artist want to illustrate someone walking downtrodden to someone's door and stumbling when tripping over an unseen lawn ornament, you can define some heavy, awkwardly balanced element within blender to appear momentarily on the character's back, unseen to the audience but just enough to apply the force and elicit a reaction from the balancing algorithm, such that it looks like a stumble. Then you appropriately place your lawn ornament asset where the stumble begins and instead of worrying about the natural physics of the jarring stop and recoil of a toe stub, or the desperate recovery of a forward fall, you just have the model produce what the character would do and you simply add (procedurally generated!) environment around the desired but much more difficult to animate behavior that is 'bipedal creature stumbling over lawn ornament.'
ps there are so many more applications like tearing simulations you could plug into the cloth physics so you can reproduce some proper hulkamania, or change the material to flesh so you can properly recreate see you at the party, Richter. During animation of a character tearing their clothes off or tearing their opponent's limbs off, you could map the output to the ever-changing position and condition of the shirt or limbs so that in those keyframes you don't have to animate it.
5 Comments
The team should also start creating another software with AI to compliment Blender
lol thanks guy, that's quite the implication. but this isn't weed thoughts. youtube.com/watch if you want an example of what i'm talking about. i'd just be adding more elements rather than cool motion-tracking textures
* @alexandrubenza * Jokes aside, I was thinking exactly at this one and I believe it's doable with the current state, just replace AI part with mocap and mix another software like After Effects for style transfer features. And flip fluid for simulation.
* @Ge Ro * perhaps i didn't explain clearly enough. what i'm talking about is having the style transfer in blender itself, not having to rely on after effects. all it would entail is literally using the code that's readily available from those papers the 'two minute papers' youtube channel showcases. the difference is the input, instead of being a highly controlled and refined dataset that the researchers used, would be the artist's model and the output would be the model+style transfer OR the model in the next frame if someone were working with keyframes and whatnot. the only catch would be that instead of having it as high-res as the researchers have it (they use cloud assets and really high powered computers), you'd have a more low res version.
but i don't think that's a problem, after all imho all an artist really needs sometimes is an idea and if they can run wild with that, the output of this thing will be enough. and the tech/model will catch up. our devices are getting more powerful by the month, solid state everything is on the horizon, and i just saw a video of someone able to run ML algos on a really low powered microprocessor. if anything, maybe blender's tech heads could kickstarter quick fab a purpose-specific external rendering device, like a little blender buddy of sorts lol, for the truly heavy renders
i'm pretty sure if i just take a blank canvas and make the background 1/3 green and 2/3 blue, with a horizontal dividing line, and a short brow line coming up from the green part, and having a green circle on top of the line and a single yellow circle in the top right corner... one can discern that i probably want a painting of a landscape scene with a single tree and a sun in the corner.
It's fun and totality not forbidden but quit the weed man! I mean you're in the right place and welcome but not in the right century ;)