motion tracking parallel processing on GPU
motion tracking and detect features take an eternity. Should be able to do it on the GPU as they don't consider each other during calculation. Considering it's currently single threaded, this would be so fast that it could even be used as a meshroom alterntive for photogrammetry!
3 Comments
What kind of algorithm are you proposing?
GPUs work differntly than CPUs. If something is slow on a CPU porting it to a GPU might not take the bottleneck away. Best to research first in why something is slow, before asking to port it to the GPU.
Otherwise we might spent months of work porting it to the GPU (as you're requesting) but still the algorithm is slow as it isn't scalable in the way how GPU processes data.
It is better to ask to improve performance of motion tracking detect features, than to propose to port it to GPU. The first (better performance) is what you want, the second (porting to GPU) is perhaps a solution that doesn't bring you to what you want.
I haven't looked into it to the extent of being able to suggest a specific library, but presumably pytorch or openCV should fit the bill?
GPU acceleration should be ideally suited to this sort of task because the individual tracking points don't cross reference each other during the tracking process to the best of my knowledge.
Perhaps building a deep learning model in pytorch would be the easiest route to a better solution? Should be a super easy one to make. If you can provide a dataset of maybe 50 accurately tracked videos (around 300 frames each) and the tracking point data for each frame, then I'll have a go in colab in a few months when I've finished my current project 👍
If you could include each tracker's 2d xy position per frame, 3d position per frame, camera fov and sensor size, camera center and distortion k values too, then the model should also be able to learn the relationship between those values and the end result too. Would need to have data from varying camera's to avoid overfitting.
this might also be of interest to you: nvidia.com/en-us/on-demand/session/gtcspring22-s41300/