Tracker should estimate the rolling shutter and use that information to make better tracking
100%
Development Status:
Open
Upvote2Downvote0
k
kevin
29 Oct 2017
5 Comments
3
3pointedit
This sounds like a distortion issue like the lens distortion tool already built into Blender, I think that there is work going on to include rolling shutter artefacts in the renderer but I don't know how you would infer the correct characteristics of each cameras rolling shutter based on the speed of the pan? You have to track the features first but they part company relative to each other, not based on their parallax but rather as a n image plane error. How could the tracker tell the difference without first solving the scene?
Could we tell tracker first that 2 or 3 features are joined as a rigid edge, then track that and solve the deviation as a distortion to be compensated for in the regular solve?
1
i
italic.rendezvous
I would be more inclined to use a rolling shutter undistortion feature rather than trying to track distortion in the footage for reconstruction. (rereading, I realize this is basically what you said, just in a different way) I'm just thinking out loud here, but maybe "undistort" could find an overall pattern in regards to track point positions relative to each other, just like shear in a transform matrix. If it finds points sheared consistently across the frame over a threshold, it could borrow data from the surrounding frames to fill in the undistorted empty spaces within the frame. I'm not sure how other software goes about doing rolling shutter reduction, but I vaguely remember reading something similar to this.
What if there was a setting for defining how long each frame's exposure was? Maybe that would give the undistorter a starting point. I can manually change the exposure length on my camera's video mode. But now my question is whether exposure time really changes the rolling shutter effect or if it just makes it blurrier.
3
3pointedit
Exposure time affects the angle of the curve based on the read out from top of frame to the bottom. Faster shutter = less time to read out the whole chip so the apparent distortion is weaker. Its just a slit scan effect. The problem with using gradient to invert the shearing is the variability of the shear from top to bottom. I have tried it in the compositor with offset but find that the gradient must be a vertical curve that alters over time.
None of this takes into account the lost information at the edges of frame,just like stabilising a video you get lots of black around the corrected sides.
1
k
kevinAuthor
to respond to comments :
Undistorting is not an option, It's not only camera moving but also objects ..
One way to do that could be : estimate the rolling shutter speed, and set key frame of the markers at "line/1080*currentTime" (for example) instead of fixed key frames and perhaps linearly interpolate to have the position at "start" of the frame
s
shubham0209
i just try to check different variations of motion blur and find which one suits best. although, it is not always correct (i have now learned to record video moving the camera very slowly so that much blurriness doesnt occur), it does the job most of the time. still, having this feature would be great
5 Comments
This sounds like a distortion issue like the lens distortion tool already built into Blender, I think that there is work going on to include rolling shutter artefacts in the renderer but I don't know how you would infer the correct characteristics of each cameras rolling shutter based on the speed of the pan? You have to track the features first but they part company relative to each other, not based on their parallax but rather as a n image plane error. How could the tracker tell the difference without first solving the scene? Could we tell tracker first that 2 or 3 features are joined as a rigid edge, then track that and solve the deviation as a distortion to be compensated for in the regular solve?
I would be more inclined to use a rolling shutter undistortion feature rather than trying to track distortion in the footage for reconstruction. (rereading, I realize this is basically what you said, just in a different way) I'm just thinking out loud here, but maybe "undistort" could find an overall pattern in regards to track point positions relative to each other, just like shear in a transform matrix. If it finds points sheared consistently across the frame over a threshold, it could borrow data from the surrounding frames to fill in the undistorted empty spaces within the frame. I'm not sure how other software goes about doing rolling shutter reduction, but I vaguely remember reading something similar to this.
What if there was a setting for defining how long each frame's exposure was? Maybe that would give the undistorter a starting point. I can manually change the exposure length on my camera's video mode. But now my question is whether exposure time really changes the rolling shutter effect or if it just makes it blurrier.
Exposure time affects the angle of the curve based on the read out from top of frame to the bottom. Faster shutter = less time to read out the whole chip so the apparent distortion is weaker. Its just a slit scan effect. The problem with using gradient to invert the shearing is the variability of the shear from top to bottom. I have tried it in the compositor with offset but find that the gradient must be a vertical curve that alters over time.
None of this takes into account the lost information at the edges of frame,just like stabilising a video you get lots of black around the corrected sides.
to respond to comments : Undistorting is not an option, It's not only camera moving but also objects .. One way to do that could be : estimate the rolling shutter speed, and set key frame of the markers at "line/1080*currentTime" (for example) instead of fixed key frames and perhaps linearly interpolate to have the position at "start" of the frame
i just try to check different variations of motion blur and find which one suits best. although, it is not always correct (i have now learned to record video moving the camera very slowly so that much blurriness doesnt occur), it does the job most of the time. still, having this feature would be great