Photogrammetry can be used to make a 3D model from a collection of photographs. A subject is captured from many different angles, and special software is used to process the images and generate a point cloud, or group of 3D points. Photogrammetry has been used to great effect in video games, and allows developers to create highly realistic backgrounds.
I recently discovered a free and open-source photogrammetry application called “Meshroom“, and I immediately wondered if it could use videos for input as well as photographs. I found other users asking similar questions on the meshroom github page. One individual recommended Zephyr, which is proprietary 3D reconstruction software. In the interest of creating a completely free videogrammetry solution, I designed a simple Zephyr-inspired Windows tool to convert video to a series of individual frames.
My program is basically a UI front-end for ffmpeg, which is a free suite of video software. Using the selected on-screen values, it builds a command line with the right parameters to extract the desired images. I also wrote some blur- and similarity-detection code with the Emgu CV library, but I didn’t end up needing those features.
Using a Panasonic Lumix G7, I recorded a video of a turtle figurine. I put the turtle on a foam board and rotated it 360 degrees.
This approach isn’t recommended, and I soon discovered why. The moving shadows confused Meshroom and I got this weird structure hanging off the bottom of the generated 3D model.
The preferred method is to move the camera around the subject being photographed, but limited space made this impossible for my setup.
Having learned my lesson, I went outside and shot some footage of a large planter. The weather was overcast, which is good for preventing harsh shadows. But I encountered another issue. Regardless of shutter speed or how steadily I held the camera, most of the frames were too blurry to be useful for 3D reconstruction.
This is a known challenge in videogrammetry, and I didn’t bother loading the images into Meshroom since I’m sure the quality would be poor. As I was recording this it began to rain, and rather than try again later, I decided to try a completely different type of video.
I went online and found a public domain clip of a chapel in Germany, recorded by a drone rotating around the building from a high elevation. The motion was smooth and the frames were sharp, even though it didn’t cover a full 360 degrees of motion. The results were surprisingly good, and I got a nice 3D model for my efforts.
Loaded into Blender, the textured model is quite realistic.
Based on these tests, I wouldn’t recommend videogrammetry over photogrammetry. Even though you can capture dozens or hundreds of images easily with just a short video clip, it’s hard to match the quality of photos snapped individually. Although this was a fun and useful experiment, the main thing I learned is that unless you only have video source material, videogrammetry probably isn’t worth it.
The Public Void 