Experimenting with Videogrammetry

Photogrammetry can be used to make a 3D model from a collection of photographs. A subject is captured from many different angles, and special software is used to process the images and generate a point cloud, or group of 3D points. Photogrammetry has been used to great effect in video games, and allows developers to create highly realistic backgrounds.

I recently discovered a free and open-source photogrammetry application called “Meshroom“, and I immediately wondered if it could use videos for input as well as photographs. I found other users asking similar questions on the meshroom github page. One individual recommended Zephyr, which is proprietary 3D reconstruction software. In the interest of creating a completely free videogrammetry solution, I designed a simple Zephyr-inspired Windows tool to convert video to a series of individual frames.

image-extractor-ui

My program is basically a UI front-end for ffmpeg, which is a free suite of video software. Using the selected on-screen values, it builds a command line with the right parameters to extract the desired images. I also wrote some blur- and similarity-detection code with the Emgu CV library, but I didn’t end up needing those features.

Using a Panasonic Lumix G7, I recorded a video of a turtle figurine. I put the turtle on a foam board and rotated it 360 degrees.

turtle-video.png

This approach isn’t recommended, and I soon discovered why. The moving shadows confused Meshroom and I got this weird structure hanging off the bottom of the generated 3D model.

turtle-blender

The preferred method is to move the camera around the subject being photographed, but limited space made this impossible for my setup.

Having learned my lesson, I went outside and shot some footage of a large planter. The weather was overcast, which is good for preventing harsh shadows. But I encountered another issue. Regardless of shutter speed or how steadily I held the camera, most of the frames were too blurry to be useful for 3D reconstruction.

planter-video.png

This is a known challenge in videogrammetry, and I didn’t bother loading the images into Meshroom since I’m sure the quality would be poor. As I was recording this it began to rain, and rather than try again later, I decided to try a completely different type of video.

I went online and found a public domain clip of a chapel in Germany, recorded by a drone rotating around the building from a high elevation. The motion was smooth and the frames were sharp, even though it didn’t cover a full 360 degrees of motion. The results were surprisingly good, and I got a nice 3D model for my efforts.

chapel-meshroom.png

Loaded into Blender, the textured model is quite realistic.

chapel-blender.png

Based on these tests, I wouldn’t recommend videogrammetry over photogrammetry. Even though you can capture dozens or hundreds of images easily with just a short video clip, it’s hard to match the quality of photos snapped individually. Although this was a fun and useful experiment, the main thing I learned is that unless you only have video source material, videogrammetry probably isn’t worth it.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s