Case Study

Foot Dataset Generation

Reducing the cost of neural network training for foot pose estimation using Synthetic Data

Will Perkins
Will Perkins

Business Challenge

For a recent augmented reality application, Laan Labs needed to estimate the position & orientation of human feet in order to display realistic footwear on a user. To solve this computer vision problem of foot pose estimation in real-time, Laan Labs used state-of-the-art neural networks- versatile machine learning models that have the unfortunate characteristic of requiring a lot of training examples.

Image data can be very costly to produce- requiring elaborate photography sessions, hiring actors, and renting obscure props. Image data annotation is labor-intensive and can be error-prone.

Laan Labs needed to gather 10,000+ annotated images that are representative of the real-world conditions a smartphone user might encounter: various camera angles, lighting conditions, footwear, and background scenery.


“with [synthetic data] ... countless variations of images and annotations can be procedurally rendered”


Approach

Laan Labs’ approach was to leverage Synthetic Data, by developing software that generates unique photorealistic images with precise ground truth labels.

A combination of artist-created and reconstructed 3D foot models were used as the seed of a 3D environment. Image annotations were computed directly from the 3D scene. Various aspects of the scene were parameterized, such as: camera angle, lighting, background imagery, post-processing effects, foot pose, and footwear. With the parameterized 3D scene, countless variations of images and annotations can be procedurally rendered using cloud computing.

Business Value

By leveraging Synthetic Data, Laan Labs was able to forego sourcing thousands of pieces of footwear, and save countless hours of photography. Without the need for human annotation, hundreds of work hours can be saved. Additionally, the assurance that images are correctly labeled eliminates a major source of error from the overall project.

Laan Labs leveraged synthetic data to build an image dataset, saving time and money.

Technologies Utilized

Synthetic Data, Computer Vision, Photogrammetry, Unity, Houdini, Python