SHOWMe: Benchmarking Object-agnostic Hand-Object 3D Reconstruction
Dataset Overview
Annotation Pipeline
Hand-Object capture and registration pipeline. We capture a video sequence of a hand holding rigidly an object and moving in front of an RGB-D camera, and we automatically segment the hand-object system in the images. We reconstruct a precise textured mesh of the hand-object in the exact same pose using an off-the-shelf 3D scanner and register this mesh to each frame to provide ground-truth annotations
Dataset Annotations and Reconstructions
The SHOWMe dataset comprises 96 videos with their associated high-quality textured meshes of a hand holding an object. For two different samples, we show on the left side, row by row, real RGB crops from the dataset, an overlay of the corresponding ground truth textured mesh, and a rendering of the texture-less mesh with Phong shading. On the right, we show the 3D reconstruction of the hand-object obtained from the RGB stream only, using one of the evaluated baselines
Reconstruction Pipeline
Hand-Object 2-stage 3D reconstruction pipeline. Given an RGB video of a hand holding an object (left), the rigid transformation between frames is first estimated. This allows to see the problem as if a set of multiple virtual cameras observe a fixed hand-object system (middle). Multi-view reconstruction can then be employed to estimate an accurate handobject 3D shape (right). We benchmark several baselines for both stages using the presented dataset
Qualitative Results
Citation
@inproceedings{showme,
title={{SHOWMe: Benchmarking Object-agnostic Hand-Object 3D Reconstruction}}, author={{Swamy, Anilkumar and Leroy, Vincent and Weinzaepfel, Philippe and Baradel, Fabien and Galaaoui, Salma and Brégier, Romain and Armando, Matthieu and Franco, Jean-Sebastien and Rogez, Grégory}}, booktitle={{ICCVW}},
year={2023} }
Dataset License
The SHOWMe dataset is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 license.
A summary of the CC BY-NC-SA 4.0 license is located here: https://creativecommons.org/licenses/by-nc-sa/4.0/
The CC BY-NC-SA 4.0 license is located here: https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode