A unified data format for visual localization, structure from motion and more.
- Robust Image Retrieval-based Visual Localization using Kapture (publication)
- kapture – A unified data format to facilitate visual localization and structure from motion. (blog article)
- One method, one pipeline: NAVER LABS Europe ranks high across three visual localization challenges at CVPR 2020 (blog article)
- 2020 European Conference on Computer Vision (ECCV) (news item)
If you work on visual localization, you’re often faced with the fact that many public datasets are provided in different formats. This means adapting data importers and exporters and almost always having to transform coordinate systems or camera parameters. Furthermore, if you want to use a combination of multiple methods in a single pipeline, a lot of data conversion is required. And, even if there are great tools like OpenMVG or COLMAP available, the provided data formats often don’t include everything you may need, such as wifi or other sensor data.
To address these issues and make it easier to use public datasets we created the kapture data format. We hope kapture will facilitate future research and development in visual localization, structure-from-motion, VSLAM, and sensor fusion.
Kapture is a data format used to describe data acquired for the applications of structure from motion (SFM) and visual localization.
It can be used to store:
- sensor parameters such as intrinsic and extrinsic camera parameters,
- raw sensor data such as camera images or lidar data,
- other sensor data such as GPS or WIFI signals,
- computed data such as:
- 2D local features (keypoints and descriptors)
- 2D-2D matches between local features
- global features (e.g. for image retrieval)
- 3D reconstructions consisting of 3D points and keypoint observations.
Secondly, kapture is a set of Python tools to load, save, and convert datasets to and from Kapture.
Thirdly, we provide a set of public datasets pre-converted to kapture. If you already have your SFM or visual localization processing tools up and running, you just need to integrate kapture support once after which you can use all the datasets without any more conversion or glue code writing.
- The main purpose of kapture is to provide a unified data format for your SFM and visual localization datasets. This will facilitate processing different datasets as well as sharing processed data (e.g. features).
- To convert datasets to and from kapture, we provide a set of converters for popular formats (e.g. COLMAP, bundler, nvm, OpenMVG, OpenSfM, and more).
- As an example, we also provide two full visual localization pipelines based on COLMAP. The first uses the COLMAP SIFT features and the COLMAP vocabulary tree matching, the second uses custom features and matches.
- On Github
Contribute to kapture:
If you find kapture useful, we invite you to contribute!
For example you can:
- Provide your own dataset in kapture format (we’re happy to help).
- Write new data converters.
- Report bugs and improvements
- Provide processed data (e.g. extracted features or matches) in kapture format.
- Add support for other kinds of data not currently supported
- Tell us what you think!
We have already converted various datasets into kapture and we are currently working on their release. In the meantime, please see our instructions on how to convert original datasets into kapture.
Kapture can be used to process all the datasets in this year’s Visual Localization Challenge organized in conjunction with ECCV 2020.