SyB3R: A Realistic Synthetic Benchmark for 3D Reconstruction from Images

Abstract

Realistic benchmark datasets are the foundation of meaningful experimental evaluation in almost all vision problems. In the context of Structure from Motion and (Multi-View) Stereo Reconstruction, these datasets are rather difficult to produce. The field is mainly divided into datasets created from real photos with difficult experimental setups and simple synthetic datasets which are easy to produce, but lack many of the real world characteristics. In this work, we seek to find a middle ground by introducing a framework for the synthetic creation of photo-realistic datasets and their ground truths. We show the benefits of such a purely synthetic approach over real world datasets and discuss its downsides. SyB3R enables new ways to evaluate reconstruction methods that hopefully will focus attention to modules that contain potential for improvement.

Images

Motivation

Bulldozer

Geological sample

Skull

Toad

Bullet

Depth of Field

Texture reduction

ECCV 2016 video

Paper

SyB3R

Authors:

Downloads:

Improved Noise Model

Downloads:

Paper, Author version, Large (soon)
Paper, Author version, Small (soon)

Code

SyB3R v1.0 First Release
Improved noise model Standalone improved noise model (soon)

Requirements

The code base is written in C++14. The entire framework was developed under Linux but, with some work, should work under windows as well.
The following libraries/programs are needed:

CMake (build system)
Boost
OpenEXR
ImageMagick
Eigen
TinyXML2
Blender + Cycles

Optional:

Exiv2
PCL
Doxygen (to build the documentation)

How to Build the Framework

To build the documentation (you'll need Doxygen), go to the "doc/" directory and execute the "buildDoc.sh" shell script, e.g.:

$ cd doc
$ ./buildDoc.sh

You will then find the main documentation in "doc/html/index.html" with instructions on how to build and use the framework.
Building the actual code is based on CMake. See the doxygen generated documentation for more details.

Datasets

Preview	Downloads	Comments and Credits
	Version 1.0	Based on the excellent work of the blender community: Toad by arenyart, released under CC-Zero
	Version 1.0	Based on the excellent work of the blender community: Lego 856 Bulldozer by Heinzelnisse, released under CC BY-NC 3.0
	Version 1.0	The amount of texture can be changed by passing one of the two python scripts as a pre render hook to the render script generator. Based on the excellent work of the blender community: Skull by ColeHarris, released under CC-Zero Spartan Helmet by matpiet, released under CC-Zero, slightly adapted for Cycles Cycles Wood Material by wesvdes
	Version 1.0	To switch between the two cameras, either select one of them in the scene properties of Blender, or specify one (or both) when generating the render script.

Citation

If you use this code or one of the datasets for your research, please cite:

@Inbook{syb3r2016,
    author="Ley, Andreas and H\"ansch, Ronny and Hellwich, Olaf",
    editor="Leibe, Bastian and Matas, Jiri and Sebe, Nicu and Welling, Max",
    title="SyB3R: A Realistic Synthetic Benchmark for 3D Reconstruction from Images",
    bookTitle="Computer Vision -- ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11--14, 2016, Proceedings, Part VII",
    year="2016",
    publisher="Springer International Publishing",
    pages="236--251"
}

@inproceedings{haensch2019sybernoise,
    title={A Digital Image Processing Pipeline for Modelling of Realistic Noise in Synthetic Images},
    author={Bielova, Oleksandra and H{\"a}nsch, Ronny and Ley, Andreas and Hellwich, Olaf},
    booktitle={2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)},
    volume={tba},
    number={tba},
    pages={tba},
    year={2019},
    month={06},
    publisher={IEEE},
    organization={},
    doi={tba},
    issn={tba}
}

License

The implementation is distributed under GPLv3. If this license doesn't fit your needs, you can get in contact with us.

Projects