- Deutsches Museum Poster [PDF - 3.1 MB - 10. August 2015]
- Light Field Camera Presentation [PDF - 4.3 MB - 16. July 2015]
To obtain 2D and 3D data from a single shot, we need mainly two things: a micro lens array (MLA) and software algorithms.
The MLA is placed in front of the image sensor inside the camera, which turns the image sensor into a micro-camera array (see figure on right), where each micro-camera sees part of the intermediate image from a slightly different perspective. That is, instead of using a large camera array that looks at the object directly, we can choose a main lens to select the desired field-of-view and create the intermediate image in front of the micro camera array. The images generated by the camera in this setup are processed on a PC with appropriate software algorithms to calculate the scene depth and to reconstruct a 2D image.
We do all our processing on a GPU, which allows us to typically process five 12 megaray raw images per second, resulting in thirty 3 megapixel 2D & 3D images per second (30 FPS).
The no-free-lunch theorem also applies here: You can obtain 2D & 3D data in a single shot but you pay with effective lateral resolution. Our cameras have a maximal effective lateral resolution of about one quarter of the sensor resolution. This effective lateral resolution also varies over the whole depth-of-field. The camera is setup by focusing the main lens to the furthest plane you are interested in. From this plane towards the camera the depth can be calculated and a 2D image reconstructed. Everything that lies behind this plane can not be reconstructed at all. The effective lateral resolution is highest at the focus plane and drops as you get closer to camera.
The depth resolution is typically 1% of the total depth-of-field, which in turn depends on the focal length of the main lens and the focus setting. You can select different main lenses for the same light field camera to adjust the depth-of-field to your needs. However, the larger the field-of-view, the larger the depth-of-field and the closer you need to be to the main lens to obtain a good depth resolution. In particular, this means that a light field camera in combination with a wide-angle lens can only see depth differences close to the main lens.
On the other hand, light field cameras work well within the macro realm, with microscopes and also with telescopes.