Saltar apartados
  • UA
  • i3a
  • Scene reconstruction and integration into an augmented reality system

Scene reconstruction and integration into an augmented reality system

Reconstruction of scenes and integration of visual information in a system of augmented reality 3D


Augmented reality is an area of growing interest because of the large number of potential applications. The main challenge is to expand the information that brings the real world with virtual objects and data that can exist in the same space and allow us to improve the user perception of their environment. We start from the hypothesis that artificial Visual systems and, in particular, those based on stereoscopic vision or 3D can help us in a significant way to reconstruct the scene, detect objects that are, in order to label the reality, add its structure, layout, color or identification information, and insert virtual objects. In fact, the stereoscopic vision is one of the mechanisms that use human beings to calculate distances to objects, avoid obstacles, identify elements, etc.

The results achieved by our research group in stereo vision and augmented reality and other works of numerous researchers allow us to suppose that disparity maps from possible recreation of an environment and use it in an augmented reality system.

In short we are taking as a starting point for the project the disparity maps obtained from a stereo vision system and take advantage of the depth information provided to us to reconstruct the scene and add objects and virtual information to the actual system of augmented reality image. The possibility of viewing these images in a 3D virtual reality helmet, would improve the immersion and the possibilities of system control.


1. construction of a map of depths from disparity from the stereoscopic vision system map. In this way, it is much easier to use the depth information in successive steps.

2. 3D reconstruction of the environment from the map of depths, applying a spatial occupancy enumeration scheme. The reconstruction of environments from images have broad interest, not only for its incorporation into an augmented reality system, but also for telecontrol of robots and other machines, for inspection of environments unknown to those who do not may be accessed, detection of the position of objects and its follow-up (traking), etc.

3. correction of reconstruction by applying the concepts of geometry perspective, spatial coherence and other geometric constraints. Correction is required as when projecting images in the camera, a dimension is lost and it introduces a transformation perspective that produces changes in the geometric relationships of objects.

4. integration of the 3D information in an augmented reality system. Intends that the user have a three-dimensional image (thanks to the use of HMD helmets), and this image is complemented with information about the position, orientation and distances of the elements of the scene.

5. generation of virtual objects and inclusion in the actual image, solving the problem of alignment of real and virtual elements.


Based on disparity maps generating sets of 3D vision we have we are obtaining associated depth map. Depth map will allow us to use techniques of enumeration of the occupation of the space, in particular, parent of voxels as a approach to reconstruct the three-dimensional shape of an object from multiple views. In this approach the scene would be built, labeling each voxel with a binary value that determines whether it belongs or not to an object in the scene.

It will be necessary to also define transformations that corrected the effects of imaging perspective of the original images.

The process that we are proposing is, schematically:

esquema reconstrucción

Once reconstructed the scene, we propose to use this reconstruction together with the depths map to incorporate information into a system of augmented reality. Virtual reality helmets have we set ourselves to capture the camera image, incorporating the information and added virtual objects, and view it on the helmets.


Industrial Computing and Artificial Intelligence (i3a)

Universidad de Alicante
Dpto.Ciencia de la Computación e Inteligencia Artificial
Grupo Informática Industrial e Inteligencia Artificial

Carretera San Vicente s/n
03690 San Vicente del Raspeig
Alicante (Spain)

Tel: (+34) 96 590 3400

Fax: (+34) 96 590 3464

For further information:, and for enquiries on this web server:

Carretera San Vicente del Raspeig s/n - 03690 San Vicente del Raspeig - Alicante - Tel. 96 590 3400 - Fax 96 590 3464