MAP |
Automated 3D Modeling of Building Interiors

Three-dimensional models of objects, sites, buildings, structures, and urban environments made of geometry and texture are important in a variety of civilian and military applications such as training and simulation for disaster management, counteracting terrorism, virtual heritage conservation, virtual museums, historical sites documentation, mapping of hazardous sites and underground tunnels, and modeling of industrial and power plants for design verification and manipulation.

While object modeling has received a great deal of attention in recent years, 3D site modeling, particularly for indoor environments, poses significant challenges. The main objective of this proposal is to design, analyze, and develop architecture and algorithms, as well as associated statistical and mathematical framework for a human operated, portable, 3D indoor/outdoor modeling system, capable of generating photo-realistic rendering of the internal structure of multi-story buildings as well as external structure of a collection of buildings in a campus.

Key technical challenges consist of system architecture, sensor selection, calibration, and error characterization, local and global localization algorithms, sensor integration, registration, and fusion, and complete and accurate coverage of all details. Using a human-operated backpack system equipped with 2D laser scanners, cameras, and inertial measurement units, we develop four scan-matching-based algorithms to localize the backpack and compare their performance and tradeoffs. In addition, we show how the scan-matching-based localization algorithms combined with an extra stage of image-based alignment can be used for generating textured, photorealistic 3D models.

 


Photos of the backpack at ARPA-E Summit - February 2014


Cheryl Martin, ARPA-E Acting Director, takes RAPMOD for a spin



Secretary of Energy Dr. Earnest Moniz discusses the backpack



U.S. Congressman Adam Schiff (California's 28th district) tries on the backpack



U.S. Senator Sheldon Whitehouse (RI - Dem) checks out the hardware


Sample 3D Models

Models with Arbitrary Geometry and Exposure Compensation - December 2013

Models shown below correspond to interior and exterior views of hotels, whose model geometries are comprised of millions of triangles at various orientations. These triangles are segmented into independent regions for texturing either in a shortest-path or a tiled method. An additional exposure compensation step is performed across each region's generated texture as well.

Houston model - exterior

Houston model - interior

Beverly Hills model - exterior

Beverly Hills model - interior

Planar Models with Shortest Path/Tile-Caching Texturing Approach - July 2012

The first two images correspond to a textured model of the hallways located on the 2nd, 3rd, and 4th floors of Cory Hall on the UC Berkeley campus. The third image is a textured model of the Swarm Lab on the 4th floor of Cory Hall. For both models, geometry is generated by fitting planes to represent the walls, ceilings, and floors. These planes are textured by using a shortest path texturing approach on walls, and a tile-caching texturing method on floors and ceilings.

3 story Cory Hall textured planar model - 1

3 story Cory Hall textured planar model - 2

Swarm Lab textured planar model - 2

Video Examples and Walkthroughs


Download Links:
Texture Comparisons
Cory Hall 4th Floor Walkthrough
Cory Hall 3 Story Walkthrough
Pier 15 Walkthrough

 

Two Story Path - Planar Model - August 2011

This is a model of the hallways located on the 3rd and 4th floor of Cory Hall on the UC Berkeley campus. This data was captured in a single run, using a stairwell to move between floors. The model shown uses detected planes to represent the walls, ceiling, and floors.

2 story planar model - 1

2 story planar model - 2

 

Two Story Path - Planar Model generated w/ Autodesk Revit Architecture - August 2011

This is a model of the hallways located on the 3rd and 4th floor of Cory Hall on the UC Berkeley campus. This data was captured in a single run, using a stairwell to move between floors. The model shown uses the Autodesk Revit Architecture software package to detect planes in a user-guided way.

2 story planar model - REVIT - 1

2 story planar model - REVIT - 2

 

Two Story Path - Planar Model, Mosaic Texture - August 2010

This is a model of the hallways located on the 3rd and 4th floor of Cory Hall on the UC Berkeley campus. This data was captured in a single run, using a stairwell to move between floors. The model shown uses detected planes to represent the walls, ceiling, and floors and uses a single image mosaic for texture on each wall.

Download this video: Video 1 (12MB WMV, playable with VLC)

 

Three Story Point Cloud - July 2011

This is a point cloud of the hallways located on the 2nd, 3rd and 4th floor of Cory Hall on the UC Berkeley campus. This data was captured in a single run, using stairwells to move between floors.

Three story point cloud - July 2011

Reconstructed Path:

Three story path - July 2011

 

Two Story Point Cloud - July 2011

This is a point cloud of the hallways located on the 3rd and 4th floor of Cory Hall on the UC Berkeley campus. This data was captured in a single run, using a stairwell to move between floors.

Two story point cloud - July 2011

 

Staircase Point Cloud

This is a point cloud of the southeast staircase between the 3rd and 4th floors of Cory Hall at UC Berkeley.

Download this video: Video 1 (3.3MB WMV, playable with VLC)

 

T-Shaped Path - Planar Model - June 2011

This is a model of the hallways located on the 4th floor of Cory Hall on the UC Berkeley campus.

Download this video: Video 1 (10.6MB WMV, playable with VLC)

 

T-Shaped Path - Triangulated Model - February 2010

This is a model of the hallways located on the 4th floor of Cory Hall on the UC Berkeley campus.

20100228_set4_tshape.jpg

20100228_set4_tshape2.jpg

Download this video: Video 1 (3.6MB WMV, playable with VLC)

 

Image based rendering of the 4th floor of Cory Hall on the UC Berkeley campus.

Download this video: Video 1 (3.4MB AVI-MJPEG, playable with VLC)

Video of this dataset rendered by our image-based renderer: Video (1.1GB AVI-MPNG, playable with VLC)

 

Reconstructed Path:

set4_tPath.pngThis path is 68.73m long, with an average position error of 0.66m.

Download this model: T-Shaped Model (180MB zipped IVE, requires osgviewer - see instructions)

 


Instructions for Viewing the Model Files

The above models are provided as textured, triangular meshes in the IVE format, which is a native format of OpenSceneGraph. To view the models, download and install OpenSceneGraph, which includes the osgviewer viewer application. Once installed, open a command prompt, change to the directory in which the models are stored and run the viewer with the command: osgviewer <model filename>, where <model filename> is the name of the .ive model file.

 


Point Clouds Obtained with the Pushcart System

In addition to the backpack system, we have a pushcart system consisting of 3 laser scanners, 2 cameras, and 1 IMU.

 

Cory 3rd Floor NW Lounge

This is a point cloud captured of the northwest lounge located on the 3rd floor of Cory Hall on the UC Berkeley campus.

 

Cory 5th Floor

This is a point cloud captured of the hallways of the 5th floor of Cory Hall on the UC Berkeley campus.

Download this point cloud: Cory 5th Floor (550MB ASCII XYZ, requires Quick Terrain Modeler or similar point cloud viewer)

 


Download Software

 


Related Publications

  • J. Kua, N. Corso, A. Zakhor, "Automatic Loop Closure Detection Using Multiple Cameras for 3D Indoor Localization," to be presented at IS&T/SPIE Electronic Imaging 2012, Burlingame, California, January 22-26, 2012. [Adobe PDF]
  • T. Liu, M. Carlberg, G. Chen, J. Chen, J. Kua, A. Zakhor, "Indoor Localization and Visualization Using a Human-Operated Backpack System," 2010 International Conference on Indoor Positioning and Indoor Navigation, Zurich, Switzerland, September, 2010. [Adobe PDF]
  • G. Chen, J. Kua, S. Shum, N. Naikal, M. Carlberg, and A. Zakhor. "Indoor Localization Algorithms for a Human-Operated Backpack System," 3D Data Processing, Visualization, and Transmission 2010, Paris, France, May 2010. [Adobe PDF]
  • N. Naikal, J. Kua, G. Chen, and A. Zakhor. "Image Augmented Laser Scan Matching for Indoor Dead Reckoning," IEEE/RSJ International Conference on Intelligent RObots and Systems (IROS), St. Louis, MO, October 2009. [Adobe PDF]

 


Press Coverage

 


Related Work