In this paper, we propose a novel multi-view method for surface reconstruction from matched line segments with applications to robotic mapping and image-based rendering. Starting from 3D line segments and poses obtained via Line-SLAM, we project segments from multiple frames into keyframes for image-space analysis. For each keyframe, a grid of image faces is created by optimized intersection of the segment projection lines. These faces define a segment adjacency graph, wherein we perform our Breadth-First Plane Search (BFPS). The found plane hypotheses are merged maximally with respect to a structure-preserving criterion by growing coplanar regions across the graph. Hypotheses violating the visibility constraint are discarded based on fast per-face and mostly nongeometrical evaluation of the scene and image graph. Finally, each image face gets back-projected onto an optimal plane to obtain a 3D surface model. The presented system is a complete and automatic solution suitable for mapping an environment in real-time scenarios like robotic exploration. We demonstrate the performance of our algorithm on several indoor scenes of varying complexity. Compared to a pure 3D analysis of segments, we see a speed-up by one to almost two orders of magnitude, while still improving on reconstruction accuracy.
