Automated Model-Based Finding of 3D Objects in Cluttered ...static.tongtianta.site/paper_pdf/3dd10f9a-3766-11e9-b891-00163e08bb86.pdfAutomated model-based ﬁnding of 3D objects in

Computer-Aided Civil and Infrastructure Engineering 00 (2017) 1–16

Automated Model-Based Finding of 3D Objects inCluttered Construction Point Cloud Models

Mohammad-Mahdi Sharif, Mohammad Nahangi*, Carl Haas & Jeffrey West

Department of Civil and Environmental Engineering, University of Waterloo, Waterloo, ON, Canada

Abstract: Finding construction components in clutteredpoint clouds is a critical pre-processing task that requiresintensive and manual operations. Accurate isolation ofan object from point clouds is a key for further process-ing steps such as positive identification, scan-to-buildinginformation modeling (BIM), and robotic manipulation.Manual isolaton is tedious, time consuming, and discon-nected from the automated tasks involved in the process.This article adapts and examines a method for finding ob-jects within 3D point clouds robustly, quickly, and auto-matically. A local feature on a pair of points is employedfor representing 3D shapes. The method has three steps:(1) offline model library generation, (2) online searchingand matching, and (3) match refinement and isolation.Experimental tests are carried out for finding industrial(curvilinear) and structural (rectilinear) elements. Themethod is verified under various circumstances in orderto measure its performance toward addressing the majorchallenges involved in 3D object finding. Results showthat the method is sufficiently quick and robust to be in-tegrated with automated process control frameworks.

1 INTRODUCTION

Automated modeling of fabricated construction com-ponents is a bottleneck in automatic and continuousmonitoring of civil infrastructure (Dimitrov andGolparvar-Fard, 2015). In particular, preprocessingthe massive data collected on construction sites is keyfor effective and electronically integrated modelingof the built environment. Automated modeling isnecessary for various key applications such as progressmonitoring, status assessment, and quality control. Forexample, imperfections and fabrication errors maycause huge rework costs to the projects if they are noteffectively monitored and corrected. In 2010, Canada’s

∗To whom correspondence should be addressed. E-mail:[email protected].

construction industries (i.e., residential, nonresidentialengineering, repair, and other construction sectors)accounted for 6% of Canada’s gross domestic product(GDP), contributing CDN $73.8 billion (StatisticsCanada, 2010). In a typical construction project, reworkcosts between 2% and 20% of a project’s contractamount (Goodrum et al., 2016). According to Dis-sanayake et al. (2003), rework is defined as: “Activitiesin the field that have to be done more than once, oractivities, which remove work previously installed aspart of the project regardless of the source, where nochange order has been issued and no change of scopehas been identified by the owner.” Geometric noncom-pliance is one of the main factors causing rework in aproject, in general, and in the fabrication processes, inparticular.

To reduce rework, rigorous and continuous inspec-tions throughout the fabrication process are required.Conventional methods for quality control and reworkmitigation utilize humans with manual direct contactmeasuring devices such as tapes and calipers. Man-ual execution of such tasks increases the subjectivityof information as well as other errors and limitationsincorporated with intervention. This includes measur-ing locations with difficult access or spots having haz-ardous materials. Furthermore, the conventional meth-ods are not only limited by human capabilities, but also,they are time consuming and may cause interruptionin the production process. This results in depriving themanagers of continuous monitoring and quality controlon the fabrication process. Consequently, utilizationof conventional methods will fail to acquire accurate,rapid, and continuous geometric compliance monitoringsystems.

Advancements in 3D imaging technology have al-lowed its users to collect spatial data from their sur-roundings in a short time period, and with an acceptableaccuracy level. Laser scanners measure the distancesto points being scanned at speeds up to thousands of

C© 2017 Computer-Aided Civil and Infrastructure Engineering.DOI: 10.1111/mice.12306

2 Sharif, Nahangi, Haas & West

points per second (Park et al., 2007). Most of the appli-cations of laser scanning in construction, including au-tomated compliance control (Nahangi and Haas, 2014),and schedule and progress tracking (Turkan et al., 2012)rely on either manual or partially automated identifica-tion, location, orientation, and extraction of the object-of-interest. Other methods rely on techniques, such asBosche and Haas (2008), that were premised on a pri-ori knowledge of scanner location and orientation withrespect to site coordinates. This is due to the indiscrimi-nate data acquisition by the capturing devices. The pointclouds acquired with a laser scanner will include clut-ter (unwanted objects in the background or surround-ings of the object-of-interest), and uncaptured surfaceswhen the objects are occluded. The variation in the den-sity of a point cloud and the existence of noise, whichusually occurs on the edge surfaces, are also amongthe challenges in the automation of the object extrac-tion process. Other contributing factors such as lightingconditions and site specific circumstances can also in-fluence the quality of the captured point cloud (Sharifet al., 2016), which will exacerbate the complexity ofthe 3D object recognition process. An incomplete pointcloud of a fabricated component is another commonchallenge. The aforementioned challenges reveal thecomplexity of formalizing an automated framework forobject-of-interest isolation from a cluttered 3D pointcloud.

The manual extraction of an object-of-interest ina cluttered point cloud is inadequate, inaccurate,and inefficient in terms of the required time and thelevel of skill required (Figure 1). An automated andrapid object finding framework has the potential to beemployed in automated object locating, robotic manip-ulation, and quality control processes in construction.A rapid framework will avoid late detection of possibledefects, and therefore the cumulative error arisingfrom infrequent fabrication monitoring. This studyaims to develop a robust framework for efficient andautomated finding of an object-of-interest in clutteredpoint clouds. This framework is capable of addressingsome of the major challenges in this area including:

� Density variation: various types of sensors offer dif-ferent levels of density in the point cloud acquired.The desired framework for object isolation must beinsensitive to the density of the point cloud used.

� Clutter presence: presence of unwanted objects in thebackground and surrounding the object-of-interest isthe key motivation for automated recognition andisolation of the objects-of-interest.

� Occlusion and incompleteness occurrence: in visualsensing and vision-based data acquisition sensors,line of sight is a substantial parameter for capturing

complete and reliable data. In the case that the ob-jects are not visually tracked by sensors, some parts orcomponents might be missing. The subsequent anal-yses and models are therefore influenced by such in-complete data (Nahangi and Haas, 2016). The desiredframework should also be relatively robust to incom-plete point clouds.

A robust framework for automated finding ofobjects-of-interest from cluttered and unprocessed 3Dpoint cloud models is presented in this article. Theframework is based on the mathematical model firstpresented by Papazov and Burschka (2010). Compar-atively, this framework has three primary steps: (1)creating and storing a library of features from pointpairs of 3D models using a local feature, (2) findingthe potential matching pairs from the point cloud withthe code library generated using a RANSAC-based hy-pothesis testing, and (3) match refinement and isolationusing an ICP-based (Iterative Closest Point) regis-tration step. The key contribution of this study is theadaptation and application of a robust framework forautomated finding of 3D objects in cluttered point cloudmodels from a construction environment (Papazov andBurschka, 2010). The framework is tested under variouscircumstances in order to investigate its performancefor addressing the major challenges discussed previ-ously, including density variation, clutter presence, andincompleteness of the captured data. First, the relatedbackground is thoroughly investigated in the followingsection to clearly identify the knowledge gap and thekey contribution of this work. Next, the proposedmethodology and its components are described. Finally,experimental results and analyses are provided toquantify the performance of the proposed method.

2 BACKGROUND

2.1 Terminology definition

For the purpose of consistency and rigorousnessthroughout this manuscript, the following terms are de-fined and described as follows:

Detection: refers to the process in which the presence ofan object is identified in an acquired point cloud.

Finding: refers to the process in which the presence ofthe object is not only sensed but also its geometriccharacteristics such as dimensions, location, and orien-tation are identified. The term “recognition” however,corresponds to identification and characterization ofall the objects that meet the recognition criteria in thescene.

Automated model-based finding of 3D objects in cluttered construction point cloud models 3

Points removed

Points kept

(a) (b) (c) (d)

Fig. 1. Clutter removal example. (a) A facility is scanned; (b) surrounding objects are removed; (c) secondary attachments in theproximity of the object-of-interest (i.e., stands and supporting objects) are removed. (d) The object is finely retrieved by manually

removing noise and other points remained. The isolated point cloud is then ready for further processing (e.g., automatedregistration for discrepancy quantification and quality control).

Segmentation: refers to the process of classifying pointsfrom the surface of an object in one set and from acluttered and noisy point cloud.

Isolation: refers to the process of extracting a segmentedobject from the 3D point cloud and representing it asa single data set.

This section focuses on a comprehensive review ofthe existing methods for finding 3D objects from var-ious perspectives with respect to some applications inconstruction automation. A general overview is firstprovided from the computer science perspective. Ex-isting challenges and various categories of 3D objectrecognition are also briefly discussed. Major applica-tions in the construction literature and the existing re-search challenges are then discussed. Although therehave been numerous research studies in automated ob-ject recognition from 2D images (Balali and Golparvar-Fard, 2015), video frames (Park and Brilakis, 2012;Zhu and Brilakis, 2010), and depth images (Ray andTeizer, 2012) for a wide range of applications in con-struction, this article only focuses on finding 3D objectsin cluttered point clouds and the research challengesinvolved.

2.2 Object recognition: general categories and existingchallenges from the computer science perspective

The problem of finding an object-of-interest has beenwidely investigated in the computer science liter-ature. Vision-based control in robotics (Chaumetteand Hutchinson, 2006), intelligent surveillance (Guoet al., 2013), and mobile manipulation (Quigley et al.,2009) are only a few applications, which are welldeveloped and widely used in the related body ofknowledge.

However, finding objects in the aforementioned ap-plications is relatively limited to 2D scene capturing ap-proaches such as 2D images or 2D snapshots from video

frames. Unprecedented opportunities have recently be-come feasible with the significant improvements in 3Ddata acquisition, and there has been extensive work on3D object recognition from 3D scenes (i.e., 3D pointclouds). Object recognition approaches from the com-puter science perspective can be grouped into model-based versus nonmodel-based approaches. Althoughmodel-based approaches perform more robustly, the re-quired processing time is the major drawback for auto-mated 3D object recognition and modeling in construc-tion automation.

According to Guo et al. (2014), object recognitionmethods can be grouped into two main approaches:(1) global features or 3D keypoint detection and lo-calization and (2) local features characterization andlocalization. The former approach includes 3D SIFT(scale invariant feature transform) (Allaire et al., 2008),3D LIFT (learning invariant feature transform) (Huanget al., 2007), and 3D SURF (speeded-up robust fea-tures) (Knopp et al., 2010), which are performed oneither depth images (2.5D) or 3D meshes. Such meth-ods are incapable of finding 3D objects from 3D pointclouds. The local features approach is thus more widelyused for 3D object recognition from 3D point clouds.This approach includes signature-based and histogram-based methods such as spin images (Johnson andHebert, 1998), point signature (Chua and Jarvis, 1997),point pair features (Papazov and Burschka, 2010), anda set of points such as 4-point congruent sets (4PCS) byAiger et al. (2008). Based on the extensive survey byGuo et al. (2014), local features have been found to bemore efficient for 3D object recognition from 3D pointclouds.

2.3 3D object recognition: applications in construction

In built environments, it is imperative to find objects-of-interest automatically and effectively to assess their


as-built status and map critical construction perfor-mance metrics. Such metrics include as-built progresscompared to the as-planned schedule (Kim et al., 2013;Golparvar-Fard et al., 2012; Turkan et al., 2012), oras-built shape or geometry compared to the as-designedgeometry (Nahangi and Haas, 2014; Kim et al., 2016). Inthis section, object recognition applications in the con-struction literature is investigated from three perspec-tives: (1) automated as-built modeling, (2) quality con-trol and automated modeling, and (3) progress tracking.

2.3.1 Object recognition for automated as-builtmodeling. As discussed by Patraucean et al. (2015),as-built building information modeling (BIM) creationis challenging due to the complexity of constructioncomponents. However, some components representedby explicit geometric shapes can be detected, recog-nized, and modeled given a 3D point cloud representingthe built environment. Some examples include MEP(mechanical, electrical, plumbing) components, ingeneral, and cylindrical objects (e.g., pipes and elbows),in particular. This research area is also known asscan-to-BIM in the related literature (Bosche et al.,2015).

Rabbani et al. (2007) presented a generalized Hough-based method for detecting and recognizing industrialand piping elements with some basic explicit shapes.The processing time for recognizing objects from pointclouds was substantial and therefore ineffective, be-cause they were using the Hough transform in 3D. Thismethod was also incapable of recognizing and modelingelbows and T-sections. Ahmed et al. (2014) presenteda method for detecting and reconstructing cylindricalobjects such as 3D pipes using a modified HoughTransform-based method. Their method overcomesthe computationally intensive 3D Hough Transformby projecting points into orthogonal slices (planes) andthen applying a 2D Hough-based circle detection. Theirapproach was also reported to be incapable of findingT-sections and elbows and it was only applicable oncylindrical objects laid out in orthogonal directions.

Son et al. (2014) presented a curvature-based cylin-drical object recognition, which was found to be capa-ble of finding elbows and intersections. However, theirmethod relied on an accurate and complete 3D pointcloud as an input, and it is therefore inadequate for find-ing complicated cylindrical branches. Lee et al. (2013)presented a skeleton-based method for 3D reconstruc-tion of industrial elements. The skeleton-based methodwas also inadequate and inaccurate in the case that anincomplete 3D point cloud is imported to their frame-work. According to Nahangi and Haas (2016), incom-plete point clouds will change the skeletons represent-

ing the centerlines, and will therefore create errors inthe radius detection and recognition.

A curvature-based segmentation method with appli-cations to MEP components was then presented byDimitrov and Golparvar-Fard (2015). Although theirmethod is sufficiently accurate in recognizing variouscomponents from a cluttered scene, it is still compu-tationally expensive. Their method requires curvaturecalculation on a resampled point cloud, which is thenused for checking connectivity of components. Assum-ing that time-effective process controllers are desirable,in practice, their curvature-based method is incapableof addressing the time-related aspects and challenges.Dimitrov et al. (2016) then extended the curvature-based segmentation to model arbitrary shapes given anoisy and cluttered 3D point cloud model. Their recentwork takes advantage of the previously segmented com-ponents. It then employs nonuniform rational B-splines(NURBS) for modeling arbitrary shapes in the form ofexplicit and closed form mathematical functions. This isdirected toward the ultimate goal of scan-to-BIM cre-ation. Zhang et al. (2015) presented a framework forplanar patch detection from cluttered point clouds. Thesegmentation of planar patches is based on normal vec-tor calculation and spectral clustering, which was foundto be robust.

2.3.2 Object recognition for quality control and as-builtstatus assessment. One other key application for auto-mated object recognition is to assess the as-built statusor geometric quality of the components compared to theas-designed drawings integrated in the BIM. This areais also known as scan-versus-BIM in the related litera-ture (Bosche et al., 2015), which is categorized as model-based approaches. On the other hand, nonmodel-basedapproaches for quality control are also developed forvarious application such as flatness control of concreteslabs (Bosche and Guenet, 2014; Tang et al., 2011).

A framework for automated discrepancy quantifi-cation of fabricated serial components was presentedbased on the as-built point clouds of components auto-matically registered and compared with their 3D mod-els. The isolation step was performed manually, whichwas disconnected from the fully automated framework.Automated isolation of the components is therefore thekey to expedite the entire process. The method was thenextended to parallel assemblies with a strategy for re-aligning the defective assemblies (Nahangi et al., 2015);however, lack of an automated step to automatically ex-tract an object-of-interest given a point cloud was stilla drawback for integration with automated fabricationprocess controllers.

A skeleton-based method for discrepancy quan-tification was then presented, in which the object


Table 1Summary of 3D object recognition methods existing in the construction

ReferenceIdentification

status MethodResearch stream in

constructionSpecific application

in construction

Rabbani et al., 2007 Recognition 3D Hough Transform 3D modeling Industrial elementsAhmed et al., 2014 Recognition 2D Hough Transform As-built BIM Cylindrical pipesSon et al., 2014 Recognition Curvature As-built BIM Cylindrical pipesLee et al., 2013 Recognition Skeleton As-built BIM Cylindrical pipesDimitrov et al., 2016; Dimitrov

and Golparvar-Fard, 2015Segmentation Curvature, NURBS As-built BIM MEP components

Zhang et al., 2015 Detection Normal vector As-built BIM Planar componentsNahangi et al., 2015 Recognition NA

Manual isolation As-built statusassessment

Serial and parallel

Czerniawski et al., 2016a Recognition Curvature As-built statusassessment

Serial and parallel

Golparvar-Fard et al., 2012 Detection Statistical Progress tracking Any typeTurkan et al., 2012; Bosche and

Haas, 2008Detection Closest points Progress tracking Any type

Kim et al., 2013 Detection SVM-based classifier Progress tracking Column, beam, slabCzerniawski et al., 2016b Recognition Normal vector Object isolation Planar regionsChen et al., 2016 Detection PCA Automated

monitoringConstruction

equipment

isolation step was still performed manually (Nahangiand Haas, 2016). Recently, Czerniawski et al. (2016a)presented a 3D model-based object-of-interest recogni-tion and isolation method, where curvature was a sig-nature or descriptor of the model. A bag-of-featureswith two-way curvature descriptors was created in or-der to represent the 3D model of an object-of-interest.The feature was then searched in a 3D point cloudtransformed to the feature space. The hypothesis test-ing and matching was then performed using a bi-variatehistogram-based voting scheme. This method was lim-ited to the objects where curvature is a meaningful rep-resentative (e.g., industrial object, in general, and cylin-drical pipes, in particular). Although, the method wascapable of extracting arbitrary 3D objects from clut-tered point clouds automatically and with a high recog-nition rate (90% in average), its computational time isstill a drawback for the applications desired.

2.3.3 Object recognition for progress tracking. Objectdetection and recognition has been widely used to trackthe progress of components compared to the as-plannedschedule integrated with the BIM. Generally, for thepurpose of progress tracking, detecting an object will besufficient to measure the as-built schedule and comparewith the as-planned schedule.

An image-based framework for automated progresstracking using statistical correspondence for object de-

tection was presented (Golparvar-Fard et al., 2012).Turkan et al. (2012) presented a framework based onthe object detection method previously developed byBosche and Haas (2008). The object detection andprogress tracking is based on the level of overlap be-tween the as-planned and as-built 3D point clouds thatare finely aligned. Kim et al. (2013) presented a training-based framework for automated progress tracking thatused an SVM-based classifier for major objects in abuilding (i.e., columns, beams, and slabs).

In summary, the problem of robust and efficient find-ing of a 3D object in 3D point cloud models as well as itsmajor research challenges has remained an elusive goal.Another perspective on the literature is the dichotomybetween model-based (Nahangi et al., 2015; Czerni-awski et al., 2016a, b) and nonmodel-based (Bosche andGuenet, 2014; Tang et al., 2011) methods, which must beconsidered as well in attempts to make advances in thisarea. The following section frames the knowledge gapfrom the conducted literature review, and identifies themajor contribution of the work in the current study. Asummary of the investigated studies along with a gen-eral categorization is also provided in Table 1.

2.4 Knowledge gap and research contribution

As discussed, for finding 3D objects in clutteredpoint cloud models of construction environments, the


(a)

(b)

(c)

(d)

Fig. 2. Graphical abstract of the presented framework. (a)3D model converted to a point cloud. (b) Cluttered 3D scene.

(c) Localized model on the scene. (d) Found and isolatedobject from the scene.

previously developed frameworks are either relativelyineffective in terms of processing time or are not fullyautomated. As well, the existing methods are limitedto explicit shapes and geometries such as MEP com-ponents (cylindrical objects) or some simple structuralcomponents (concrete beams and columns with rectan-gular cross section). This article presents an automatedand robust framework for finding 3D object-of-interestwithin cluttered and noisy point clouds. Our search en-gine is efficient in isolating objects from cluttered laserscans and it is robust to the challenges discussed pre-viously. A simple and abstract representation of theframework is illustrated in Figure 2. The frameworkdeveloped is capable of addressing some of the majorresearch challenges discussed previously (e.g., density,noise, and incompleteness). The method takes advan-tage of existing 3D models integrated with the BIM. Themodel-based 3D object-of-interest finding framework isdescribed in the following section.

3 METHODOLOGY

An overview of the implemented methodology for find-ing arbitrary shapes within cluttered point clouds is il-lustrated in Figure 3. It is derived primarily from Papa-zov and Burschka’s (2010) basic algorithm and adoptedto the class of construction object recognition problemsaddressed here. The results are then extensively exam-ined for performance. The method has three primarysteps: (1) model library generation, (2) scene represen-tation, and (3) matching. The first step can be performedin the offline phase, meaning that the library can begenerated and stored for further calculation. The sec-

Fig. 3. Proposed methodology for BIM-based object findingof construction assemblies has three major steps: (1) model

library generation, (2) scene representation, and (3)matching.

ond step is to calculate features for hypotheses testedin the matching step (Step 3). The primary steps forfinding arbitrary objects are described in the followingsections.

3.1 Inputs and preprocessing

The required inputs for the proposed algorithm are thefollowing:

(1) 3D Model denoted by M : in order to generate themodel library the 3D model should be availablein the point cloud format. The solid objects ex-isting as the CAD drawings integrated with theBIM are converted to 3D point clouds using oneof the methods well discussed by Corsini et al.(2012). Poisson disk sampling is used in this workfor converting 3D solid objects into point cloudmodels.

(2) 3D point cloud or the Scene denoted by S: thatrepresents the as-built state or the scene beinginvestigated. Both M and S are preprocessed byconstructing their weighted octree structures. Binsubdivision in weighed octree is calculated basedon the mean of all points that each bin con-tains whereas in the normal octree subdivisionssplits bins at their central coordinate (i.e., one binsubdivides into 8 equally sized bins). This stepis required to normalize the density of the in-put point clouds. Moreover, octree represents auniformly resampled point cloud resulting fromthe original input point cloud. Such a process issimilar to voxelization for down sampling or re-sampling a 3D point cloud. A hypothetical ex-ample of weighed octree construction of a typi-cal point cloud (Model and Scene) is illustrated inFigure 4.


(a)

(b)

(c)

(d)

Fig. 4. Density normalization using weighed octree. A modelin the point cloud format (a). A model after density

normalization using octree (c). A zoomed-in window isshown for illustrating density before (b), and after (d) density

normalization using octree structuring.

Fig. 5. Local feature descriptor used for object extraction.The distance between the point pair is set constant. This

assumption reduces the level of complexity and thereforereduces the processing time for feature space creation.

3.2 Model library generation

This step can be performed beforehand, because it re-mains unchanged for a given shape or geometry. Inother words, a library of objects can be created andstored in a database for further processing. The fea-ture space used in this article is similar to the fea-ture defined in Papazov and Burschka (2010). Thelocal feature set used in this work is illustrated inFigure 5.

As illustrated in Figure 5, a 3D local feature descrip-tor is used to represent the model. The feature set for apoint pair (p1, p2) is denoted by F(p1, p2) and is calcu-lated as follows:

F(p1, p2) = ( f1, f2, f3) (1)

in which, f1 = ∠(n1, d), f2 = ∠(n2, d), f3 = ∠(n1, n2).The operator ∠ returns the angle between the two in-put vectors, and d is the distance between the pointpair (Figure 5). In contrast with the previous work byNahangi et al. (2016), the feature set in this frame-work uses a constant distance between the point pairs.The assumption of reducing one dimension from the lo-cal feature set makes it computationally less intensiveand therefore more time effective. Moreover, removalof the distance element from the feature set, results insimilar dimensionality for the remaining elements, andtherefore it reduces the complexity of the feature space.This distinctive feature is useful in storing the featuredescriptors more efficiently. All points in the modelare uniformly resampled and the feature set is thencreated.

The key for calculating the feature descriptor is thenormal vector at a resampled point cloud. The nor-mal vector is calculated using a four-step algorithm asfollows:

(1) Calculate k-nearest neighbors (KNN) given apoint in a point cloud (p ∈ P) using kd-tree.

(2) Assign the calculated neighborhood to the point p.(3) Fit a plane to the neighborhood.(4) Assign the plane’s normal vector to the point p

(n.pi ).

The k value for identifying the size of the neighbor-hood around a point will affect the accuracy of normalvector calculation and therefore the isolation retrieval.The framework has been found very robust to the sizeof the neighborhood for normal vector calculation. Theprocedure for normal vector calculation takes advan-tage of principal component analysis (PCA). More de-tail about normal vector calculation can be found in Cz-erniawski et al. (2016a).

A 3D hash table is used to store the library of fea-tures. The feature elements F =( f1, f2, f3) are used tohash the entries in the table. The hash table is dividedwith an arbitrary cell size, which is found to have anegligible impact on the robustness of the framework.The calculated feature sets are then assigned to thecorresponding cell in the table. This method has beenfound very efficient for the search phase, and thereforeimproves the time-related aspects of the framework.Figure 6 shows the creation of a hash table for a hypo-thetical 3D shape.

To find various models, the hash table and the modellibrary can be extended; that means the features forvarious models can be accumulated in an original hashtable. This method for storing the model feature setsin the same library avoids recalculating features for


Hash Table

(a) (b)

Fig. 6. A hypothetical 3D shape is illustrated to show theprocedure for generating the model library and storing thefeature descriptor. (a) Arbitrary point pairs are resampled

from the model point cloud and the feature set is calculated.(b) The feature sets are then used to hash the table for

representing the model (i.e., points with similar feature setswith a threshold value �θ are hashed in a similar cell).

previously generated shapes. Model library generationwith the required steps is summarized in Algorithm 1.

3.3 Scene representation

Once the models are described using the feature setexplained, and the library of the objects are created,the online mode is performed. The online mode startswith a RANSAC (Random Sample Consensus)-basedsearch. The RANSAC search for 3D objects is ineffi-cient and almost impossible for realistic or practical-sized point clouds. In contrast, the feature library isused for making the search more efficient and robustfor real-sized point clouds. The sampling process for theRANSAC-based matching is illustrated in Figure 7.

To start searching for potentially matching pairs, theScene (S) is uniformly resampled (s1). For each pointresampled, all points {s2} that are distanced at d arestored to create a potential pair (s1, s2). The feature

Hash Table

Step 1

Step 2

Hash

Step 3 Step 4 Step 5

Fig. 7. Scene representation for one typical iteration in theRANSAC-based matching algorithm. Step 1: An arbitrarypoint s1 is selected. Step 2: All possible s2s are calculated.

Step 3: All possible pairs (s1, s2) are created, and the featuresare calculated. Step 4: Potential matching pairs are extractedfrom the hash table using the features calculated. Step 5: The

transformation T is then calculated.

elements are then calculated to identify the matchingpairs from the Model (M) stored in the hash table. Thepoint set (s1, s2) and the corresponding normal vectors(n.s1, n.s2) are then matched with the existing pairs andnormal vectors in the hash table. In other words, the po-tential matching pairs from the Model and Scene createa hypothesis to be tested in a RANSAC-based match-ing step. The matching step is described in the followingsection.

3.4 Matching

The matching step is combined with the Scene represen-tation. The matching step is an iterative process basedon the criteria defined for testing the hypotheses cre-ated from S. For all potential matching pairs identifiedfrom the hash table, the transformation (T ) that alignsthe features (i.e., points and their normal vectors) fromthe Model and Scene is calculated. For calculating thetransformation T , PCA (principal component analysis)is used for aligning two sets of four points in the Modeland the Scene. The set of four points include the twopoints of the pair being investigated and two arbitrarypoints located on the normal vectors starting from thepoints. In this article, the two points are located at theend of the unit normal vector starting from the point.Figure 8 illustrates the calculation of the point sets offour to be matched from the two data sets.

The method described for transformation calculation(hypothesis generation) is found to be robust and quick.


Set of four points from the Model (M) Set of four points

from the Scene (S)

Fig. 8. Calculation of the point sets to be matched usingprincipal component analysis. The four points include the

pairs as two points: (m1, m2) or (s1, s2), and two points,(m3, m4) or (s3, s4), located at the end of a unit normal vectorstarting from each point. ‖m1 − m3‖ = ‖m2 − m4‖ = 1 and‖s1 − s3‖ = ‖s2 − s4‖ = 1. The rigid transformation T can

then be calculated using PCA.

The entire Model is then transformed using the previ-ously calculated transformation: M∗ = T × M .

M∗ and S are then compared to test the hypothesisgenerated. For this purpose and to test the goodnessof the transformation calculated (hypothesis), the num-ber of inliers is computed. A support term (λs) is there-fore defined to investigate the appropriateness of thehypothesis. In other words, λs identifies an additionalcriterion for the RANSAC-based matching algorithmused here.

For each hypothesis (T ) generated, the support termis calculated as λs(M, T ) = ms/m, where, ms is the num-ber of points that support a matching criterion, and m isthe size of the Model M . Such a matching criterion isdefined as the number of points from the transformedModel (M∗) in close proximity to the Scene (S).

The matching step is performed until either a maxi-mum number of iterations is reached or a pre-definedportion of the points in the Model are retrieved fromthe scan. These two criteria are identified to stop theRANSAC-based hypothesis testing framework.

Once the best hypothesis is found using the previ-ously explained framework, the match is refined usinga post iterative closest point (ICP) alignment. The ini-tial coarse alignment will be improved using a post-ICP alignment with a few number of iterations to findthe best match between the two data sets. This stepincreases the accuracy of the method to find the ob-ject of interest. This is due to the resampling operationwhich improves the effectiveness and reduces the timeof the framework. Some information is missing duringresampling because of the reduction in the density ofthe point cloud; however, this issue is compensated us-

Table 2Values of the effective parameters for the set of experiments

performed

Parameter Description Value

d Distance between the point pairs 0.75ρa

�θ Cell size for the hash table 12°λs Overlap ratio for the RANSAC 0.15t Time criteria for RANSAC 20 sIterations ICP iterations for post refinement 5

aρ = diam(M), where diam returns the largest distance between apair in a point set.

ing this post-ICP refinement. Algorithm 2 summarizesthe processing tasks explained in Steps 2 and 3.

The effective parameters are summarized in Table 2and were established using experiments in this study.The effectiveness of the parameters is reported in Sec-tion 4.3.

4 PROOF OF CONCEPT

In this section, the described framework is implementedand its performance is measured by designing a set ofexperiments. The method is tested on two cases to eval-uate its capability on various geometries and shapes.The framework is implemented and programmed in aMATLAB-based platform integrated with C++ and afunction library distributed by Papazov and Burschka


Table 3Summary of physical properties of the 3D scanning device

(FARO LS 840-HE)

Factor Value

Accuracy ± 3 mm at 25 mScanning range 0.6–40 mAcquisition speed 120,000 points/sAngular resolution 0.009°

(2010). The processing times reported in the followingsections are benchmarked on a processing machine witha 3.7 × 12 GHz processing unit and a 32 GB RAM.

4.1 Design of experiments

For the proof of concept, a set of experiments are de-signed and performed. The experiments are carried outon a small-scale pipe spool (as a curvilinear object) anda structural frame (as a rectilinear object). The object-of-interest is in a laboratory environment, where otherunwanted objects are scanned in the background or inthe close proximity of the object-of-interest. For 3Dpoint cloud acquisition, laser scanning is employed inthis study. A FARO LS 840-HE is used for scanningthe lab facilities. Physical properties of the laser scan-ner used in this study can be found in Table 3.

For comparing the results in the cases investigated,and measuring the performance of the framework, twocritical metrics are reported:

(1) Processing time is the required time for bothmodel library generation (offline phase) andmatching (online phase). Tracking the processingtime enables the applicability of such a methodfor developing real-time frameworks for processcontrol.

(2) Retrieval accuracy is the average error betweenthe transformed Model and the Scene. This isrepresented by a root mean square (RMS) ofthe Euclidean distance between the correspondingpoints.

As mentioned earlier, two construction componentsare used in the experiments: (1) a small-scale pipespool and (2) a small-scale structural frame. The latteris chosen to verify the robustness of the proposed al-gorithm for finding structural elements from clutteredlaser scans. Previous studies (Czerniawski et al., 2016a)were directed toward recognizing cylindrical objects(e.g., pipe spools). However, the method presented inthis article can find any shape and geometry robustly

Table 4Construction components used in the experiments

Pointcloud

Boundingdimensions

Elementtype

ID usedfor results

Pipespool

3 m × 2 m × 0.5 m Curvilinear PS

Boxframe

3.5 m × 2 m × 2 m Rectilinear BF

and effectively. More information about the compo-nents used in the experiments is provided in Table 4.

4.2 Effective variants

To investigate the capability of the framework for ad-dressing the existing challenges for efficient finding ofobjects (discussed earlier), the experimental setup istested under various circumstances. Three major vari-ants are investigated: (1) density of the 3D point cloudused in the isolation framework, (2) clutter existing inthe Scene, and (3) completeness of the object-of-interestin the 3D point cloud acquired. A wide range of suchvariants is considered and their impact on the verifica-tion metrics (processing time and retrieval accuracy) isanalyzed in the following sections.

4.2.1 Density. For investigating the effect of density onthe results, a dimensionless metric is defined. The met-ric is called density ratio, which is the proportion ofthe number of points in the Scene to the constant num-ber of points in the Model. Various density ratios areinvestigated by down sampling the originally acquiredpoint cloud (Scene). Down sampling is performed in-crementally to evaluate how the recognition rate is af-fected. Another metric is also defined to monitor therecognition rate. Recognition rate (RR) is defined asfollows:

RR = TP‖s‖ (2)

in which the nominator TP (True Positive) is the num-ber of truly found points, and the denominator ‖s‖ isthe size of the object-of-interest (s) in the Scene (s ⊆ S).Tables 5 and 6 show the summary of the analyses for theeffect of density on the recognition rate. Figure 9 showstypical results using various density ratios for the isola-tion of PS from cluttered point clouds.

For both cases, the RMS values remain relatively un-changed for the successfully isolated objects. This showsthat the RMS value remains unchanged for the success-ful cases. In other words, the negligible change in theRMS value implies that the object-of-interest isolated


Table 5Summary of the analyses on the effect of density on the

critical metrics for pipe spool

Densityratio

Number ofpoints RR RMS (cm) Time (s)

0.3 3,000 0 Failed Failed1 10,000 0 Failed Failed2 20,000 0 Failed Failed5 50,000 0.956 1.34 27.58 80,000 0.930 1.28 26.8

10 100,000 0.929 1.31 25.120 200,000 0.924 1.25 19.2

Table 6Summary of the analyses on the effect of density on the

critical metrics for box frame

Densityratio


0.3 3,000 0 Failed Failed1 10,000 0 Failed Failed2 20,000 1 2.92 15.85 50,000 0.995 2.86 22.9

10 100,000 0.979 2.82 22.916 160,000 0.983 2.81 22.9

from the laser scan is robustly identified using variousdensity ratios. Another observation is that the recog-nition rate (RR) increases as the density ratio passes aminimum threshold value and it then remains relativelyunchanged.

As shown in Tables 5 and 6, for lower density ratios,the isolation of the object-of-interest is unsuccessful.Unsuccessful isolation means that the final transforma-tion found by the algorithm is incorrect and the isolatedpoint set does not correctly correspond to the object-of-interest. This might be due to the over simplificationof the scan occurring during the down sampling phase.Down sampling may also cause inaccuracies in the cal-culation of normal vectors. As explained previously, theaccuracy of the normal vector calculation step is a keyin the recognition and isolation process. Therefore, forlower density ratios, the object may not be representedsufficiently densely, which fails accurate normal vectorcalculation, and consequently, the object isolation givena cluttered point cloud.

The processing time reported in Tables 5 and 6 is thetime required for the alignment of the 3D model withinthe point cloud. As noted, the processing time is sta-ble for some cases, because the processing time is thecritical metric for stopping the RANSAC-based search.Because RANSAC is a random process, it is possibleto get a successful match with a lower processing timein another run of the program; however, for the casesreported in this article, the processing time remains sta-ble. The isolation time, which requires nearest neighborcalculation, and a post-ICP refinement is excluded fromthe processing time reported in the results. The postpro-cessing time for calculating the closest points and refin-ing the match is expected to be exponentially increasingas the density of the point cloud increases (Rusinkiewiczand Levoy, 2001).

(a) (b) (c)

Failed matching

Fig. 9. Effect of density on the accuracy of the object-of-interest isolation. Three density ratios are illustrated. Density ratios are20 for (a), 5 for (b), and 1 for (c). The top figures are cluttered point clouds with the model aligned with the object-of-interest, and

the bottom figures show the isolated object from the point clouds.


(a) (b) (c)

(d)(e)(g)

(f)

Fig. 10. Typical results for the effect of clutter on the accuracy of the object extracted from 3D point clouds. A cluttered scene isinvestigated in five stages: (a) the scene is fully cluttered, (b) background is removed, (c) some obviously unwanted objects areremoved (structural components), (d) planar clutter (ground, walls, and ceilings if any) is removed, (e) secondary and support

attachments (holder jacks and stands) are removed. (f) The isolated object from manually cleaned point cloud. (g) The isolatedobject from fully cluttered point cloud.

4.2.2 Clutter. For investigating the effect of clutter onthe object recognition and isolation framework, the ex-perimental objects are tested under various circum-stances with varying clutter. The 3D point cloud used asthe Scene is tested with an incrementally increased clut-ter around the object-of-interest. For quantifying theamount of clutter in the S, clutter ratio is defined as:

Clutter ratio =∥∥S − (S ∩ s)

∥∥

‖s‖ (3)

in which S is the 3D point cloud (Scene), and s isthe object-of-interest. In other words, clutter ratio isthe proportion of the amount of clutter to the sizeof the object-of-interest (Figure 10). The clutter ratiois dimensionless. The recognition rate (RR), as definedpreviously, is then calculated for each clutter ratio. Ex-perimental results for the PS and BF objects are sum-marized and reported in Tables 7 and 8.

As seen in Tables 7 and 8, the recognition rate de-creases as the clutter ratio increases. It signifies thatclutter presence affects the accuracy of the isolated ob-ject from a point cloud; however, the object is still suc-cessfully and robustly found. The RMS value of theisolated object is calculated for various clutter ratios.For both objects, the RMS values remain relatively un-changed. This signifies that the isolated object is ro-bustly isolated for the various clutter ratios.

Table 7Summary of the analyses on the effect of clutter on the

critical metrics for pipe spool

Clutterratio


51.295 322,191 0.915 1.30 20.912.191 81,272 0.929 1.28 13.27.466 52,156 0.974 1.28 15.31.473 15,236 0.985 1.28 8.10.707 10,514 0.984 1.28 0.8

Table 8Summary of the analyses on the effect of clutter on the

critical metrics for box frame

Clutterratio


20.794 322,191 0.916 2.81 17.417.604 279,072 0.925 2.81 15.89.579 158,692 0.903 2.81 13.25.087 91,302 0.920 2.83 12.13.053 60,796 0.967 3.69 10.9

As seen in Tables 7 and 8, recognition rate decreasesas the clutter ratio increases for both cases. This mightbe due to the existing noise in the scene. However,


the object (3D model) is successfully aligned within thepoint cloud, and it is therefore successfully isolated fromthe scene. The level of recognition rate achieved evenin the most cluttered case in the experiments is suffi-ciently reliable for enhancing further assessments on theisolated object. Such further assessments include qual-ity control, deviation analysis and discrepancy quantifi-cation (Nahangi et al., 2015; Nahangi and Haas, 2014;Bosche et al., 2015). Figure 10 shows a typical exam-ple of the effect of existing clutter on the isolation ofthe PS object. In this case, clutter is gradually removedmanually, and the framework is applied. Figure 10ashows the fully cluttered point cloud (original scan) andFigure 10e shows the least amount of clutter existingaround the object-of-interest (PS object). Figures 10fand g show the final results after the recognition andisolation framework is applied.

4.2.3 Completeness. In order to investigate the effect ofcompleteness (partial occlusion) on the isolation frame-work, various combinations of the comprising branchesand elements of the investigated objects are tested. Thedesired pipe spool to be isolated from the point cloud iscomprised of multiple branches. Branches are manuallyremoved from the input point cloud (S) to test the ca-pability of the framework for recognizing and isolatingthe object, in the case of incomplete and missing data.Incomplete data might be due to occlusion or the line-of-sight during data acquisition. Completeness ratio isdefined as the metric to quantify completeness vs. in-completeness of the data, in other words, the percent-age of the object that has been occluded. Completenessratio is the proportion of the size of the object in the im-ported point cloud, to the size of the completely scannedpoint cloud of the object-of-interest. It should be notedthat density and clutter ratios are kept unchanged whileincompleteness is being investigated. Rather than therecognition rate, success rate is calculated for measur-ing the effect of incompleteness. Success rate (SR) is abinary metric (1 if successfully isolated and 0 if isolationis failed). Tables 9 and 10 show the effect of incomplete-ness on the success rate for recognizing and isolating theinvestigated objects.

Table 9Effect of completeness on isolating pipe spool from a

cluttered laser scan

Completeness ratio Number of points SR Time (s)

0.840 7,384 1 22.80.822 7,231 1 22.90.539 4,743 1 23.10.798 7,021 0 Failed

Table 10Effect of completeness on isolating box frame from a

cluttered laser scan

Completeness ratio Number of points SR Time (s)

0.943 19,569 1 20.20.917 19,046 1 20.10.908 18,858 1 19.70.724 15,024 1 190.569 11,799 1 19

As reported in Tables 9 and 10, a threshold in thecompleteness ratio must be met in order to ensure rec-ognizing and isolating the objects successfully. Figure 11illustrates how various branches are manually removedfrom the imported point cloud into the recognitionframework. Various branch removal results in differentcompleteness ratios that affects the success rate in therecognition framework.

4.3 Parameters effectiveness

The effective parameters reported in Table 2 were es-tablished using experiments in this study. However, twoscenarios were identified that caused the finding algo-rithm to fail using the proposed parameters. The twoscenarios are (1) multiple objects being recognized and(2) failure to detect object of interest. In order to re-solve the first issue, λs was increased so that only theobject with the maximum overlay percentage would re-main as the isolated object. In the second case, the re-quired time for RANSAC algorithm was increased upto 40 seconds and in further cases λs was also reduced.To measure the effectiveness of the proposed parame-ters, an effectiveness ratio (ER) was defined. Effective-ness ratio was defined as the proportion of the times thatthe object of interest was isolated using the proposedset of parameters to the total number of times the ob-ject of interest was successfully isolated from the pointcloud. Tables 11 and 12 illustrate the results of using theproposed parameters. Results show that the proposedparameters are sufficiently robust to be integrated withthe automated finding algorithm.

5 CONCLUSIONS AND FUTURE WORK

A model-based 3D object recognition and isolationframework (Papazov and Burschka, 2010) was adaptedand examined experimentally to extract the construc-tion elements of interest from cluttered laser scans.The framework was desired to be sufficiently robustand therefore reliable to be integrated with automated


(a)(b)

(c) (d)

Fig. 11. Object recognition and isolation with incomplete and missing data. The point cloud in the middle shows a completelyscanned object. Each branch is manually removed in four different steps and the capability of the algorithm developed is tested

under missing and incomplete data. The object recognition only fails in (d) because the removed branch contains critical featuresin finding the correct transformation. In cases (a), (b), and (c), object recognition and isolation is successful.

Table 11Effectiveness ratio for the proposed parameters on isolating

pipe spool from a cluttered laser scan

Parameter Value ER

λs 0.15 0.92t 20 s 0.92

Table 12Effectiveness ratio for the proposed parameters on isolating

box frame from a cluttered laser scan

Parameter Value ER

λs 0.15 0.91t 20 s 0.91

and integrated construction process controllers. Insummary, this method employs a local feature of pointpairs as a descriptor or a local signature. The method-ology for 3D object recognition and isolation has threeprimary steps:

(1) Model library generation for the elements exist-ing in the building information model. The libraryof objects and their describing features calculatedare stored in a hash table that enhances an efficientand quick search.

(2) Scene representation by calculating the featuresfor potential point pairs and testing hypotheses ina RANSAC-based hypothesis testing engine.

(3) Matching and refining by transforming the 3Dmodel on the acquired point cloud and refining thematch by a post-ICP registration step.

An experimental study is performed for two differentconstruction objects: a pipe spool (PS) as an MEP com-ponent and a box frame (BF) as a structural element.Density, clutter, and completeness are thoroughly in-vestigated to test the robustness of the framework. Pro-cessing time and recognition rate are recorded as theverification metrics in the various cases are tested andinvestigated. Some interesting observations and insightsof the experimental study are listed as follows:

� It was shown in the experiments that if a thresh-old value is met as the required level of density, theobject-of-interest is robustly isolated from the pointcloud.

� It was demonstrated that even with a cluttered pointcloud, the algorithm is capable of extracting the ob-ject from the clutter surrounding it; however, thenumber of incorrectly found (false positive) points in-creases inevitably, as the clutter increases. It there-fore requires a finer post refinement for the removalof such points.


� The algorithm also works in cases that an incompletepoint cloud is imported (Scene) to find and isolate theobject-of-interest. This capability addresses the un-avoidable occlusion challenge on the data acquisitionphase.

The framework examined in this article can be usedto find a wide range of curvilinear and rectilinear con-struction components and elements, in contrast with theprevious methods that were focused on some specificand explicit geometries. Because the feature set used torepresent an object is not limited to an explicit geom-etry, it can even extract very complicated geometriesincluding sophisticated connections and surfaces. Thiswas verified and validated by testing two relatively so-phisticated geometries within various construction sec-tors (i.e., MEP and structural elements).

As the framework is robust to the density and com-pleteness of the point cloud acquired to represent thescene, there is an emerging potential for integratingthe framework with image-based 3D point cloud tech-niques. Currently, inadequate number of images orinsufficient level of overlap between the images arethe sources of inaccuracies in the image-based andstructured-light based techniques for 3D point cloudgeneration. However, because the developed frame-work is robust to incompleteness and density of thepoint clouds used, such inadequacy might be bypassed.Moreover, considering that the utilization of image-based frameworks for data acquisition is less expensivecomparing to the laser-based techniques, it is impor-tant to explore the integration of the framework withimage-based or structured-light based sensors in futureresearch.

Although the recognition and localization of the 3Dmodel is performed in a significantly faster time frame,the isolation module takes the dominant part of thetime required for processing. Faster and more effectivesearch strategies such as graph theory may improve theprocessing time for the isolation, and this inadequacymay be appropriately addressed. This could be a po-tential research direction for future work. In the casethat the isolation module is effectively utilized and theprocessing time is reasonably quick, the entire frame-work may be integrated with structured-light based dataacquisition sensors for the development of (near) real-time process controllers. Such integrated platforms arecurrently being developed by the authors.

Future work will also include verifying and testing theperformance in case deviations exist in an assembly. Itis anticipated that the method performs robustly withslightly deviated sections in an assembly. This will pro-vide the opportunity to quantify the deviations in defec-tive segments.

ACKNOWLEDGMENTS

The authors would like to thank Rick McEwen, StaceyJensen Rose, and Scott Waters from Aecon, Canada,for their invaluable insights and feedback on this work.This research is partially funded by Natural Science andEngineering Research Council (NSERC) Canada, andAecon Industrial West (AIW), Edmonton, Canada.

REFERENCES

Ahmed, M., Haas, C. & Haas, R. (2014), Automatic detectionof cylindrical objects in built facilities, Journal of Comput-ing in Civil Engineering, 28, 04014009.

Aiger, D., Mitra, N. J. & Cohen-Or, D. (2008), 4-points con-gruent sets for robust pairwise surface registration, ACMTransactions on Graphics (TOG), 27(3), #85, 1–10.

Allaire, S., Kim, J. J., Breen, S. L., Jaffray, D. A. & Pekar,V. (2008), Full orientation invariance and improved featureselectivity of 3D SIFT with application to medical imageanalysis, in Proceedings of the IEEE Computer Society Con-ference on Computer Vision and Pattern Recognition Work-shops, IEEE, Anchorage, AK, 1–8.

Balali, V. & Golparvar-Fard, M. (2015), Segmentation andrecognition of roadway assets from car-mounted cam-era video streams using a scalable non-parametric imageparsing method, Automation in Construction, 49(Part A),27–39.

Bosche, F., Ahmed, M., Turkan, Y., Haas, C. T. & Haas, R.(2015), The value of integrating Scan-to-BIM and Scan-vs-BIM techniques for construction monitoring using laserscanning and BIM: the case of cylindrical MEP compo-nents, Automation in Construction, 49, 201–13.

Bosche, F. & Guenet, E. (2014), Automating surface flatnesscontrol using terrestrial laser scanning and building infor-mation models, Automation in Construction, 44(0), 212–26.

Bosche, F. & Haas, C. T. (2008), Automated retrieval of 3DCAD model objects in construction range images, Automa-tion in Construction, 17(4), 499–512.

Chaumette, F. & Hutchinson, S. (2006), Visual servo control.I. Basic approaches, IEEE Robotics & Automation Maga-zine, 13(4), 82–90.

Chen, J., Fang, Y., Cho, Y. K. & Kim, C. (2016), Principalaxes descriptor for automated construction-equipment clas-sification from point clouds, Journal of Computing in CivilEngineering, 04016058.

Chua, C. S. & Jarvis, R. (1997), Point signatures: a new rep-resentation for 3D object recognition, International Journalof Computer Vision, 25(1), 63–85.

Corsini, M., Cignoni, P. & Scopigno, R. (2012), Efficient andflexible sampling with blue noise properties of triangularmeshes, IEEE Transactions on Visualization and ComputerGraphics, 18(6), 914–24.

Czerniawski, T., Nahangi, M., Haas, C. & Walbridge, S.(2016a), Pipe spool recognition in cluttered point clouds us-ing a curvature-based shape descriptor, Automation in Con-struction, 71(Part 2), 346–58.

Czerniawski, T., Nahangi, M., Walbridge, S. & Haas, C.(2016b), Automated removal of planar clutter from 3Dpoint clouds for improving industrial object recognition,


Proceedings of the International Symposium on Automationand Robotics in Construction, Auburn, AL.

Dimitrov, A. & Golparvar-Fard, M. (2015), Segmentationof building point cloud models including detailed archi-tectural/structural features and MEP systems, Automationin Construction, 51, 32–45.

Dimitrov, A., Gu, R. & Golparvar-Fard, M. (2016), Non-uniform B-spline surface fitting from unordered 3D pointclouds for as-built modeling, Computer-Aided Civil and In-frastructure Engineering, 31(7), 483–98.

Dissanayake, M., Fayek, A. R., Wolf, H. & Van Tol, A.(2003), Developing a standard approach for construc-tion field rework assessment, in Proceedings, ASCE, Con-struction Research Congress: Wind of Change: Integra-tion and Innovation of Construction, Honolulu, HI, 1–8,https://doi.org/10.1061/40671(2003)13.

Golparvar-Fard, M., Pena-Mora, F. & Savarese, S. (2012), Au-tomated progress monitoring using unordered daily con-struction photographs and IFC-based building informationmodels, Journal of Computing in Civil Engineering, 29,04014025.

Goodrum, P. M., Miller, J., Sweany, J. & Alruwaythi, O.(2016), Influence of the format of engineering informationand spatial cognition on craft-worker performance, Jour-nal of Construction Engineering and Management, 142(9),04016043.

Guo, Y., Bennamoun, M., Sohel, F., Lu, M. & Wan, J. (2014),3D object recognition in cluttered scenes with local surfacefeatures: a survey, IEEE Transactions on Pattern Analysisand Machine Intelligence, 36(11), 2270–87.

Guo, Y., Wan, J., Lu, M. & Niu, W. (2013), A parts-basedmethod for articulated target recognition in laser radardata, Optik—International Journal for Light and ElectronOptics, 124(17), 2727–33.

Huang, F. J., Boureau, Y. & LeCun, Y. (2007), Unsupervisedlearning of invariant feature hierarchies with applicationsto object recognition, in Proceedings of the IEEE Confer-ence on Computer Vision and Pattern Recognition, 2007,IEEE, Minneapolis, MN, 1–8.

Johnson, A. E. & Hebert, M. (1998), Surface matching for ob-ject recognition in complex three-dimensional scenes, Im-age and Vision Computing, 16(9–10), 635–51.

Kim, C., Kim, C. & Son, H. (2013), Automated construc-tion progress measurement using a 4D building informationmodel and 3D data, Automation in Construction, 31, 75–82.

Kim, M., Wang, Q., Park, J., Cheng, J. C., Sohn, H. & Chang,C. (2016), Automated dimensional quality assurance of full-scale precast concrete elements using laser scanning andBIM, Automation in Construction, 72, 102–14.

Knopp, J., Prasad, M., Willems, G., Timofte, R. & Van Gool,L. (2010), Hough Transform and 3D SURF for robust threedimensional classification, European Conference on Com-puter Vision, Springer, 589–602.

Lee, J., Son, H., Kim, C. & Kim, C. (2013), Skeleton-based3D reconstruction of as-built pipelines from laser-scan data,Automation in Construction, 35, 199–207.

Nahangi, M., Czerniawski, T., Haas, C. T., Walbridge, S. &West, J. (2015), Parallel systems and structural frames re-alignment planning and actuation strategy, Journal of Com-puting in Civil Engineering, 30, 04015067.

Nahangi, M., Czerniawski, T., Rausch, C. & Haas, C. (2016),Arbitrary 3D object extraction from cluttered laser scansusing local features, in Proceedings of the International

Symposium on Automation and Robotics in Construction,Auburn, AL, 15 pp.

Nahangi, M. & Haas, C. T. (2014), Automated 3D compliancechecking in pipe spool fabrication, Advanced EngineeringInformatics, 28(4), 360–69.

Nahangi, M. & Haas, C. T. (2016), Skeleton-based discrep-ancy feedback for automated realignment of industrial as-semblies, Automation in Construction, 61, 147–61.

Papazov, C. & Burschka, D. (2010), An efficient RANSAC for3D object recognition in noisy and occluded scenes, AsianConference on Computer Vision, Springer, 135–48.

Park, H. S., Lee, H. M., Adeli, H. & Lee, I. (2007), A new ap-proach for health monitoring of structures: terrestrial laserscanning, Computer-Aided Civil and Infrastructure Engi-neering, 22(1), 19–30.

Park, M. & Brilakis, I. (2012), Construction worker detectionin video frames for initializing vision trackers, Automationin Construction, 28, 15–25.

Patraucean, V., Armeni, I., Nahangi, M., Yeung, J., Brilakis,I. & Haas, C. (2015), State of research in automatic as-builtmodelling, Advanced Engineering Informatics, 29, 162–71.

Quigley, M., Batra, S., Gould, S., Klingbeil, E., Le, Q. V.,Wellman, A. & Ng, A. Y. (2009), High-accuracy 3D sensingfor mobile manipulation: improving object detection anddoor opening, ICRA, Kobe, Japan, 2816–22.

Rabbani, T., Dijkman, S., van den Heuvel, F. & Vosselman,G. (2007), An integrated approach for modelling and globalregistration of point clouds, ISPRS Journal of Photogram-metry and Remote Sensing, 61(6), 355–70.

Ray, S. J. & Teizer, J. (2012), Real-time construction workerposture analysis for ergonomics training, Advanced Engi-neering Informatics, 26(2), 439–55.

Rusinkiewicz, S. & Levoy, M. (2001), Efficient variants ofthe ICP algorithm, in Proceedings of the Third Interna-tional Conference on 3-D Digital Imaging and Modeling,Quebec City, Quebec, Canada, 145–52.

Sharif, M., Nahangi, M., Haas, C., West, J. & Ibrahim, M.(2016), A preliminary investigation of the applicability ofportable sensors for fabrication and installation control ofindustrial assemblies, Resilient Infrastructure, London, On-tario, Canada.

Son, H., Kim, C. & Kim, C. (2014), Fully automated as-built3D pipeline extraction method from laser-scanned databased on curvature computation, Journal of Computing inCivil Engineering, 29, B4014003.

Statistics Canada (2010), Construction data. Available at:http://www.statcan.gc.ca/pub/11-402-x/2011000/chap/cons-truction/construction-eng.htm, accessed July 2016.

Tang, P., Huber, D. & Akinci, B. (2011), Characterization oflaser scanners and algorithms for detecting flatness defectson concrete surfaces, Journal of Computing in Civil Engi-neering, 25(1), 31–42.

Turkan, Y., Bosche, F., Haas, C. T. & Haas, R. (2012), Auto-mated progress tracking using 4D schedule and 3D sensingtechnologies, Automation in Construction, 22(0), 414–21.

Zhang, G., Vela, P. A., Karasev, P. & Brilakis, I. (2015), Asparsity-inducing optimization-based algorithm for planarpatches extraction from noisy point-cloud data, Computer-Aided Civil and Infrastructure Engineering, 30(2), 85–102.

Zhu, Z. & Brilakis, I. (2010), Concrete column recognition inimages and videos, Journal of Computing in Civil Engineer-ing, 24(6), 478–87.

Automated Model-Based Finding of 3D Objects in Cluttered ...static.tongtianta.site/paper_pdf/3dd10f9a-3766-11e9-b891-00163e08bb86.pdfAutomated model-based ﬁnding of 3D objects in

Documents