How can today's advanced technology solve the challenges that many organizations face after obtaining vast 3D point cloud datasets, including the management, storage, registration, fusion and extraction of useful and actionable information?
Instruments for digitizing the 3D real environment are becoming smaller, more lightweight, lower-cost and more robust. Accordingly, they are finding increasingly widespread usage, not only on surveying tripods for the highest accuracy, but also on mobile platforms such as autonomous vehicles, drones, helicopters, aircraft, robotic vacuum cleaners, trains, mobile phones, satellites and Martian rovers. Lidar uses laser scanning, while photogrammetry records images from one or more cameras which may be moving. Each laser scan records tens of millions of data point positions and colours in a point cloud, and hundreds of such point clouds may be combined. This article discusses the challenges that many companies and organizations face after obtaining vast 3D point cloud datasets, including the management, storage, registration, fusion and extraction of useful and actionable information.
Cloud computing
The first challenges users face in performing 3D point cloud data processing include:
Data Storage: The amount of data recorded grows exponentially with time, creating large data repositories.
Processing: The computing power required increases as new algorithms with useful functionality are released and with the volume of data.
Sharing: There are multiple stakeholders spread geographically around the world on mobile platforms who all need to view the most up-to-date data at the same time.
Previously, a software application ran on a dedicated server in a data centre but, if the computer hardware broke down, the user either had to find a backup (which had to be standing by and ready) or would suffer an interruption in service. Many companies guarantee a 24/7 level of service and so cannot tolerate this. However, Cloud Computing now gives users access, over a network, to applications running on a set of shared or pooled servers in a globally communicating network of data centres, giving speed and productivity improvements, resulting in increased competitiveness.
Figure 1: 30 Terrestrial laser scans of a central London library, fully automatically aligned using the Vercator software.
Big data analytics
Users face the difficult challenge of how to boil down the vast amounts of 3D point cloud data to generate useful and actionable information. Current methods for creating Digital Twin BIM models of buildings require users to inspect vast 3D point clouds to manually recognize and mark the outline positions of surfaces, straight edges, walls, floors, ceilings, pipes, and objects, which is time-consuming and susceptible to error. Some semi-automatic methods on laptops require users to recognize and mark part of these and the program finds the rest. Again, such objects can be mislabelled. Fully automatic methods are becoming available on laptops but do not find all the useful information, so users must add and correct what is found. Sometimes the automatic method makes so many mistakes it is quicker for the user to find and mark the structures manually.
“Useful information” in one application may be different from that in another application. For example, in autonomous vehicles, it is an accurate 3D terrain model which can be used for safe navigation. In electricity pylon scanning, it is whether the pylon has its safety warning sign in place clearly visible and whether nearby vegetation is gradually encroaching on the power lines. In railway scanning, it is whether there has been any slippage or sag as well as an estimate of when gradually encroaching vegetation will become a hazard. Electricity supply companies and Network Rail are under UK government obligations to regularly inspect their assets and to perform preventative maintenance to ensure continuity of supply and travel.
Geometrical object recognition
Correvate has developed a suite of machine learning geometric image processing methods for fully automated basic object recognition – walls, floors (figures 2 and 3), edges (figure 4) and pipes (see figure 5).
See below for the rest of the article.
Figure 2: Automatic Wall and Floor Recognition in a recently poured concrete shell of a building under construction in London (16 aligned scans).
Figure 3: Automatic Wall Recognition in a recently poured concrete shell of a building under construction in London (16 aligned scans).
Figure 4: Automatic Edge Detection followed by fitting of straight-line segments in UCL circular/octagonal library under the iconic central dome (21 aligned scans)
Figure 5a (top), Pipe scan and 5b (bottom), Automatic Pipe Recognition in a Boiler Room 3.5 million point cloud; 98% cylinders correctly found (2 aligned scans red and blue).
Artificial Intelligence
Artificial neural networks are extremely simplified models of living brains, which are trained and learn like people rather than being programmed by a master programmer. The learned knowledge or skills are stored in a distributed manner in the strengths or weights of the neuron interconnections. Some artificial neural networks learn on their own while others require a teacher or instructor to tell them when they are right or wrong. Gradually, they get better and better at performing a task during the iterative learning cycles which usually take a long time and require thousands of examples of the training data. Artificial neural networks are particularly good at recognition, classification and optimization tasks. However, their performance depends crucially on how they are trained, the types and the amount of training data. Many types of neural network have been developed and, most recently, Convolutional Neural Networks (CNN) used to perform Deep Learning have become very popular and achieve very good results. In the case of object recognition, if the neural networks are only trained with examples of objects one wants to find, then all input data will be classified as one of those objects, even if it is not one of those objects. So, the performance of the neural network is only as good as the way it was trained and the data that was used to train it. Neural networks are not as new as you might imagine given their current popularity in the media. Over 30 years ago, Selviah (1989) proved that the weighted interconnection layer of neural networks performs the same operation as a collection of correlators, operating in parallel, matching images from a database with input data and then the non-linear part of the neurons decide which image matches the input most closely. The clever part is the way in which the training automatically works out what images to store in the database in the first place.
In the conference room image, figure 6, you see the impressive recognition results after training a new type of CNN with data from the Stanford Large-Scale 3D Indoor Spaces Dataset (S3DIS) using 70,000 3D objects of 13 types, structural objects: ceiling, floor, wall, beam, column, window, door, and movable objects: table, chair, sofa, bookcase, board and clutter in 11 types of room. Each category of object is marked in a different colour, for example, ‘chairs’ are marked in yellow, ‘boards’ are marked in orange, ‘beams’ are marked in red, ‘door’ is marked in green, ‘walls’ are marked in dark green, ‘floor’ is marked in blue, etc. The accuracy of classification of objects is around 93.5% comparable to human accuracy. The objects to be recognized can be chosen for each application simply by changing the training database.
Figure 6: AI Automatic 3D object recognition. Plan view of original point cloud data for a conference room and 3D recognised objects. The ceiling was removed for clarity in viewing the inside of the room.
Artificial intelligence in the cloud As the AEC sector embraces digital technology, the amount of data produced grows exponentially, creating large data repositories. To generate useful and actionable information from this ‘big data’ requires leveraging smart analytical tools such as AI that are becoming more accessible, especially when hosted from the cloud. Both the cloud computing infrastructure and artificial intelligence supply the tools to leverage and enable digital technology by providing convenient methods of working at scale, thus lowering the barriers to entry for users to these new ways of working. Artificial intelligence (AI) neural network and deep learning require vast databases of thousands of examples for training, which can be conveniently stored in elastic expandable cloud storage on demand. AI software requires highly parallel processing on many parallel processors to carry out the training in a reasonable time, again easily available in cloud computing infrastructures.
Intelligent combination and use of available techniques such as laser scanning, automatic alignment, cloud computing and artificial intelligence can not only speed up analysis of vast data sets but also improve accuracy and release human activity to ensure that a product is correct and useful.
BENEFITS OF USING THE CLOUD Auto-Application Updates: applications are updated automatically, so the user always has access to the most up-to-date optimised software and bug fixes. Responsivity: dedicated development support teams continuously monitor user experience to optimise and, if necessary, rewrite code. Scalability, flexibility and agility: Scalable elastic cloud environments on pools of servers, storage and networking resources scale up and down according to the number of users and the volume of their usage. They automatically scale up and down as users’ needs change. Capital expenditure free: users have access to the highest power computers. There is efficient use of hardware as users do not need to purchase, manage and maintain large amounts of computer and storage hardware, resulting in lower hardware, power, cooling and IT management costs. Users only pay for what they use as the cloud resources automatically scale, so it is easier for small businesses to manage their business at any time of day, from anywhere. High speed: multiple computers run in parallel so many different parts of the same point cloud can be processed at the same time and many different users have no effect on speed or quality. Security: the data is stored and communicated securely with a level of encryption chosen by the user. If security is a paramount concern, the software can run on a private cloud without internet connections in-house. Clouds can be configured to make use of certain data centres, such as within one country if intercountry security is a concern.
Availability: if one server is busy or not available then another server takes its place to provide full availability. Disaster Recovery: data is stored in multiple locations at the same time so if storage hardware in one data centre breaks down, the calculation proceeds with little interruption as the data is backed up elsewhere. Data archiving facilities are automatically provided. Latency: if latency is important, the cloud can be configured so that local clouds provide low latency to the user. Increased collaboration: many users, located globally, and mobile users, can store, process, share and view datasets at the same time without any loss of speed or responsivity. Reliability: the application software can make use of resources on cloud computing infrastructure provided by different vendors in different global regions. Forward compatible: an open cloud architecture is forward compatible to match higher power computing resources as they are rolled out. Sharing: all point cloud datasets are secure in one place and accessible at any time from
Author: David Selviah
Last updated: 04/08/2020
In the wake of Industry 4.0, many companies have tried to utilise automation and data exchange in manufacturing technologies. This is especially prevalent in the construction industry where the need for increased efficiency and delivering a quality product both, physically and digitally has now become a necessity rather than an indulgence. Many technologies have sprung up to meet the challenge, such as artificial intelligence (AI) and drones.
Call it a drone, Unmanned Aerial Vehicle (UAV), Unmanned Aerial System (UAS) or Remote Piloted Aircraft System (RPAS), it usually involves a flying platform that is remotely controlled by a pilot assisted by flight software, onboard sensors and Global Positioning System (GPS) / Global Navigation Satellite System (GLONASS). It has a payload which is usually a camera system, but could also be technologies such as LIDAR (Light Detection and Ranging) and thermal cameras. There is shared telemetry between the drone and the ground control station which enables the pilot to fly in a stable manner.
The numbers
DroneDeploy, a cloud software platform for commercial drones has compiled statistics on drone usage based on 100 million aerial images from 400,000 job sites in 180 countries in 2018. Below are some of the findings:
• The construction industry has seen an increase of 239 percent in the adoption of drone technology. The other two industries directly related to construction, namely surveying and real estate, have an increase of more than 100 percent for each industry.
• There are many benefits that are associated with the use of drones in construction, namely increased safety, cost saving and better data collection and usage.
• Drones are primarily used for progress tracking and communication, preconstruction and site planning, quality control and assurance, bid process preparation and job site risk mitigation.
55 percent of DroneDeploy customers report increased safety as a result of implementing drones.
The why
Accenture indicated in their article titled, “A business approach for the use of drones in the Engineering & Construction industries” that drones “optimise project and maintenance costs”. In the current business climate, being able to optimise project costs and maintenance costs is crucial for the viability of a business.
A drone allows for tasks to be automated and conducted in parallel with operations. Inspection can now be done while the construction is being undertaken. Various drone platforms now allow for automated drone operations that provide vital information for construction, such as cut and fill parameters, volumetric analysis of stock piles, comparison against design data and conducting accurate and repeatable topographical surveys.
Drones also “reduce workers exposure”. Health and safety are a crucial element on all construction sites. Drones allow access to dangerous areas (working at height, chemical exposure, heat exposure) which were previously deemed as high risk to personnel and cost intensive. For example, an inspection of a rooftop would utilise scaffolding and harnesses, which take time and effort to setup, whilst a drone could capture a wealth of data in a fraction of the time and cost.
But that’s not all as drones also “enable best decisions to improve quality”. The key element in the progression of technology is the ability to provide humans with better information to be able to make better decisions. A bird’s eye view of a live construction site allows for accurate decision making based on real-time information as opposed to relying on narratives and benchmarks. Another element that drones bring is that data collected can be reviewed for lessons learnt, comparison for benchmarking and general archiving.
The who and where
Many construction companies around the world have started using drones as a vital tool in their projects. International companies such as Kier, Balfour Beatty, Vinci Construction and Mitie have started to use drones as a tool on site. On the Malaysian front, projects like the Tun Razak Exchange (TRX) and various rail projects such as the Mass Rapid Transit (MRT) extension extensively use drones for project monitoring and various land use analysis. It is becoming more and more commonplace in Malaysia for drone operation in data collection and analyses.
Drones are perfect for all manner of construction, engineering and inspection projects as they provide the ability to work in an automated manner and collect data that allows for better decision making.
The when
Now is the time for companies to make the shift towards the many benefits that come with responsible drone operations. There are two ways to achieve this. The first is to hire a professional drone company that complies with the various regulatory requirements. Some discussion is usually required at the start to ensure the deliverables are in line with the need of the construction project.
The second option is to develop drone capabilities in-house to the company. The best way to achieve this is to hire an external consultant that could guide you through the process, thus speeding up the time to setup a competent drone team. The consultant will guide you through the process including purchasing, operation manual setup, audits, maintenance plan, training and software selection.
The next step
Drones will continue to improve and become commonplace in many industries. With AI starting to move into the drone space, the amalgamation of these two cutting edge technologies will produce a quantum leap in useable data that will help reduce cost, increase safety and maximise performance. The construction industry has to maintain a view of the future which will certainly include the use of drones, so as to ensure that it remains relevant and competitive in this ever-changing world.
*This article was contributed by 27 Advisory and Aerial Ascent. The 27 Group is a 100 percent Malaysian owned local consulting firm that is fast, flexible and focused with unique expertise that blends local socio-economic policy settings, global engineering-built assets and detailed financial analyses. Read more about 27 Group’s consulting services at 27.group. Aerial Ascent is a drone specialist company.
27 Advisory and Aerial Ascent
20 December 2019
Unmanned aerial vehicles (UAVs or 'drones') are enabling construction professionals to access more data and detailed insights to create the reliable, sophisticated design models that are needed to stay ahead of the competition.
It’s no secret that ensuring construction projects remain on time and in budget is crucially important for all stakeholders and demand continues to grow for faster, more readily available and up-to-date information. The goal? To optimise operational productivity and efficiency during projects and help the industry evolve and grow. Indeed, reports claim that if construction productivity were to catch up with the global economy, the sector’s value would increase by an estimated $1.6 trillion — and add approximately 2% to the global economy.
Recent changes in legislative measures are driving change across the industry, however. Technological advances, such as the Building Information Modelling (BIM) framework, have significantly improved the way in which construction companies share information and data on a project and help to future-proof their operations. The introduction of unmanned aerial vehicles (UAVs), or drones, has also further facilitated this change. Drones have enabled construction professionals to access to more data and detailed insights to create the reliable, sophisticated design models that are needed to stay ahead of the competition.
Why drones?
Drones are increasingly well recognised in the construction space as an important tool for collecting valuable data and insights during the construction process. Unlike traditional topographic methods, drones can cover large areas quickly and easily and generate detailed 3D imagery of the project site and surrounding area. For instance, carrying out a drone survey in the initial stages of a project can offer important benefits for both architects and customers, who can use the drone data to navigate the site in 3D before any construction starts. This can be invaluable for capturing current conditions and making subsequent design changes. Increasing the number of data points on a survey can also add significantly more accuracy to the planning phase. In road construction projects, for example, drones have proved efficient at making more precise stockpile material calculations. This data can help civil engineers to plot volumes of earth that require moving, as well as inform site managers of the number of resources needed, such as people, vehicles, supplies and services. By building a better picture of these requirements in the early stages, there is also an opportunity to save valuable costs and time across the lifespan of a project.
In addition to the extensive mapping and surveying prior to the start of construction, drones can also be deployed to assess ongoing activity and provide real-time insights for a broad range of stakeholders. Acting as a reliable monitor for construction site progress, a UAV can track progress and provide visibility regularly to the whole team, including surveyors and quality controllers. This data can be crucial for ensuring work remains on-track and allows project managers to implement any required changes as they arise, rather than carry out extensive rebuilding later. Together with tools such as Conflict Management, drone insights are helping to add further data points to design models, allowing for algorithms to automatically detect any errors in construction as they arise. Detecting potential issues soon as possible in the process is crucial to ensuring seamless workflows and reducing the cost impact to projects in the event of any necessary re-works.
Drones in action
As drones become increasingly integrated in construction projects, they can also help to provide improved accountability during the development process. Since a UAV can monitor sites daily, there can be plenty of documentation made available to refer to, should any errors be made. This accurate track of activity can help to monitor and prevent potential mistakes – ensuring quality, responsibility and effect cost-management throughout the project.
In addition to monitoring quality, UAVs have become an important tool for site management and improving safety processes, which remains a key issue in the industry. Thanks to their aerial data capture functionality, drones are proving key to minimising the need for high numbers of on-the-ground staff when mapping an area, helping to reduce the volume of health and safety hazards on active construction sites. Indeed, drones can navigate high-risk areas and monitor material, equipment and even people, to form detailed insights that can identify potential safety issues. For instance, ahead of deliveries of large-scale materials, UAVs can first check the clearance and positioning of certain equipment, check access road slopes and simulate the position of cranes to ensure safer, more seamless operations.
UAV technology also recently helped to add value to the mapping of an expansive construction site at a mine in Arizona spanning 630 acres, with highwalls reaching over 2,000 ft tall. The site was inaccessible using traditional methods, but using a drone allowed the team to monitor the pit walls for movement and any potential sloughing or deterioration. Using drone technology also enabled site traffic to continue as normal, without the need to temporarily shut down operations to allow surveyors access.
The right fit
Innovation in drone technology has meant that fixed-wing UAVs are increasingly the hardware of choice for mapping larger, more expansive construction sites. The latest designs are capable of longer flight times, more range and improved mission capability, ensuring they are optimised for flight efficiency.
Advances in UAV technology have also seen fixed-wing drones become equipped for Beyond Visual Line of Sight (BVLOS) operations, which offers particular benefits in construction applications. Thanks to improved motors and safe airframes, which have been verified through rigorous impact testing, construction professionals can safely fly further, beyond their line of sight, and map larger areas to gain more data and insights in a shorter time. In addition to offering cost, time and operational efficiencies, this also opens up exciting opportunities in the construction space by paving the way for more complex operations. Greater public acceptance and scalability, facilitated by fit-for-purpose regulatory frameworks, will be key to making this a reality.
Although fixed-wing drones are already widely recognised for their benefits in mapping large, open spaces, they are a growing area of interest for construction professionals operating in dense, more urban environments. While adoption can be much slower, due to the stringent regulations surrounding, for instance, operations over people (OOP) and restricted airspaces like airports, when correctly managed UAVs bring valuable data from smaller sites in urban applications and their benefits are being increasingly recognised as legislation moves more in line with technology advances. Indeed, as UAVs become more widely implemented across the globe, it is likely that drones will be able to integrate more effectively with other airspace traffic, allowing a greater range of applications in the construction sector in the future.
Forward-thinking solutions
Fixed-wing drones present a cost and time effective option for construction companies, however to get even more detailed insights, at a higher resolution, professionals are increasingly exploring options with drone fleets, including both rotary and fixed-wing devices, which offer their own unique benefits. Rotary devices do not require a substantial landing and take-off area and can therefore benefit from improved manoeuvrability. Thanks to their ability to hover at close proximity to areas of interest, these drones are also ideal for operations where close proximity to an object is required. With several drones working together as part of a fleet, large quantities of data can be collected more quickly and enable teams to be more proactive. This can be particularly useful when working across multiple projects.
Aerial surveying methods can also be complemented with integrated software solutions, allowing users to both accurately map terrain and analyse the data afterwards. For instance, leading flight planning and management software allows teams to assess the feasibility of the project prior to each flight, so they can plan their route more effectively. Flight boundaries can also be outlined in advance, enabling take-off and landing zones to be pre-determined, which helps to increase efficiencies when in the field. By creating a link between drone and software, construction professionals not only facilitate the handling of data, but also ensure output is as fast as possible, a crucial factor in boosting project efficiency and profitability.
Furthermore, processes such as BIM, are only as good as the data that’s collected. By providing large amounts of reliable data from UAVs, including orthomosaics, digital surface models (DSMs) and detailed point clouds, stakeholders – including architects and site managers – can have access to powerful insights that offer a more up-to-date model of the site and add real value. This is crucial to enabling informed, data-driven decisions, and by utilising drone solutions that offer greater absolute accuracy – even without the use of ground control points (GCPs) – construction professionals can obtain valuable, reliable insights through highly accurate geo-rectified imagery quickly, efficiently and safely. By having access to 3D build models that are more detailed than ever before, together with collaborative tools like BIM, designs can be easily updated and modified. BIM workflows also continue to be simplified and optimised thanks to ongoing advances in platforms used to process drone data, which can now be rapidly uploaded, processed and made available on the cloud for additional time efficiency benefits. In short, drones present an opportunity to make the BIM model more efficient – data capture, processing and sharing has never been simpler, optimising workflows and ensuring the entire team are on the same page.
The right direction
UAV technology is rapidly gaining ground in the construction sector, but the need to integrate new tools can mean that uptake is often still slower than desired. While there is still knowledge to be gained, examples of how drones have helped improve workflows are helping to make this change a reality and broaden the opportunities available for constructions companies looking to build or expand their drone fleet. UAVs offer a cost-effective solution to help streamline operational efficiencies throughout the entirety of a construction project, by increasing profit margins and improving safety. By changing how data is made available and shared on construction projects, it is further driving innovation across the industry — and offering a range of possibilities for the future of mapping in the construction space.
Author: Benjamin Pinguet
28/07/2020