From Flight Plan to Model and Back – Data Acquisition 2.0

The future of drone inspection workflows is automation at a level that requires accurate 3D environment data – the same kind of data that is produced in a drone inspection. A chicken and egg problem? Not quite.

Almost all drone workflows involve the generation of a 3D digital model of the inspected asset. Be it a 3D mesh, or a point cloud, these models comprise an intuitive, visual and relatively compact digital summary of the asset. Put simply, these models allow us to make sense out of the piles of image data that are collected around the asset – bringing them one step closer to providing real value through business intelligence.

As a side effect, asset owners start accumulating such 3D models of their assets from previous inspections. Although they may be somewhat outdated, or not centimeter-accurate, these data points are also the best existing digital representation of the asset and their environments, and hence the best basis for autonomous data acquisition.

Planning missions around a cell tower 3D model obtained from previous inspections

Gaining the Fourth Dimension by Closing the Loop

Some call them 4D workflows, or 4D engineering. Others call them 4D construction modelling. They are workflows that strive to maintain a time-referenced sequence of 3D models of an asset, that enables not only spatial insight into the state of the asset at a given moment in time, but also on its progression in the course of the construction, or maintenance cycle. The additional temporal dimension surfaces incremental changes in the assets’ condition, thus enabling early identification of deviations from plans, early detection of faults and errors and more – all having tremendous benefits for quickly evolving assets, or assets with short maintenance cycles.

The caveat, though, is that such workflows require an order of magnitude more accurate raw data, as the asset site needs to be captured numerous times in the span of the project. A data capture process that is not automated will be neither cost-effective, nor sufficiently repeatable to guarantee the quality of the resulting 4D workflow.

The solution is to close the data capture-photogrammetry loop, namely to replace the linear disconnected workflow in which the classical inspection blueprint: data capture-photogrammetry-reporting is repeated independently numerous times, with a workflow in which old photogrammetric data from previous instances of the inspection are fed back into the data acquisition engine to enhance the automation and improve the overall efficiency and repeatability of the process.

Mission planning based on 3D model of a church created by Drone Harmony partner Softdesk.pl

Bootstrapping the Process

The vast majority of assets that will be digitized in the upcoming years still lack accurate 3D mesh, point clouds or CAD models that could be used to perform data acquisition. So, where can the initial shape and environment information come from?

The answer depends on a variety of variables, such as the available GIS data around the asset (accuracy of geo-referenced maps, digital elevation models, CAD drawings etc.), the assets’ geometric complexity and more. The more supporting data is available, the more the data acquisition process can be automated.

Capturing a dam without available GIS data: Bootstrapping with a 3D box model world.

• 

  3D orthoreferenced model overlaying

• 

In some cases the available data and the existing tool set in Drone Harmony’s data capture platform already enables automated data capture (e.g. using Drone Harmony’s Cell Tower Scan for communication towers). In other cases, a simple survey flight can be generated in Drone Harmony to first construct a simplified model of the environment. This model can then be used to automate the data capture workflow for the actual inspection. For over a year, Drone Harmony customers have successfully employed this two-stage workflow in the field of terrain surveying. With upcoming features, this workflow will be possible in a variety of new use cases.

The hidden drone data revolution

In summary, when it comes to drones, the data revolution has a hidden dimension that is often ignored. We all enjoy thinking about the amazing new perspective that drones provide on the world around us through their unprecedented maneuverability, but on the flip side of the coin, for the most part, drones are blind. To bring real value at scale, they need a reliable guidance system that is aware of the structure of the world around it – the same kind of data they themselves are set out to provide. This hidden drone data revolution will drive the ever growing ensemble of environment data back into the brains that guide the drone data capture workflows, increasing automation and reaching unprecedented echelons of value generation for large enterprises. Drone Harmony is working hard to make this happen.

5 new things you can do with Drone Harmony Cloud right now

Drone Harmony Cloud has just been released. In this post we list the five most important things it enables right now.

1. Automatic Synchronization. Drone Harmony customers never need to worry about maintaining and managing their Drone Harmony states and missions, as this is now done automatically. With the release of Drone Harmony Cloud, saved states on either the mobile device, or the web platform are automatically archived and pushed to the cloud (either immediately, or the next time internet connection is available). Furthermore, every time you enter the load state menu, the up to date state information is pulled from the Drone Harmony Cloud, guaranteeing that your mission data is always up to date, wherever you are.
2. Visualize missions in any browser. Load any Drone Harmony state in the web interface to visualize scenes and missions on the computer screen. A new visualization of camera headings of all waypoints in 3d enhances your understanding of the planned drone actions along the flight path.
3. Edit missions in 2d and 3d. The web interface allows you to easily edit scenes and missions by using the comfort of a computer screen and mouse. New edit options in 3d enable easy and intuitive editing of vertical missions.
4. Create simple scenes and missions in the web. Define scene objects, and create manual waypoint missions in the web interface (Automatic generation of missions is still only available in the mobile application, but will soon be made available on the web as well).
5. Tag and Archive your missions. Add tags to easily sort, organize and navigate through all stored missions. Saved states are automatically archived to maintain a history of your organizations’ data capture activities. Easily load the latest version of a state, or any archived version.

Visualize, edit and manage Drone Harmony States – from the comfort of your web browser

This is just the beginning. Drone Harmony Cloud is evolving quickly into Drone Harmony’s main mission planning tool. To learn more about what Drone Harmony Cloud will enable in the near future, read “Introducing Drone Harmony Cloud“.

Do you have ideas for improvement? A feature you would like to see? Post your ideas on our idea platform and interact with us and other Drone Harmony users on our forum.

Introducing Drone Harmony Cloud

Drone Harmony Cloud is about to become the focal point of your organizations’ data capture operation. Want to know what you can do with Drone Harmony Cloud today? Read “5 new things you can do with Drone Harmony Cloud right now“.

The Drone Harmony Cloud platform has been released. With it, Drone Harmony’s customers gain two new dimensions to their data capture workflows: cloud storage of mission data and a web-based planning interface. Combined with Drone Harmony’s mobile application, Drone Harmony is now offering a fully integrated data capture platform for drone service providers and enterprises.

The Drone Harmony web application

Automated sharing and collaboration

Automatic cloud synchronization

A key capability introduced by the launch of Drone Harmony Cloud is the sync feature. The sync feature automatically consolidates and creates backups in the Drone Harmony Cloud of all saved mission planning data (including the scene, missions, flights, logs, etc.) across all mobile devices associated with the Drone Harmony account. Furthermore, a team collaborating on a site can have immediate access to all the missions planned by peers. The benefit for the manager is the ability to see an up to date snapshot of all the data acquisition activities in the comfort of the office, without the need to task her pilots to manually submit a report or export the Drone Harmony state. In short, the automatic sync feature is the automated archive for your organizations’ data acquisition activities.

Plan, edit and visualize – from any web browser

With the release of Drone Harmony Cloud, Drone Harmony’s customers gain access to the Drone Harmony web platform, a powerful new interface for exploiting Drone Harmony’s technology. The web application makes it very easy to interact with most of Drone Harmony constructs. With the aid of a large screen and a mouse, customers can now easily visualize and edit stored scenes and missions in ways that were not possible before. The web platform also offers some manual mission planning capabilities. In the near future, all automated planning features from Drone Harmony’s mobile application and a number of new features will be ported to the web platform as well.

The web platform allows flexible editing of missions in 2- and 3d

A platform for drone data acquisition

Drone Harmony Cloud is much more than just another mission planning tool. It is a platform for organizing and managing all the information that is relevant to your organizations’ data acquisition activities.

Drone Harmony Cloud enables tagging and archiving of missions for better organization of the data acquisition workflow

The platform addresses the needs of larger enterprises performing drone data acquisition, providing the enterprises’ operation management a birds’-eye view of all asset sites while enabling seamless sharing and collaboration capabilities with operators. Smaller organizations can keep a history of their mission planning activity, perform comparisons and obtain an organizational single point of reference.

Upcoming updates of Drone Harmony Cloud will include the possibility to simultaneously visualize on the map groups of missions to gain further insight into ongoing and completed data capture activities.

What’s coming

Drone Harmony is hard at work on turning the cloud platform into the focal point of all drone data acquisition efforts for its customers.  In particular,  while on-site mission planning on the mobile device will remain a key capability, the aim is to position the web interface as the primary mission planning tool in the future. As a first step, all mission planning features from the Drone Harmony mobile application will be migrated to the web application. In later iterations, users can expect the web applications’ plan catalog to include some unique advanced features.

In consequence, our customers can expect the future generations of the web platform to include improved versions of Drone Harmony’s full 3D planning interface, new advanced cloud-based mission planning capabilities, the possibility to store advanced scene data (point clouds, meshes etc.) as well as drone data in the Drone Harmony Cloud and integrations of key services and products in the drone and GIS space.

Just like our mobile application, Drone Harmony Cloud is a fast-evolving product that is designed to address Drone Harmony’s customers’ data acquisition needs. We encourage you to post your questions and feedback on our idea platform and interact with us and other Drone Harmony users on our forum.

Drone Harmony 2019 – an outlook

In 2019 Drone Harmony will continue to push boundaries of automation. As new markets start to adopt drone technology at scale, the need for repeatable, robust workflows will drive the drone technology market.

Demand for process automation is higher than ever before and this trend is only likely to accelerate as more demand for drone data will come from complex use cases and environments.

The role of your data

A drone is an excellent tool for generating data around a vertical  infrastructure asset, such as a cell tower, or a building. Whether it is consumed raw out of the camera, or undergoes additional transformation (3d modelling, report generation etc.) to generate business intelligence, the quality of this data determines the value the drone-based digital workflow has for the business.

This same pivotal role that drone data plays is the reason why Drone Harmony takes a broader approach to the role of data in automating drone workflows. This approach stems from the inevitability that to generate quality drone data automatically, you have to use accurate 3d data about the asset and its’ environment, just like self-driving cars need to have a great understanding of the shape of the road and other vehicles on the road to navigate safely. The sources of such data range from maps and DEMs to full meshed or CAD models for the asset. In fact, the drone data itself and its derivatives, captured perhaps at earlier time, can serve as an excellent 3d representation for the purpose of mission planning.

In 2019 Drone Harmony will continue to improve the value for its customers by further automating the mission planning workflow through the seamless integration of 2- and 3d data platforms in the planning workflow.

Drone Harmony is expanding to new platforms

As Drone Harmony transitions from being a mission planner to becoming a data platform directly interacting with both down- and upstream pieces in the digital workflow, it becomes necessary to expand the scope of computational platforms that run it. Therefore, we are excited to announce that early 2019 will see the release of Drone Harmony’s web-based desktop application. The web-based application will not only introduce Drone Harmony’s Customers to a new interface for mission planning, but will also comprise a powerful computational platform enabling a myriad of new capabilities designed to generate better drone data in a robust and reliable way.

A final prediction

We predict that 2019 will be a breakthrough year for the industrial drone market, in which many new industries will include drones at scale in their digital workflows. This transition will be triggered by increased robustness of drone-based workflows, driven by unprecedented strides in process automation, in which Drone Harmony will play an important role.

2018 has been an exciting year for Drone Harmony and our active community. Read about the main highlights in our recent article Drone Harmony, 50,000 autonomous flights later – 5 lessons that we’ve learned.

50,000 flights later – 5 lessons that we’ve learned

By the end of 2018, Drone Harmony’s users have logged over 50,000 autonomous flights. This milestone could be achieved due to our teams’ commitment to both our vision and our customers’ valuable feedback. So what have our users taught us? In this article, we summarize some of the most valuable lessons we’ve learned.

1. When it comes to drone operations, you can never automate enough

Drone Harmony has made it its mission to make drone data acquisition a one-click operation, even in complex industrial use cases. When our customers start using Drone Harmony, they immediately appreciate its added value. But they also realize the potential of automation and begin to envision how additional manual, error-prone aspects of their work can be automated. Drone Harmony is working vigilantly to make this vision a reality, as every new benchmark reached represents a new breakthrough in the potential of drone-based workflow to scale and reach new markets.

2. The value of automation is in the quality of the data

What is the single most valuable aspect of automating mission planning and flight execution to our customers? It turns out that it is not the reduced costs, risks and time on site. While all the latter are important advantages, the true value stems from the fact that autonomous flights guarantee high quality, complete, non-redundant data every single time.

3. Customers want to use their data

Digital data about assets and environments is available in a variety of forms, both on a global scale, and on databases belonging to individual organizations and assets. In most cases, customers are more than willing to use this data to simplify their drone operations. The most common forms of such data include DEMs, CAD, and meshed shape data and aerial imagery. Drone Harmony is working hard on integrating new data formats into the Drone Harmony mission planning framework to enable our customers to make good use of this data.

Customers want to use their data about the asset to enable autonomous drone workflows

4. Simplicity is a requirement, not a feature

Drone mission planning is a complex task, especially in the difficult environment of an industrial asset. It does not mean, however, that an automation tool can afford cutting corners on simplicity and intuitiveness. Needing to do 10 clicks when it is possible to achieve the same with 5, is just an unnecessary source of errors, especially when performed in the filed, under time pressure and unfavorable weather conditions. We have thus learned to always rethink what we consider to be “easy” and “intuitive”.

5. Operators only really believe their eyes

What cannot be visualized convincingly is a source of uncertainty and anxiety for the operator. There is a level of trust that an operator needs to have in a tool that operates her 10k$ drone. The more the tool is able to visualize the planned behavior, as it will play out in the 3d environment, the more confidence will the operator have in the tool, and the more manual control will she be willing to give up and achieve the benefits of automation.

The importance of visual feedback

We are learning new things every day from our community. We are grateful to all the customers that are sharing their opinions and ideas with us. We are listening and doing our best to make your wishes for Drone Harmony come true in 2019.

Partnership with DroneLogbook

Drone Harmony has partnered with DroneLogbook.

DroneLogbook is an industry leader for automating UAV log and document management for commercial drone compliance. In addition, DroneLogbook provides tools to manage and track pilot currency, fleet management, detailed maintenance tracking, flight operations and more. As your business scales DroneLogbook can grow and scale with your business even providing a private label version.

In conjunction, Drone pilots will benefit from the two seemlessly integrated flight planning and flight analysis platforms.