​Creating meaning from drone data

We need to first start with the types of drone datasets we are talking about. Specifically, unstructured geospatial data. To understand unstructured data, we need to define structured geospatial data. Structured geospatial data has a standardized way to relate it to the real world. This means the user can understand where in the world it is, what measurement units are being used, and at what scale and projection it is representing the location. For example, a map can contain a combination of lines for roads (vectors), elevation information (TINs), and orthophotography for a background map (rasters). All three examples have a well-defined structure that includes all the information you need to understand its location in the world. This allows us to have a common way to relate other information as needed.
Unstructured data on the other hand does not contain these characteristics. A still frame picture by itself is not great for geospatial processing. It lacks a lot of important information such as how the camera lens distorts the objects in the scene (internal orientation) and where and at what angle it was taken (external orientation). With drones, we can add this information but even then, it’s difficult for an algorithm to answer the two most important questions about still imagery which is: what it is and why is it important. To answer these questions, we need to give it more context, a relationship with our structured data.

Connecting unstructured data to GIS

GIS is built to answer what, why, and most importantly, where. We structure our databases to answer these things. A database can easily tell you what something is and provide you with the data that helps you understand why it is important as well as how those things relate to other information. The modern world runs on relational databases. Behind most of the things you do day after day, there is a database that is driving the processes including tracking your internet shopping or booking your vacation.  GIS is a database containing geometry. Geometry tells the user “where” something is. This adds a whole new level of analysis. Now, not only can we look at the relationships between the information inside the database, the table, but we can also look at the relationship of where that object is, the geometry. For example, if you needed to know all the distribution poles installed before 1970 within 20 ft of a major road, you can easily do that. That is the power of GIS
So, what does drone imagery have to do with GIS? When we link the two together, we create an opportunity to communicate about something that is both easy for human interpretability (the image) and structured for workflow management and data analysis (GIS). This creates a snapshot in time that can be shared with others. It also allows us to ask complex questions both on the drone and in the office. It allows us to collectively pull our subject matter expertise and remove much of the communication barriers that exist and gives us context for better decision making.
For complex connected workflows to work, this is the first connection required. We need to make data intelligence with drones a two-way process. This is fundamentally different than workflows of the past. In the future analysis and intelligence will be conducted both in the field and in the office. Not only will users gather information from the drone, but they will also get information from it.

The cloud, 5G, and data connectivity 

In the past, our data was stored in a building behind a firewall inaccessible from outside the building. These days, its more and more common to have that data accessible with different devices in different locations. This “Cloud” connectivity has opened the ability to use this information in different ways. For example. GIS was always analyzed and manipulated using desktop computers. With cloud connected data accessible through a user portal, that information can now be served up to mobile applications where new information can be created and data is integrated directly into field processes.
To have a connected world, you need to have data connectivity. It is difficult to understand the impact 5G will have. 5G will make our data connectivity up to 10 times faster. That means instead of taking an hour to share lifesaving information from the field, it can be done in just over 5 minutes. Quick connectivity from the field to the office is the key to make these workflows possible.  
Just as important as getting raw information back to the cloud, it’s important that we can get answers back to the field crews. Context can be the differentiator between a decision that saves millions and a life-threatening choice. For example, if we can get field collected information about an active fire, we can use cloud processing to analyze real-time information to determine the most likely locations that the fire will spread next and we can give that information back to the field personnel in minutes so that they can make informed decisions to take action.

Making it easier for humans 

At the end of the day, the technology needs to serve one purpose and that is to make things easier for humans. Computers and algorithms can do some things very well, but we should not underestimate the processing power of the human brain and a lifetime of experience. We believe that technology should not be designed to replace humans. It needs to be designed to empower them to do their jobs with greater ease, efficiency, and information. So how do you build a geospatial data management and inspection system to do that? We believe the answer is Augmented Reality.
Augmented Reality (AR) is the most intuitive way to serve up mapping information. Instead of having to correlate what you are seeing on the live video with your mapping. Your mapping shows up exactly where it should on the live video. This is an example of making it easier for humans. They can now spend less time thinking about what and where something is and more time thinking about what needs to be done. Adoptability is the most difficult task with technology. We want to remove this barrier by making it intuitive and easy to use. This is key in a connected workflow.

Robots working for you 

A big step towards using drones for intelligence and analysis is data analytics. Data analytics from drones are only as good as the answers they provide and a person’s ability to act on those answers. One of the major challenges with Artificial Intelligence (AI) from drones is how to both integrate the drone into the workflow but also how to create a pipeline and process of using the information created from them?
The truth is that computers are a long way away from the processing power of the human brain and most likely will never get there. What is a much more likely scenario is using AI to help answer easy questions reliably and quickly to help people make decisions quicker. For example, AI can help sense and avoid to keep the drone from striking an object and falling out of the air, but it’s still not to the point where it can identify suspicious activity while monitoring a location. These types of use cases require both experience and human subjectivity.
However, AI can be tremendously beneficial with drone data. Certain techniques can be used to extract meaningful information from imagery such as a bent utility pole, a water leak, or the size of a crowd in an image. Using AI to give human decision makers information quickly is an extremely powerful tool.

Connecting all the dots 

So how do we begin to put all the pieces together and connect the dots? This is our goal at BirdOne. To create meaningful connectivity with these technologies we need to create a human in the loop process that are both simple and usable. The technology should not be the focus for workers, the processes they drive should. At the end of the day its about getting work done.
As these technologies advance, we need to understand the potential of connecting all these pieces together to create tremendous value that improve our capabilities as humans, reduce risk, and increase our efficiency. We are living at a pivotal time where all the pieces are coming together to allow technology to be a tremendous servant of good in the world. We at BirdOne welcome that world where humans are empowered to be an everyday superhero with tools and technology working for you.
 ​

Our infrastructure is no different. Proper preventative maintenance can keep things from falling and as we all know, when infrastructure fails, the results can be unimaginable. Failures to inspect have resulted in a huge loss of life as well as costly and time-consuming repairs. Several major fires in the United States started because of improperly maintained right of ways where vegetation was allowed to grow into the powerlines. After a major disaster like these, we typically do a post-mortem and find out that these events were preventable if only we knew about the issues in advance. The question then becomes, how do we find out, AND most importantly, how do we catalog and manage these issues?
We have been using GIS to manage these issues for a long time. It is a major part of any asset management program. We send out field personnel, they perform their routine inspections and preventative maintenance, we catalog those efforts in our GIS using some proprietary tools or something like Esri’s Survey123 or Collector. When we are done with our inspections, we sync that information back to our GIS database that is accessible by the rest of our team using a dashboard, a portal, or directly through the server. All the information is in one place, with a properly structured system for managing the work that is performed every day. This is a well-established way of doing things that we have been doing for a long time that works great (most of the time).
So, what happens when things are not easy to access for inspections? What happens when accessing those areas becomes dangerous such as a utility pole or transmission tower? Wooden utility poles specifically often rot at the top first. That type of rot is difficult to inspect from the ground. To see that traditionally, you would need to climb that pole to get a look. Doing so is dangerous and time consuming not to mention there are over 185 million distribution poles in the United States alone. That is an absurd amount of assets that could potentially fail with catastrophic results.  
One of the ways we have started to address this issue is with the use of drones. Drones are the perfect tool for hard to reach asset inspections. Under Part 107, the commercial drone pilot regulations in the United States, we can fly about a half-hour at a time, line of sight, which is typically a mile per flight. In a half-hour, we can safely inspect a heck of a lot more poles than if we had to walk each one let along climbing them. Drone adoption continues to grow in various asset management domains such as electric utility, oil and gas, water/wastewater, and telecommunication. It truly is a tremendous tool for high value hard to reach assets. 
With that said there is a downside. Drones have allowed us to get to these difficult areas and they allow us to collect massive amounts of imagery showing any issues in stunning clarity. The question is, how do you now manage all this information you have collected with the drone? It’s still independent and not well integrated into your GIS systems. How do you get all that data into your asset management system so that imagery can become action? That is ultimately what we want to do. We want to turn imagery into action. Identifying the problem is only half the battle. 
So how do we do this? How do we fully integrate the drone into our GIS and asset management program? We believe the answer is, we take the things that have worked on the ground for GIS asset management and we bring that to the drone. To do this, we need to first bring the GIS with us when we fly. We have found that the best way to do this, in our opinion, is by using augmented reality. In this context, think of augmented reality in the same way you would a 2D map on your mobile device or computer. Live video becomes your base map. Now you can fly around your map and go to the different things in it to inspect them. Just like you would with Esri Collector, you can select an attribute, you can see all your attribute information and you can begin to edit that information, as necessary. Attach images, select pre-defined ranges or domains or use related tables to manage the inspection. The beauty of handling your drone data this way is that you have embedded the GIS into the exact process you would have performed with that drone anyways. By doing it this way, you have saved 12 steps that used to be required after you land. All of that is baked into the workflow. Now instead of putting data on a hard drive, loading data on a server, finding a way to correlate that back to a location in your GIS, joining that information in some way to the geometry, and then entering your field notes, Its already complete. You can do it all right there. To us, this is the most logical way to approach drone inspections. 
So, in summary, drone adoption in inspection continues to grow rapidly and rightfully so as we discussed today. It’s a clear and obvious choice for certain applications. We believe that there still is a major adoption challenge and unrealized potential because these systems to date have not been properly integrated into organizations' workflows. Our hope is that we have shown you, or at least provided some food for thought on how those challenges may be overcome in the future.  Even if it's not our solution to the problem specifically, we hope that in the future that clients will demand that their drone asset inspection workflows are managed in their GIS in a way that is similar to the rest of their operations. As our systems grow more and more sophisticated our workflows must adapt to that. We need to ensure that boots on the ground have proper context with the existing information we have, that they have a way to manage the content they are collecting that reduces the amount of potential human error, and importantly that their workflow facilitates better communication with everyone involved so that better decisions can be made quickly to prevent as many catastrophic failures as we can.