Dealing with Big Data: How to Keep Drone Imagery Organized
Admit it: Your flight record is a mess. You can’t even find the good stuff you flew last week! Taking the time to classify and keep drone imagery organized in a proper storage system is an important offseason bit of busy work that can yield dividends down the road.
Nov 7, 2016
If you’ve got anything to do with modern agriculture, then you probably know already that farming now has a whole new way for you to lose sleep with what’s come to be known as “Big Data”. Much of what comes off a drone is raster-based data; some of the biggest data out there.To make matters worse, raster data alone does not lend itself well to being natively annotated and labeled; which means that if you’re not actively organizing your data, it could come around to bite you in the tail later. My advice: if you’re not flying your drone during the off-season (heck, even if you are), then take the time to learn to catalog and then keep drone imagery organized. Two or three seasons down the road as the flights pile up for you and your clients, you’ll be glad you did!
Don’t overlook subscription fees.
There’s a big temptation to let image processing subscriptions lapse while you’re not flying. While this may mean immediate cost savings, be sure to read the fine print and know what happens when you decide to quit paying the bill. Can you still download your data? Can you log into your account at all? Limiting your accessibility to your data in the off season can be equivalent to cutting off your nose to spite your face. That time when the aircraft isn’t in the skies is a fantastic opportunity to get your act together; and you may not be able to best keep drone imagery organized in a “neutered” account. Likewise, procrastination in reactivating your account again as your operation gets along towards planting may cost you more than the nominal fees associated with account upkeep. Most service providers have the numbers pretty well dialed in by now, which means that if you’re paying for the year as opposed to by the month, you’re essentially getting those off-season days for free.
Use the tools you’re paying for.
To the aforementioned point, if you’re paying for a processing service, take time to understand the depth and breadth of the tools you shelled out that hard-earned money for. Many processing services have pretty good search and organizational tools if the user just takes the time to understand them and work within the parameters of the system. Getting to know the tools on-hand can be the first step in the efforts to keep drone imagery organized. For example, DroneDeploy has live search capabilities that [at the time of this writing] filter by name and geographies. That said, there is room for improvement, and filtering options by date and keyword admittedly aren’t that hot right now. How to cope? Put the things you know you’re gonna search by right into the flight name. We all know a person that uses every bit of that 129 character limit when naming a Word doc. It’s time to quit snickering and emulate what they do. A typical flight named in-software by yours truly commonly references an ISO-standard date, followed by customer, followed by a brief description, followed by any keyworks I think I may search on later. The result in practice? “2015 demo” in my search yields all 2015 flights that I tagged as demo (plus a little bit extra!).
Redundancy is key. Redundancy is key.
Set up an organizational structure for processed data; whether it be for your own farm or for each customer. Treat each deliverable as it’s own project. I usually begin with an empty folder containing a few text files to remind myself as to what goes in the folder. From there, it’s a matter of duplicating the template with each iteration of a project. Don’t be so inflexible as to not be willing to change the template along the way; the template should change as you data collection methods improve and you refine your processes. The end result means that when a customer or your agronomist calls with a question, you’ll have immediate (and definitive) answers. To that point, if you’re spending all that time to maintain a structure for the ages, by all means go buy that Drobo, or pony up for the Pro plan for Dropbox and keep track of all that hard work. Heck do both! This borderline-OCD data hoarder does! Data storage is a cheap as it’s ever been; and I can’t remember the last time anyone got made fun of for having too many backups!.
Look, I know organizing data ain’t easy. For that matter, organizing your socks isn’t that much fun either. However, there should be at least a little something to grab onto and run with out of what passes for my system to keep drone imagery organized. That said, y’all should have no reason now to store folders permanently on your desktop with names like “Larry’s Farm,” “South 40,” or “Wheat in April”. It’s for your own good! 🙂 Got a trick or best practice that you employ to how to keep drone imagery organized? We’d love to hear it by way of the link to the comments for this post!