Posts Tagged: Remote Sensing
Last week IGIS was very pleased to partner with CERES Imaging Inc. (http://www.ceresimaging.net/) to provide a workshop on GIS and Remote Sensing for Crop Agriculture, in Davis CA. This particular event represents an example of how UC ANR's IGIS Program is working with private industries to better deliver valuable services (in this case training) to public audiences who are eager to put the services/information into action; reinforcing UC ANR's priorities of public service and Cooperative Extension.
This event was partially funded by a Department of Water Resources (DWR) grant to CERES, which among other things has helped CERES to provide very affordable, high resolution, multi-spectral, thermal and NDVI data to agriculturalists around California. The objective of the DWR funding was to help facilitate more agile farming practices for water conservation, through the adoption of newly available aerial image products, while introducing farmers to contemporary image processing and mapping methods. By partnering with CERES in this effort, IGIS is helping farmers to better utilize CERES's image products, to ideally make their operations more efficient and profitable.
Many of us have watched in horror and sadness over the previous week as fires consumed much of the beautiful hills and parts of the towns of Napa and Sonoma Counties. Many of us know people who were evacuated with a few minutes' notice - I met a retired man who left his retirement home with the clothes on his back. Many other friends lost everything - house, car, pets. It was a terrible event - or series of events as there were many active fires. During those 8+ days all of us were glued to our screens searching for up-to-date and reliable information on where the fires were, and how they were spreading. This information came from reputable, reliable sources (such as NASA, or the USFS), from affected residents (from Twitter and other social media), and from businesses (like Planet, ESRI, and Digital Globe who were sometimes creating content and sometimes distilling existing content), and from the media (who were ofen using all of the above). As a spatial data scientist, I am always thinking about mapping, and the ways in which geospatial data and analysis plays an increasingly critical role in disaster notification, monitoring, and response. I am collecting information on the technological landscape of the various websites, media and social media, map products, data and imagery that played a role in announcing and monitoring the #TubbsFire, #SonomaFires and #NapaFires. I think a retrospective of how these tools, and in particular how the citizen science aspect of all of this, helped and hindered society will be useful.
In the literature, the theoretical questions surrounding citizen science or volunteered geography revolve around:
Accuracy – how accurate are these data? How do we evaluate them?
Access – Who has access to the data? Are their technological limits to dissemination?
Bias (sampling issues)/Motivation (who contributes) are critical.
Effectiveness – how effective are the sites? Some scholars have argued that VGI can be inhibiting.
Control - who controls the data, and how and why?
Privacy - Are privacy concerns lessened post disaster?
I think I am most interested in the accuracy and effectiveness questions, but all of them are important. If any of you want to talk more about this or have more resources to discuss, please email me: email@example.com, or Twitter @nmaggikelly.
Summary so far. This will be updated as I get more information.
Outreach from ANR About Fires
ANR has a number of programs dedicated to fire preparedness, recovery, and prevention.
Core Geospatial Technology During Fires
Fire perimeters from https://www.geomac.gov/services.shtml
The Active Fire Perimeters layer is a product of Geospatial Multi-Agency Coordination (GeoMAC). In order to give fire managers near real-time information, fire perimeter data is updated daily based upon input from incident intelligence sources, GPS data, infrared (IR) imagery from fixed wing and satellite platforms.”
MODIS hot spots from USFS Active Fire Mapping Program. MODIS The MODIS instrument is on board NASA's Earth Observing System (EOS) Terra (EOS AM) and Aqua (EOS PM) satellites. In addition to lots of other data, MODIS delivers Channel 31 brightness temperature (in Kelvins) of a hotspot/active fire pixel.
For more on how these are made: http://www.arcgis.com/home/item.html?id=b4ce4179b04f47e4ba79e234205565c1
Media using maps (super short list)
NYTimes: Minutes to Escape: How One California Wildfire Damaged So Much So Quickly. https://www.nytimes.com/interactive/2017/10/21/us/california-fire-damage-map.html?smid=fb-nytimes&smtyp=cur
NYTimes: How California's Most Destructive Wildfire Spread, Hour by Hour. OCT. 21, 2017. https://www.nytimes.com/interactive/2017/10/21/us/california-fire-damage-map.html?smid=fb-nytimes&smtyp=cur
- SFGate focusing on SkyIMD: http://www.sfgate.com/news/article/incredible-aerial-photos-wine-country-fires-12285123.php
Twitter: #sonomafires, #napafires, #tubbsfire
Flickr: SonomaFires, TubbsFires, NapaFires, etc.
Core Technology for Post-Fire Impact
High resolution imagery collection and analysis
Planet has made fire imagery available: https://www.planet.com/pulse/northern-california-wildfire-satellite-data-available-for-access/
Digital Globe + MapBox made a post-fire tool: (Author: @robinkraft
Email: firstname.lastname@example.org, Github repo, Source: Overview News // DigitalGlobe 2017): https://robinkraft.github.io/norcal-fires-imagery/compare.html; https://blog.mapbox.com/santa-rosa-fire-satellite-imagery-a31b6dfefdf8
- Digital Globe - Open Data Program: https://www.digitalglobe.com/opendata
Sonoma Veg Map Program has a few links to interesting stuff including ArcGIS Server Drone and Digital Globe Imagery: http://sonomavegmap.org/blog/2017/10/17/fires/
I know SkyIMD was flying the entire time.
- First Map - Flights 10/11-10/14/17 Here: http://www.skyimd.com/napa-sonoma-fire-imagery-map/
- Second Map - Flight 10/20/17 - Here: http://www.skyimd.com/napa-sonoma-fire-imagery-map-2/
Greg Crutsinger from @droneScholars made this available: Aftermath of #TubbsFire: https://www.mapbox.com/bites/00382/#18/38.4763/-122.74861
Open Aerial Map has one drone image not sure the source https://map.openaerialmap.org/#/-122.7497720718384,38.471986484020334,16/square/023010201213/59e62be93d6412ef7220c4c0?_k=hm0j7l
UC ANR's IGIS program hosted 36 drone enthusiasts for a three day DroneCamp in Davis California. DroneCamp was designed for participants with little to no experience in drone technology, but who are interested in using drones for a variety of real world mapping applications. The goals of DroneCamp were to:
- Gain an broader understanding of the drone mapping workflow: including
- Goal setting, mission planning, data collection, data analysis, and communication & visualization
- Learn about the different types of UAV platforms and sensors, and match them to specific mission objectives;
- Get hands-on experience with flight operations, data processing, and data analysis; and
- Network with other drone-enthusiasts and build the California drone ecosystem.
The IGIS crew, including Sean Hogan, Andy Lyons, Maggi Kelly, Robert Johnson, Kelly Easterday, and Shane Feirer were on hand to help run the show. We also had three corporate sponsors: GreenValley Intl, Esri, and Pix4D. Each of these companies had a rep on hand to give presentations and interact with the participants.
Day 1 of #DroneCamp2017 covered some of the basics - why drone are an increasingly important part of our mapping and field equipment portfolio; different platforms and sensors (and there are so many!); software options; and examples. Brandon Stark gave a great overview of the Univ of California UAV Center of Excellence and regulations, and Andy Lyons got us all ready to take the 107 license test. We hope everyone here gets their license! We closed with an interactive panel of experienced drone users (Kelly Easterday, Jacob Flanagan, Brandon Stark, and Sean Hogan) who shared experiences planning missions, flying and traveling with drones, and project results. A quick evaluation of the day showed the the vast majority of people had learned something specific that they could use at work, which is great. Plus we had a cool flight simulator station for people to practice flying (and crashing).
Day 2 was a field day - we spent most of the day at the Davis hobbycraft airfield where we practiced taking off, landing, mission planning, and emergency maneuvers. We had an excellent lunch provided by the Street Cravings food truck. What a day! It was hot hot hot, but there was lots of shade, and a nice breeze. Anyway, we had a great day, with everyone getting their hands on the commands. Our Esri rep Mark Romero gave us a demo on Esri's Drone2Map software, and some of the lidar functionality in ArcGIS Pro.
Day 3 focused on data analysis. We had three workshops ready for the group to chose from, from forestry, agriculture, and rangelands. Prior to the workshops we had great talks from Jacob Flanagan and GreenValley Intl, and Ali Pourreza from Kearney Research and Extension Center. Ali is developing a drone-imagery-based database of the individual trees and vines at Kearney - he calls it the "Virtual Orchard". Jacob talked about the overall mission of GVI and how the company is moving into more comprehensive field and drone-based lidar mapping and software. Angad Singh from Pix4D gave us a master class in mapping from drones, covering georeferencing, the Pix4D workflow, and some of the checks produced for you a the end of processing.
One of our key goals of the DroneCamp was to jump start our California Drone Ecosystem concept. I talk about this in my CalAg Editorial. We are still in the early days of this emerging field, and we can learn a lot from each other as we develop best practices for workflows, platforms and sensors, software, outreach, etc. Our research and decision-making teams have become larger, more distributed, and multi-disciplinary; with experts and citizens working together, and these kinds of collaboratives are increasingly important. We need to collaborate on data collection, storage, & sharing; innovation, analysis, and solutions. If any of you out there want to join us in our California drone ecosystem, drop me a line.
Thanks to ANR for hosting us, thanks to the wonderful participants, and thanks especially to our sponsors (GreenValley Intl, Esri, and Pix4D). Specifically, thanks for:
- Mark Romero and Esri for showing us Drone2Map, and the ArcGIS Image repository and tools, and the trial licenses for ArcGIS;
- Angad Singh from Pix4D for explaining Pix4D, for providing licenses to the group; and
- Jacob Flanagan from GreenValley Intl for your insights into lidar collection and processing, and for all your help showcasing your amazing drones.
Esri has been holding these Imagery and Mapping Forum days prior to the main UC. I was here for the day, as an invited panelist for the Executive Panel and Closing Remarks session on Sunday. During the day Ihung out in the Imaging and Innovation Zone, in front of the Drone Zone (gotta get one of these for ANR).
Notes from the day: Saw demos from a range of vendors, including:
- Aldo Facchin from Leica gave a slideshow about the Leica Pegasus: Backpack. Their backpack unit workflow uses SLAM; challenges include fusion of indoor and outdoor environments (from transportation networks above and below ground). Main use cases were industrial, urban, infrastructure. http://leica-geosystems.com/en-us/products/mobile-sensor-platforms/capture-platforms/leica-pegasus-backpack
- Jamie Ritche from Urthecast talked about "Bringing Imagery to Life". He says our field is "a teenager that needs to be an adult". By this he means that in many cases businesses don't know what they need to know. Their solution is in apps- "the simple and the quick": quick, easy, disposable and useful. 4 themes: revisit, coverage, time, quality. Their portfolio includes DEIMOS 1, Theia, Iris, DEIMOIS-2, PanGeo + . Deimos-1 focuses on agriculture. UrtheDaily: 5m pixels, 20TB daily, (40x the Sentinel output); available in 2019. They see their constellation and products as very comparable to Sentinel, Landsat, RapidEye. They've been working with Land O Lakes as their main imagery delivery. Stressing the ability of apps and cloud image services to deliver quick, meaningful information to users. https://www.urthecast.com/
- Briton Vorhees from SenseFly gave an overview of: "senseFly's Drone Designed Sensors". They are owned by Parrot, and have a fleet of fixed wing drones (e.g. the eBee models); also drone optimized cameras, shock-proof, fixed lens, etc (e.g. SODA). These can be used as a fleet of sensors (gave an citizen-science example from Zanzibar (ahhh Zanzibar)). They also use Sequoia cameras on eBees for a range of applications. https://www.sensefly.com/drones/ebee.html
- Rebecca Lasica and Jarod Skulavik from Harris Geospatial Solutions: The Connected Desktop". They showcased their new ENVI workflow implemented in ArcGIS Pro. Through a Geospatial Services Framework that "lifts" ENVI off the desktop; and creates an ENVI Engine. They showed some interesting crop applications - they call it "Crop Science". This http://www.harrisgeospatial.com/
- Jeff Cozart and McCain McMurray from Juniper Unmanned shared "The Effectiveness of Drone-Based Lidar" and talked about the advantages of drone-based lidar for terrain mapping and other applications. They talked through a few projects, and wanted to demonstrate the economies for drone-based lidar. The main advantages are in the data, not in the economics per se. They partner with Reigl and YellowScan from France. Showcased an example in Colorado between lidar (DJI Matrice was platform) and survey - the cost was 1/24th as expensive as the field survey. They did a live demo of some of the ArcGIS tools: classification of ground, feature extraction, etc. http://juniperunmanned.com/
- Aerial Imaging Productions talked about their indoor scanning - this linking indoor to outdoor - making data truly geo - is a big theme here. Also OBJ is a data format. From Wikipedia: "The OBJ file format is a simple data-format that represents 3D geometry alone — namely, the position of each vertex, the UV position of each texture coordinate vertex, vertex normals, and the faces that make each polygon defined as a list of vertices, and texture vertices." Used for 3D graphics, but increasingly for indoor point clouds in our field.
- My-Linh Truong from Reigl talked about their new static, mobile, airborne, uav lidar platforms. They've designed some mini lidar sensors for smaller UAVas (3lbs; 100kHz; 250m range; ~40pts/m2). Their ESRI workflow is called LMAP, and it relies on some proprietary REIGL software processing at the front end, then transfer to ArcGIS Pro (I think). http://www.rieglusa.com/index.html
We wrapped up the day with a panel discussion, moderated by Esri's Kurt Schwoppe, and including Lawrie Jordan from Esri, Greg Koeln from MDA, Dustin Gard-Weiss from NGA, Amy Minnick from DigitalGlobe, Hobie Perry from USFS-FIA, David Day from PASCO, and me. We talked about the promise and barriers associated with remote sensing and image processing from all of our perspectives. Some fun things that came out of the panel discussion were:
- Lawrie Jordan started Erdas!
- Digital Globe launched their 30cm resolution WorldView-4. One key case study was a partnership with Associated Press to find a pirate fishing vessel in action in Indonesia. They found it, and busted it, and found on board 2,000 slaves.
- The FIA is increasingly working on understanding uncertainty in their product, and they are moving for an image-base to a raster-based method for stratification.
- Greg Koeln, from MDA (he of the rad tie) says: "I'm a fan of high resolution imagery...but the world is a big place".
- Multi=sensor triangulation (or georeferencing a stack of imagery from multiple sources to you and me) is a continual problem, and its going to get worse before it gets better with more imagery from UAVs. On that note, Esri bought the patent for "SIFT" an automated tool for relative registration of an image stack.
- Space Junk!
Notes and stray thoughts:
- Esri puts on a quality show always. San Diego always manages to feel simultaneously busy and fun, while not being crowded and claustrophobic. Must be the ocean or the air.
- Gotta get behind the ubiquitous "analytics" replacement of "analysis" in talks. I am not convinced everyone is using the term correctly, but hey, it's a thing now: https://en.wikipedia.org/wiki/Analytics#Analytics_vs._analysis
- 10 years ago I had a wonderful visitor to my lab from Spain - Francisco Javier Lozano - and we wrote a paper: http://www.sciencedirect.com/science/article/pii/S003442570700243X. He left to work at some crazy company called Deimos in Spain, and Lo and Behold, he is still there, and the company is going strong. The Deimos satellites are part of the UrtheCast fleet. Small world!
- The gender balance at the Imagery portion of the Esri UC is not. One presenter at a talk said to the audience with a pointed stare at me: "Thanks for coming Lady and Gentlemen".
Good fun! Now more from Shane and Robert at the week-long Esri UC!
So much to learn! Here is my distillation of the main take-homes from last week.
Notes about the workshop in general:
- Making participants do organized homework and install software is necessary for complicated workshop content: http://neondataskills.org/workshop-event/NEON-Data-Insitute-2017
- NEON used tips from software carpentry workshops, particularly the green-and-pink sticky tag tip. When you put up a pink sticky on your computer, you need help; when you put up a green sticky on your computer, you are all good. This makes everything go smoother and means participants don't have to hold up their hand to get attention.
- Having lots of helpful, friendly faces around to help during the coding bits, and having access to the code when you got behind was critical.
- The workshop content and daily schedule:
NEON data and resources:
- All the NEON airborne data can be found here: http://www.neonscience.org/data/airborne-data
- For more on when data rolls out, sign up for the NEON eNews here: http://www.neonscience.org/
Other misc. tools:
- For cleaning messy data - check out OpenRefine - a FOS tool for cleaning messy data http://openrefine.org/
- Excel is cray-cray, best practices for spreadsheets: http://www.datacarpentry.org/spreadsheet-ecology-lesson/
- Morpho (from DataOne) to enter metadata: https://www.dataone.org/software-tools/morpho
- Pay attention to file size with your git repositories - check out: https://git-lfs.github.com/. Git is good for things you do with your hands (like code), not for large data.
- Markdown renderer: http://dillinger.io/
- MIT License, like Creative Commons for code: https://opensource.org/licenses/MIT
- There is a new project called "Feather" that allows compatibility between python and R: https://blog.rstudio.org/2016/03/29/feather/
- All the NEON airborne data can be found here: http://www.neonscience.org/data/airborne-data
- Information on the TIFF specification and TIFF tags here: http://awaresystems.be/, however their TIFF Tag Viewer is only for windows.
- All NEON point cloud classifications are done with LASTools. Go LASTools! https://rapidlasso.com/lastools/
- Check out pdal - like gdal for point clouds. It can be used from bash. Learned from my workshop neighbor Sergio Marconi https://www.pdal.io/
- Reflectance Tarps are made by GroupVIII http://www.group8tech.com/
- ATCOR http://www.rese.ch/products/atcor/ says we should be able to rely on 3-5% error on reflectance when atmospheric correction is done correctly (say that 10 times fast) with a well-calibrated instrument.
- NEON hyperspectral data is stored in HDF5 format. HDFView is a great tool for interrogating the metadata, among other things. https://support.hdfgroup.org/products/java/hdfview/