Posts Tagged: IGIS
GO FAIR is an initiative to promote and support data stewardship that allows data to be Findable, Accessible, Interoperable, and Reusable. I was pleased to attend the launch of the first North American FAIR network last week at the UC San Diego Supercomputing Center.
Coping with a Data Tsunami
To say that we live in a data rich world is an understatement. We live a data drenched world (a fact I'm constantly reminded of by the 'hard drive full' warnings that pop-up on my computer on a weekly basis). Thanks to simultaneous, order-of-magnitude, advances in our ability to produce, disseminate, and store all manner of data, people working in fields from economics to physics to agriculture are struggling to benefit from, rather than be paralyzed by, the volume and diversity of data we produce. And this is by no means a problem only affecting academics, as more and more individuals, private companies and organizations are collecting and working with large volumes of data, from personal health sensors to drones.
Adding to the challenge, there are often major barriers to get data to talk to each other. They may be stored in different formats, use different scales or units of analysis, or be under different restrictions. If you've ever carried personal health data from one doctors office to another by hand, you know what I mean.
FAIR Data Stewardship Principles
These are not new problems, but have taken on increased sense of urgency as the challenge gets worse and the demand for integrated analyses of complex problems grows. GO (Global Open) FAIR is a European based initiative that has two faces: i) a set a principles for data stewardship, and ii) a growing network of institutions and programs that are taking tangible steps toward a world in which data are Findable, Accessible, Interoperable, and Reusable. FAIR certainly doesn't mean that collected data have to be free or open access, but data stewardship should have a way to share information about the existence of data, and a means for access when appropriate.
The FAIR principles mirror what open science advocates have argued for many years. As a program, GO FAIR has gained more traction than many of its predecessors. Following endorsements from the European Commission and other international bodies, the EU has already committed €2 billion to the first phase of implementation. Starting in 2018, the major EU funding agencies will require applicants to submit data stewardship plans that align with the FAIR principles. The initiative is also investing a lot in training people to use metadata standards and tools, many of which already exist.
How is This Relevant for ANR?
ANR academics are impacted by the data psunami in at least two ways (neither for good). Like all practicing scientists, we have to deal with the usual challenges of managing large volumes of data, the frustrations of not being able to find or use data that others have collected, and the burden of all the gymnastics one must do to combine data from different sources into a robust, repeatable analysis. On top of that, as public servants whose work is funded by taxpayers, we have an additional moral and legal responsibility to be good stewards of all data collected for our public mission, which means ensuring the data we collect remains discoverable and accessible for other studies. Similarly, our extension mission also requires us to help California growers and land stewards get the most value from the data they collect, with tools that address their requirements for privacy and security.
While this may all seem like a lot to think about and additional work, the rewards are pretty exciting as the following video shows:
How Close are Your Data to Being FAIR?
For many us, putting the principles of FAIR data stewardship into practice will require a step or two we're not accustomed to, such as i) generating metadata in a format that can be read by both people and machines, and ii) storing our data (and metadata) for the long-term. The table below from a recent Nature article breaks down the gold standard a little further.
F1. (meta)data are assigned a globally unique and persistent identifier
F2. data are described with rich metadata (defined by R1 below)
F3. metadata clearly and explicitly include the identifier of the data it describes
F4. (meta)data are registered or indexed in a searchable resource
I1. (meta)data use a formal, accessible, shared, and broadly applicable language for knowledge representation.
I2. (meta)data use vocabularies that follow FAIR principles
I3. (meta)data include qualified references to other (meta)data
A1. (meta)data are retrievable by their identifier using a standardized communications protocol
A1.1 the protocol is open, free, and universally implementable
A1.2 the protocol allows for an authentication and authorization procedure, where necessary
A2. metadata are accessible, even when the data are no longer available
R1. meta(data) are richly described with a plurality of accurate and relevant attributes
R1.1. (meta)data are released with a clear and accessible data usage license
R1.2. (meta)data are associated with detailed provenance
R1.3. (meta)data meet domain-relevant community standards
Wilkinson, Mark D., et al. "The FAIR Guiding Principles for scientific data management and stewardship." Scientific Data 3 (2016): 160018.
As a research technology unit, I think we're doing fairly well in terms of keeping our data organized and accessible for the long-term. However after looking at our data management practices through the FAIR lens, I now see our metadata misses some important characteristics, a lot of the quality metrics aren't machine readable, and need to learn more about metadata repositories and discoverability, particularly for our drone data. These are challenges common to many new sources of geospatial data, and we look forward to engaging with the new arm of the GO FAIR network to develop solutions.
Under the Fly4Fall campaign, amateur drone hobbyists across the globe are invited to take aerial 360 photos with their drone and contribute them to a collection of fall landscapes that will grow over time.
Never taken an aerial 360 photo before? Me either, but fortunately it recently got a whole lot easier with a free iOS app called Hangar 360. The Hangar app flies your DJI drone for you, climbing to the height you program and then taking about 25 photos in a circle at three different angles to the horizon. The whole thing takes about 2 minutes, and you can collect multiple panos per flight. You then land the drone (but don't turn it off just yet!), transfer the photos from the drone to your phone over the WiFi, and then upload the photos to Hangar. Hangar stitches the photos for you in the cloud (also free!), and sends you a link. The results are stunning! See the panoramic photo below of Kearny REC made by IGIS's Robert Johnson earlier this week.
Inspired by citizen science initiatives like the Christmas Bird Count and Project BudBurst, where large numbers of naturalists record observations in a coordinated way, Fly4Fall is part non-professional science project, part art, part community building, and a whole lot of fun. Crutsinger discussed some of the potential science angles in a recent LinkedIn post.
Full instructions can be found at Fly4Fall.com. Currently, the Hangar app only works on iOS, unfortunately, and only with DJI drones (but the list includes most of the popular ones). Android enthusiasts can check out Litchi, which includes similar functionality but costs $25 and you have to process the images on your own (look for tutorials online).
Of course like any drone flight you have the follow the rules - only fly in permitted areas, don't fly directly over people, and be safe!
We look forward to seeing the Fly4Fall panoramas coming in. Feel free to use the comment box below to share your experiences and thoughts!
With the ever-changing world of technology and knowledge we have a need to get this information out to the public in a timely manner. One of the new tools that we can use are Storymaps from ESRI. These web based application combine text and maps into an intuitive and interactive experience. For more information about storymaps please go to the esri storymap website.
We at IGIS have helped build several storymaps for different groups within University of California – Agriculture and Natural Resources Division (UCANR). These story maps have covered topics ranging from the issue of conifer encroachment in the oak woodlands of northern California to information about the UCANR Research and Extension Centers (RECS). These different sites can be viewed at the following sites:
California Naturalist Program Partners
Did you know! REC Tour
Oregon white oak and California black oak loss due to conifer encroachment
Our Partners - Master Gardener Program
UC Hopland Research & Extension Center Call for Proposals
In the coming months we will be offering training opportunities that will highlight these tools and how to build storymaps that can highlight your work. To build storymaps you will need an ArcGIS Online account. If you are part of the UCANR network please fill out the following form and we will help you get an account so that you can start building these storymaps yourself./table>
Drone Mapping California is a moderated email list intended to share news, information, and questions about using drones for mapping and data collection. That covers a lot - technology, training, regulations, hardware, software, analytical techniques, etc. We hope this list will be a channel through which new and seasoned drone operators and researchers can share and grow their knowledge and expertise.
The list has a California focus, but all are welcome. If you are interested in collecting data with drones, please subscribe here! IGIS will administer the list for the foreseeable future, including moderating messages to prevent spam, but we are always open to comments and suggestions.
Top: Matrice 100 with dual RGB and multispectral sensors
Bottom: mNDGI image of a field at Desert Research and Extension Center
Photos by Sean Hogan
Today was a great day at the ESRI User conference. I attended several sessions covering the use of raster and vector data in big data analyses, the use of python and r for data science, and the use of arcpy to create and modify maps in ArcGIS Pro.
Speaking of big data and arcgis, over the past 5 years I have watched ESRI develop their software from a desktop environment to a distributed computer platform capable of analyzing millions of spatial features in minutes and hours instead of days or weeks. These tools are now accessible to the average users of arcgis without the need to understand the underlying big data frameworks and software. We now have the ability to install tools such as GeoAnalytics Server, the Spatial Temporal Datastore, and ArcGIS Pro and take advantage of big data analytics within ArcGIS Pro. By using these tools we do not have to know how to setup and manage tools like apache spark, elasticsearch, or other tools.
The above tools are used to store and analyze vector data. To analyze multidimensional rasters, we have the ability to use mosaic datasets to store and analyze multiple raster datasets and multiple raster formats. Over the past years I underestimated the capability and the utility of mosaic datasets. These datasets have the ability to calculate custom raster functions on the fly. These datasets can also be shared via ArcGIS Server and ArcGIS Online. I look forward to utilizing the datasets in the future.
One thing to note at the ESRI User Conference is that like last year all of the presentations that I have seen by ESRI staff this year have been on ArcGIS Pro. The ArcGIS Pro software is getting more powerful and ESRI is building more functionality into this new software platform.
I cannot wait to see what day 4 brings tomorrow.