We’ve had several opportunities to refine GeoGig (formerly GeoGit) workflows in real-world situations, but among the most fulfilling was assisting with the response to Typhoon Yolanda (also known as Typhoon Haiyan) in the Philippines. It was the strongest cyclone to make landfall in recorded history, resulting in an urgent need to share data about the damage to help with recovery and reconstruction.
To meet this need, the Global Facility for Disaster Reduction and Recovery (GFDRR) teamed up with the American Red Cross and the Humanitarian OpenStreetMap Team (HOT) and launched an open data platform to gather and share data about Yolanda. The ROGUE project, which helps develop GeoGig, was asked to help manage and distribute extracts of OpenStreetMap data. As described below, we created a powerful bidirectional workflow with OpenStreetMap that enabled us not only to derive and publish up-to-date data for response and recovery efforts but also to contribute back to OpenStreetMap.
Importing OpenStreetMap Data
Thanks largely to HOT’s efforts, a large number of damaged and destroyed buildings were mapped into OpenStreetMap using commercial satellite imagery distributed under the Next View license or the State Department’s Imagery to the Crowd program. GeoGig was used to extract data from OpenStreetMap and transform it into formats more useful to traditional GIS applications.
While GeoGig supports reading and writing from OpenStreetMap data in a variety of ways, the Yolanda efforts started with the daily
.pbf downloads from geofabrik that were then imported into a GeoGig repository using the
geogit osm import command. This initial import command brings the data into the standard node and way layers in a GeoGig repository with all of the OpenStreetMap tags attached to each feature. During the initial few imports we were able to find and solve some performance bottlenecks that reduced the import time from over an hour to just a few minutes.
Mapping to a Schema
Once imported, the
geogig osm map command was used to map the data into more traditional sets of layers, using the tags as attributes. A JSON mapping file specifies which tags were used to separate out the features into layers and assign attributes to each feature. The key mapping involved taking nodes and ways tagged with
typhoon:damage=yes and translating those into
damage_line layers with associated attributes. Over the course of mapping the data, we were able to make improvements to the codebase and workflow in several areas.
Sharing Up-to-Date Data
Once the repository had the data organized into the right schema, we used the
geogig export pg command to load snapshots into a PostGIS database and serve them to the web. Since we wanted to provide the most current data, we used the
geogig osm apply-diff command to update the repository with daily updates from OSM planet. This ensured that our repository always reflected recent edits and that layers were exported and updated on the site.
Contributing Back to OpenStreetMap
In addition to staying in sync with the global OpenStreetMap planet, GeoGig made it possible to change layers in our repository and apply them back to OpenStreetMap — enabling a fully round-trip or bidirectional workflow. For example, we found many misspellings or inconsistent use of tags in the data where able to correct them. We fixed these issues against our PostGIS snapshot, applied the changes back to the repository, generated a changeset using the
geogig osm create-changeset command, and finally uploaded the changeset using JOSM. In the process, we were once again able to improve these functions based on real-world usage.
These tools enable a powerful bidirectional workflow with OpenStreetMap. We demonstrated that data can be imported from OpenStreetMap into a local repository, mapped into a set of layers with a well-defined schema, and served via OGC services. Repositories can be kept in sync with OpenStreetMap over time and, if changes are made to the local repository, GeoGig enables us to produce changesets that can be contributed to the global OSM dataset. Using this same workflow, it becomes possible for users to effectively work with a local extract of OSM data for both making and applying local edits as well as incorporating upstream changes.
Jeff Johnson will present more about GeoGig-based OpenStreetMap import workflows on April 13th at State of the Map US.