The FME Evangelist

FME Evangelism #19: OpenStreetMap, GeoJSON, Transformers Guide

August 19th, 2008

Contents:
1) Open Street Map Reader
2) GeoJSON 1.0
3) Workbench transformers guide
4) FME and 3D Rock Stars
5) Science Lab: Multi-Threading Follow-Up
6) PADS Walk for Independence

Introduction
Hi FME’ers
You don’t need me to tell you how this new world of Neogeography has opened up spatial data to the masses, and how some of the biggest projects at the moment are collaborative efforts by “amateur” cartographers.

Given this, and Safe’s inclination to support any emerging technology with a bold vision, it was pretty much inevitable that we would add support for two up and coming formats: Open Street Map XML and GeoJSON 1.0

I heard a quote about the iPod Touch that went “who’d have thought ten years ago that Unix machines would look like this?” Along the same lines, who would have thought ten years ago that FME would look like this: that we could take a worldwide set of volunteer submitted data and (with FME Server) stream it out in a community defined format?

Incredible eh? Check below for more info. The OSM reader in particular is very cool.

Open Street Map Reader
As wikipedia states, OpenStreetMap (OSM) is a collaborative project to create maps using data from primarily volunteer-submitted GPS data.

On using their web mapping interface you are given the option to save data as “OpenStreetMap XML Data” (below). It is this format that FME is able to read.

FME’s reader appears very powerful (yes, I’m biased, but it does), and there are a number of useful settings.

Below: The OpenStreetMap Reader Settings Dialog:

Firstly this reader is able to operate in one of three different modes: RAW (Basic), BROAD and SPECIFIC. The choice of mode determines what level of feature type definition is provided.

• RAW provides feature types for basic element definitions: node, way, relations, etc.
• BROAD adds feature types for categories of features: highway, railway, power, etc
• SPECIFIC provides feature types at a detailed level: highways_residential, highways_secondary, etc

Alternatively, a user can define a custom pipeline file to get a user-defined set of feature types.

Below: Part of a SPECIFIC schema with highway_motorway_junction annotated for clarity:

Secondly, the reader provides a means to close area features. As I understand it, all OSM data is made up of line features, and an area (polygon) is simply a “way” (line) that closes and is tagged. For example, a feature that closes and is tagged Natural-Water is considered as an area.

Below: From the OSM wiki. The filled triangular symbol indicates this can be an area feature:

In FME it’s simple to create polygons from lines where the first and last point is identical, and this reader will do that where the tag indicates. However, to give more control, there are two other closing options:

• Auto-Close Default: Automatically turn all closed features into areas, even if the tag is not set.
• Auto-Close Custom: Automatically turn all features into areas, if they appear in a user-defined list.

Below: OSM Data for the Olympic city of Beijing as shown in the FME Universal Viewer.

GeoJSON 1.0
In June, a message in the GeoJSON mailing list reported:

“The GeoJSON Authors are proud to announce the finalization of the GeoJSON 1.0 Specification.
Representing more than a year’s worth of community discussion and development, the GeoJSON specification describes an easy to use, extensible format for transferring geographic data over the web.”

A little over a month later support for the 1.0 specification was added to the FME2009 beta.

For those not in the know, the “JS” stands for JavaScript; so GeoJSON defines spatial data in a way that JavaScript web browsers can use instantly. With both reader and writer, FME enables users to read GeoJSON services and use the data as required, and/or allows a user to transform data and write it back to GeoJSON.

Add in an FME Server and you can even stream the data back out as a GeoJSON service.

Transformers Guide

If you haven’t yet used our Transformer Quick Reference Guide, then you are missing out on a vital resource for helping FME users navigate their way around the product. This guide lists the most used (and useful) transformers, gives a brief summary of each, and includes a little diagram or screenshot to show the uses to which they can be put.

The latest news on this resource is that it has been revamped for FME2008, but also that it is now available in a layout designed to be printed as a small booklet. In fact, anyone attending a Safe-hosted training course will automatically receive this guide in printed form, as a booklet.

So if the range of possible transformations seems too over-whelming, this guide will help you narrow down the choices and select the best tool for your job.

FME and 3D Rock Stars

FME Certified Professional Hans van der Maarel sent me a useful link to his blog posting on creating a 3D visualization of Radiohead singer Thom Yorke.

Again, while FME is not a visualization tool, it is very useful in transferring data into visualization applications. In this case Hans used FME to convert a set of Lidar data into a 3D Shape dataset, which he subsequently rendered into an anaglyph (one of those red/blue 3D images) using a product called VNS (Visual Nature Studio).

Science Lab: Multi-Threading Follow-Up

To follow-up on the item about multi-threading, it turns out that one of the operations improved by recent multi-threading work is a vital part of how we write Shape datasets. Therefore, from what I thought was an insignificant change, anyone who writes Shape data (and who doesn’t) on a multi-core machine should find significant performance improvements in FME2009.

I haven’t had the opportunity yet, but I plan to test this soon: I’ll let you know what sort of performance improvement we get over FME2008.

PADS Walk for Independence
I hope you’ll excuse this personal note about PADS (the Pacific Assistance Dogs Society) who raise and train assistance dogs for people with some form of physical disability. My family volunteers by helping to socialize a puppy in training, Bewick.

This weekend (Aug 24th) we’ll be participating in a PADS fundraising event: The Walk for Independence.

The actual walk is only 2km, which seems sad when compared to the current Olympic activity. On the other hand, it’s twice as far as I run in the average soccer game, and in any case the I think the importance of the cause offsets any feebleness of the challenge.

So, if you would like to donate any amount of money to this very worthwhile cause, then please visit our fundraising page or go direct to the sponsor form. Alternatively - if you live in Western Canada - consider volunteering as a puppy raiser! They are always needed.

Thank you, thank you, thank you.

Below: Bewick. How could you turn down a look like that?

This Edition of the FME Evangelist…
…was written to the tune of Simon Smith and his Amazing Dancing Bear.

I don’t know why; but I do know that the best version is by the Muppets!

Entry Filed under: Data Transformation, FME Desktop, GIS, Geoweb, Miscellaneous, Science Lab