April 17th, 2012
Whenever we release a new version of FME, there are always a stack of new transformers in the mix. Very often such transformers can revolutionize the way you use FME - if only you know about them.
So the next two posts of mine are going to make sure you have no such excuse!
I’ll describe six new transformers, in alphabetical order so we start today with the AttributeExpressionRenamer, RasterToPolygonCoercer, and SliverRemover, then finish by concentrating on the holy trinity of new XML transformers (XMLFlattener, XMLSampleGenerator, XMLUpdater) that appear in 2012!
Incidentally, this post coincides with the first day of the FME World Tour 2012 in North America. We have two teams out there meeting with users and evangelizing about all that is new in 2012. If you get the chance to join them it would be well worth your while.
There are still lots of events to come - worldwide, of course - in the next few weeks.
We already had a transformer called the AttributeExpressionRemover for deleting attributes according to a search string, but - let’s face it - most users rename more attributes than they delete. But this new transformer doesn’t just rename attributes according to an expression! Oh no, it does way more.
Firstly, you don’t need to enter an expression. If you want to select ALL attributes then there is a simple option to do that:
Secondly, there are a whole bunch of standard renaming methods built-in:
Perhaps the most useful option - Change Case - actually comes last in the list, so let’s look at that.
A multiple-attribute case-change is useful for those times when you read from a format that supports lower case and write to one that supports upper case only, or vice versa.
In that scenario, this one transformer will save you a lot of effort in mapping and case changing attributes, just like this:
The case change option lets you convert to UPPERCASE, lowercase, Title case, or Full Title Case. The big thing is not to get confused with the CaseChanger transformer. The CaseChanger changes the case of the attribute values, not the attribute names.
The other key renaming options are for prefixes and suffixes. Take this case:
Here I am reading from an XML dataset of FME training courses. The incoming data is prefixed with the string “Course.” because it is nested inside a feature path called Course. I don’t really want this prefix, so I apply the AttributeExpressionRenamer to remove it:
As well as removing a prefix I could, of course, add a new prefix, or add/remove suffixes as required. There are several scenarios I can think of where this might be of use.
Besides these, there is always the option that the transformer name suggests: to rename attributes according to a regular expression:
Here I’m taking the case change example from above, and applying a second AttributeExpressionRenamer in order to remove all underscore characters from the attribute names.
Remember, the key here is that I don’t need to manually specify which attributes are to be renamed - I can apply it to every attribute in a “dynamic” way.
The idea was to take a raster DEM and convert it to a set of vector polygons, one for each level of elevation. The RasterCellCoercer creates a square polygon for each cell, then the workspace used a Dissolver transformer with a group-by set to elevation to merge similar ‘cells’ together.
The new RasterToPolygonCoercer not only merges that functionality and wraps it up in a nicer way, it is also way, way, way faster. If you clicked on that FMEpedia link you’ll notice that if the raster wasn’t resampled to a coarser resolution, it took 6+ hours to complete.
Replacing that with the RasterToPolygonCoercer, I can get the same result in 94 seconds. That’s over 200 (two-hundred) times faster than before!
Kudos to our raster development team for that speedup!
A similar use for this transformer is to pick out features from a raster thematic map like this:
It’s a straightforward task to turn this park feature into a polygon, like so…
I suppose you could even extract a road network in the same way, or do things like look for swimming pools, areas of vegetation, or buildings; depending on the value of the raster cells.
The SliverRemover is designed for those awkward moments when you find gaps in your supposedly continuous polygon layer, like this:
Snapping doesn’t really work in these cases and, not only will it not work, it may leave badly formed bits of geometry in the output; hence the SliverRemover is a good solution. It isn’t a full conflation tool, but it gets us quite a bit closer to that goal.
If you download the FMEData dataset from www.safe.com/fmedata and open the Parks dataset:
…you’ll see some ugly gaps and overlaps where parks are supposed to meet up:
…but the SliverRemover puts them right in no time:
The above are polygons, by the way, I just turned off the fill for reasons of clarity.
The transformer has four different methods of fixing the slivers (not six as the documentation suggests). The question is really one of deciding which polygon gets assigned the sliver. Planar Partitioning is the term for this, I believe.
The Longest Boundary method produces the nicest-looking result, because it is more of a smoothing process. But if you need to ensure that one feature is preserved over another, then you can set a priority value.
There is also a random mode, although I’m not sure if a GIS data administrator would use a tool called “random”, however well it might work! But if you need to repair slivers, and don’t care how, then it’s as good a method as any.
One drawback is that the transformer won’t fix (seal) openings at the edge of a polygon:
…but there are other ways to deal with that (for example the Snapper may be a good choice at that point).
Incidentally, measures and 3D parts are carried through the transformer, as we are trying very hard to make sure we don’t drop that information.
OK, that’s 3 of the six transformers I am going to highlight. In my next post I’ll be doing a mini XML-special by focussing on the XMLFlattener, XMLSampleCreator, and XMLUpdater.
Don’t forget, these are all available only in FME 2012 or newer. You can upgrade (to 2012-SP2) by checking out www.safe.com/downloads
Bye for now,