This post demonstrates the use case of creating a dataset suitable for visualisation of density (or clustering) information. It’s a good example of how just a few transformers can give impressive results.
Continue Reading April 5th, 2010
Contents:
XQuery: Reading XML Contents,
XQuery: Handling a Metadata Repository,
Fun Stuff: FME2009 Wallpapers,
Your Spy at Safe: New Tester Sneak Preview,
FME Blog: New and Updated!
This fine week the FME Evangelist highlights some FME2009 XML functionality that anyone might want to use. Plus we explore reading and writing to a CSW metadata repository and take a sneak peak into development of a new Tester transformer.
Continue Reading February 17th, 2009
Contents
A very quick heads up that the latest FME betas include support for GeoJSON v5. Also the .json extension is now recognized by FME as a supported dataset. For more details, check out http://wiki.geojson.org/GeoJSON_draft_version_5.
There’s been a lot of noise recently about AutoCAD Object Data and FME, and you may be wondering, “why is this so significant?”
Well, as I understand it, Object Data is a way of storing attribute information inside a DWG file, and is therefore Autodesk’s way of providing intelligence to their native format. This increases its ability as a format for uses other then CAD (as well as increasing the options to which a CAD system can use it).
If you type autocad object data into Google the first line that comes up is… “Object Data can only be created using Autodesk Map, Land Desktop or Civil 3D”.
So being able to read and write this type of data outside of an Autodesk product is a very significant fact indeed!
Safe has been working very closely with Autodesk to enable this functionality and is justifiably proud of how it has implemented this technology within the existing FME framework.
Anyway, as of build 5149 a beta version of this writer has been exposed in FME, as Safe’s Christmas gift to its AutoCAD users! Enjoy.
3) Raster Updates - New Transformers and Interpretation
This week sees a number of new items of raster functionality.
Although some originally vector transformers are capable of handling raster, some new transformers - being dedicated to raster - are more user friendly in that regard.
RasterTiler
The new RasterTiler transformer divides a raster feature up into a number of smaller ’tiles’.
A user would do this to make raster data management and viewing easier, by working with many smaller raster images and not one gigantic image.
From a user perspective, the big difference between this and the standard Tiler transformer (which can also accept raster) is that you get to choose how many rows and columns of tiles you want, and FME does all the work. To get 4×4 tiles using the regular Tiler you would need to know the size of the raster in pixels, divide by 4 and enter that number.
An example can be found on fmepedia at:
http://www.fmepedia.com/index.php/RasterTiler_Example
RasterPyramider
The new RasterPyramider transformer creates pyramids of raster data; ie layers of rasters with different resolutions.
When viewing most types of spatial data a key concept is that you view the coarse data when zoomed out, but finer data when you zoom in.
If there was such a book as Raster Pyramiding for Dummies it would say “This technique is similar to the one used by Google Earth - open Google Earth and the data is very coarse, because to show the highest quality data when viewing the entire Earth would cause major performance issues, but zoom in and it gets refined because you are looking at less surface area. The slight delay in display when zooming in (and the status bar says “Streaming”) is Google loading larger pyramid levels that have a better image quality and resolution.”
So what this transformer does is turn a raster feature into a set of layers of varying resolution that you can use in a raster viewer to improve performance.
An example workspace using the RasterPyramider can be found on fmepedia at:
http://www.fmepedia.com/index.php/RasterPyramiderxample
When you have just one raster feature that you want to both pyramid and tile then you would use the Pyramider first then the Tiler to divide the data up (the number of tiles would relate to the layer number).
An example workspace using both Pyramider and Tiler can be found on fmepedia at:
http://www.fmepedia.com/index.php/RasterPyramiding_Example
RasterSubsetter
The RasterSubsetter allows a user to create a subset of a raster - in effect extract, or clip out the centre area - defined in pixels (or cells). This is different to the existing clip and tile functions which only operate on a spatial reference system.
An example can be found on fmepedia at:
http://www.fmepedia.com/index.php/RasterSubsetter_Example
Raster Interpretation Methods
Previously FME had three interpretation methods - Nearest Neighbor, Bilinear and Bicubic.
Now it also has Average 4 and Average 16
Average4 and Average16 are types of averaging filter useful on numeric rasters such as DEMs.
Average4 averages each pixel according to the values of the 4 adjacent pixels, and Average16 presumably does the same but on a wider spread of data.
The idea is to create a smoother surface and reduce spikes or sharp depressions. If you tried this on a color (image) raster they would have the effect of slightly blurring the data.
These methods are applicable in any transformer using some form of interpretation; for example reprojection, resampling, pyramiding, etc
I just wanted to quickly highlight the fact that the FME User group has had its busiest year… ever!
By my count the current number of messages for 2008 is 2250 (and rising) beating the previous best (2002 - 2181) by some way.
Last month (November) was also the busiest month ever… 260 messages.
Thanks to all users and Safe staff who contribute to this useful tool.
If you (or your FME customers) aren’t a member - join now at:
http://groups.google.com/group/fmetalk
5) Tester and ExpressionEvaluator in Custom Transformers
Harbinder in our development team tells me that automatic publishing of parameters (when you first duplicate a custom transformer) will now check the attributes used in Testers and ExpressionEvaluators to see if the names are already exposed - if so the transformer parameters will not be published.
This improvement (build 5153+) should please users who are big on custom transformers because it makes the runtime dialog a lot easier to understand.
6) See What We Can Do With Solids
As an example of using FME solids, see this Dmitri-generated screenshot of a world DEM, as depicted in ArcScene.
Read on for some Safe holiday poetry… FME Poetry
Since it’s a holiday season I hope you’ll indulge me evangelising in prose this week, with a poem inspired by the Dr Seuss classic Green Eggs and Ham…
After reading this, even your children will understand FME (and want one for Christmas).
Do you use our
F-M-E?
I do not use your
F-M-E.
Your data could be here or there,
Your data could be anywhere!
My data’s here, on this CD,
so that does not apply to me,
I do not need your F-M-E!
Your data could be from Japan,
Spain, China, France, or Vietnam!
My data’s only English see,
What’s more it’s all in plain ASCII,
And it’s all here on this CD,
So I’ve no use for F-M-E.
Your data might be polylines,
paths or rasters, text or splines!
My data’s single points, 2D,
And like I said, in plain ASCII,
And it’s on one English CD,
So I shan’t want your F-M-E!
Your data, it might need transforming,
Cleaning up, to be conforming.
My data’s simple A-B-C,
And that’s the way it’s meant to be,
Unchanging points that aren’t 3D,
So where’s the role for F-M-E?
Your data you might need to fix,
To join it up to Informix,
My data’s all one table, see?
It doesn’t have a foreign key,
An outer join’s no use to me,
So I won’t need your F-M-E!
Your data could be MID and MIF,
Shape, Idrisi, GeoTIFF!
Oracle or Map 3D,
S-D-F or C-S-V,
N-T-X or Smallworld 4,
Shape, or SAIF or hundreds more!
Our data’s not proprietry,
We use a shareware app for free,
Inter-what-ability?
What’s the point in F-M-E?
But what about the need to train?
Don’t your staff ever complain?
[Ye Gods - this man has half a brain]
My staff all have a PhD,
And they can’t train, they’re too busy,
In fixing data quality,
There’s no time for F-M-E!
What if your data….. oh, but hark!
Is that the trilling of a lark?
No it’s my Blackberry…
A call from our most valued customer…
…you want what?
…when?
But…
That data could be here or there,
That data could be anywhere!
That data could be from Japan,
Spain, China, France, or Vietnam!
That data might be polylines,
paths or rasters, text or splines!
That data, it might need
transforming,
Cleaning up, to be conforming.
That data we might need to fix,
To join it up to Informix,
That data could be MID and MIF,
Shape, Idrisi, GeoTIFF!
Oracle or Map 3D,
S-D-F or C-S-V,
N-T-X or Smallworld 4,
Shape, or SAIF or hundreds more!
And what about the need to train?
My staff are going to complain!
[Ye Gods - I must have half a brain]
mmmmmm…
OK. I will try your F-M-E.
Can I be an evaluee?
Wow!
My F-M-E I’ll never trade,
Now I’ve drunk that cool kool-aid
I’ll use it when my data’s there,
Or when my data’s anywhere
I’ll read that data from Japan,
And Unicode from Vietnam
And if my data’s awkward splines,
Workbench makes me polylines
And when my data won’t conform,
Transformers turn it to the norm
And if my Informix is full,
We’ll join instead to Oracle
Translating some S-57,
Is truly a foretaste of heaven
But if my raster’s SID or SPOT,
F-M-E won’t care a jot
And it’s so easy for to learn,
We’ll finish quick, with time to burn
YES! I love your F-M-E,
Thank you
Thank you
I’m filled with glee!
December 21st, 2007
New in FME this week is a DonutBridgeBuilder transformer – that works out a route among the holes or islands on a donut polygon.
Dmitri created a short demo workspace for this which you can find on fmepedia at: http://www.fmepedia.com/index.php/DonutBridgeBuilder_Example
One common complaint among users is that the formats gallery dialog contains a large amount of information, but defaults to a small size in which most columns are not fully displayed. As you would imagine, it becomes tedious having to resize the dialog every time you use it.
An update this week didn’t simply make the gallery larger – because that might cause problems for users with small displays – but what it does is store the size details for use next time the dialog is opened. Therefore a user needs only to resize the dialog once and it will always be that size when opened.
Above: Before the default window was small, but now I can see every field.
The ecstatic response from one of the satisfied users was “Hallelujah”!
Picking apart Dale and Dmitri’s serpentine numberer recently (www.fmepedia.com/index.php/SerpentineNumberer) I found they had used a funky programmer’s short-cut to avoid using a bunch of testers in the workspace.
The shortcut was to use a C language conditional operator in the expression to both test and assign a value all in one.
Because I think it’s a useful tip for more than just the programmers amongst us, I put some examples up on fmepedia under:
http://www.fmepedia.com/index.php/ExpressionEvaluator_Example
Using the expression (x == 4) ? 1 : (y == 4) ? 2 : 3 (which isn’t as complicated as it looks) reduced one workspace from this:
…to this:
NB: As an example of how many times a new page gets viewed when advertised on the user group, this example got 60 hits in the first 24 hours.
Aaron Koning came up with a custom format to read geo-coded photos from a Ricoh GPS camera.
It’s on fmepedia at: http://www.fmepedia.com/index.php/Ricoh_GPS_Photo_custom_format
The really interesting thing is that Ricoh seem to be targeting this specifically at GIS users, and a lot of the camera resellers are also GIS resellers or consultants.
It might be worth someone going through the list to see if any are also FME users/resellers so we can be sure to pass this on.
http://www.ricohsolutions.com/geo/
http://www.ricohsolutions.com/resellers/
5) Transformer Drag-Insert On/Off Setting
A user on the FMETalk group bemoaned the fact that he couldn’t turn off the new transformer drag-drop-insert behaviour (which he needed to do because when zoomed out to the full workspace extent the links were too small to properly make out, even when highlighted).
By the time he checked back into the group we’d already made the fix for him (build 5138+).
His response… “Wow! One day out of the office and I come back to this. That’s what I call a response. Thanks to all the Safe guys.”
You get to this setting under Tools > Options > Workbench on the Workbench menubar. In 5139+ a toolbar button will also be available as a shortcut to this setting.
6) Point Thinning Custom Transformer
Designed for a user on the FMETalk group, this custom transformer thins out a cloud of points so that no two points are within x distance of each other (x being a user defined number). The method is OK (Mark S thinks a triangulation method would work more efficiently) but my real puzzle is that I sometimes get incorrect results unless a stick a NULL writer into the workspace!
See http://www.fmepedia.com/index.php/PointCloudThinner
Anyway it’s a good way of thinning out sample points (eg a DEM) and even if you don’t have a use for that, it’s a good example of using custom transformers, loops, lists and global variables.
Have feedback?
Email FME Evangelist, Mark Ireland at mark.ireland@safe.com.
November 30th, 2007
RSS 2.0