Contents
1) GeoJSON Support
A very quick heads up that the latest FME betas include support for GeoJSON v5. Also the .json extension is now recognized by FME as a supported dataset. For more details, check out http://wiki.geojson.org/GeoJSON_draft_version_5.
2) Object Data Writing
There’s been a lot of noise recently about AutoCAD Object Data and FME, and you may be wondering, “why is this so significant?”
Well, as I understand it, Object Data is a way of storing attribute information inside a DWG file, and is therefore Autodesk’s way of providing intelligence to their native format. This increases its ability as a format for uses other then CAD (as well as increasing the options to which a CAD system can use it).
If you type autocad object data into Google the first line that comes up is… “Object Data can only be created using Autodesk Map, Land Desktop or Civil 3D”.
So being able to read and write this type of data outside of an Autodesk product is a very significant fact indeed!
Safe has been working very closely with Autodesk to enable this functionality and is justifiably proud of how it has implemented this technology within the existing FME framework.
Anyway, as of build 5149 a beta version of this writer has been exposed in FME, as Safe’s Christmas gift to its AutoCAD users! Enjoy.
3) Raster Updates - New Transformers and Interpretation
This week sees a number of new items of raster functionality.
Although some originally vector transformers are capable of handling raster, some new transformers - being dedicated to raster - are more user friendly in that regard.
RasterTiler
The new RasterTiler transformer divides a raster feature up into a number of smaller ’tiles’.
A user would do this to make raster data management and viewing easier, by working with many smaller raster images and not one gigantic image.
From a user perspective, the big difference between this and the standard Tiler transformer (which can also accept raster) is that you get to choose how many rows and columns of tiles you want, and FME does all the work. To get 4×4 tiles using the regular Tiler you would need to know the size of the raster in pixels, divide by 4 and enter that number.
An example can be found on fmepedia at:
http://www.fmepedia.com/index.php/RasterTiler_Example
RasterPyramider
The new RasterPyramider transformer creates pyramids of raster data; ie layers of rasters with different resolutions.
When viewing most types of spatial data a key concept is that you view the coarse data when zoomed out, but finer data when you zoom in.
If there was such a book as Raster Pyramiding for Dummies it would say “This technique is similar to the one used by Google Earth - open Google Earth and the data is very coarse, because to show the highest quality data when viewing the entire Earth would cause major performance issues, but zoom in and it gets refined because you are looking at less surface area. The slight delay in display when zooming in (and the status bar says “Streaming”) is Google loading larger pyramid levels that have a better image quality and resolution.”
So what this transformer does is turn a raster feature into a set of layers of varying resolution that you can use in a raster viewer to improve performance.
An example workspace using the RasterPyramider can be found on fmepedia at:
http://www.fmepedia.com/index.php/RasterPyramiderxample
When you have just one raster feature that you want to both pyramid and tile then you would use the Pyramider first then the Tiler to divide the data up (the number of tiles would relate to the layer number).
An example workspace using both Pyramider and Tiler can be found on fmepedia at:
http://www.fmepedia.com/index.php/RasterPyramiding_Example
RasterSubsetter
The RasterSubsetter allows a user to create a subset of a raster - in effect extract, or clip out the centre area - defined in pixels (or cells). This is different to the existing clip and tile functions which only operate on a spatial reference system.
An example can be found on fmepedia at:
http://www.fmepedia.com/index.php/RasterSubsetter_Example
Raster Interpretation Methods
Previously FME had three interpretation methods - Nearest Neighbor, Bilinear and Bicubic.
Now it also has Average 4 and Average 16
Average4 and Average16 are types of averaging filter useful on numeric rasters such as DEMs.
Average4 averages each pixel according to the values of the 4 adjacent pixels, and Average16 presumably does the same but on a wider spread of data.
The idea is to create a smoother surface and reduce spikes or sharp depressions. If you tried this on a color (image) raster they would have the effect of slightly blurring the data.
These methods are applicable in any transformer using some form of interpretation; for example reprojection, resampling, pyramiding, etc
4) FMETalk User Group
I just wanted to quickly highlight the fact that the FME User group has had its busiest year… ever!
By my count the current number of messages for 2008 is 2250 (and rising) beating the previous best (2002 - 2181) by some way.
Last month (November) was also the busiest month ever… 260 messages.
Thanks to all users and Safe staff who contribute to this useful tool.
If you (or your FME customers) aren’t a member - join now at:
http://groups.google.com/group/fmetalk
5) Tester and ExpressionEvaluator in Custom Transformers
Harbinder in our development team tells me that automatic publishing of parameters (when you first duplicate a custom transformer) will now check the attributes used in Testers and ExpressionEvaluators to see if the names are already exposed - if so the transformer parameters will not be published.
This improvement (build 5153+) should please users who are big on custom transformers because it makes the runtime dialog a lot easier to understand.
6) See What We Can Do With Solids
As an example of using FME solids, see this Dmitri-generated screenshot of a world DEM, as depicted in ArcScene.
Read on for some Safe holiday poetry… FME Poetry
Since it’s a holiday season I hope you’ll indulge me evangelising in prose this week, with a poem inspired by the Dr Seuss classic Green Eggs and Ham…
After reading this, even your children will understand FME (and want one for Christmas).
Do you use our
F-M-E?
I do not use your
F-M-E.
Your data could be here or there,
Your data could be anywhere!
My data’s here, on this CD,
so that does not apply to me,
I do not need your F-M-E!
Your data could be from Japan,
Spain, China, France, or Vietnam!
My data’s only English see,
What’s more it’s all in plain ASCII,
And it’s all here on this CD,
So I’ve no use for F-M-E.
Your data might be polylines,
paths or rasters, text or splines!
My data’s single points, 2D,
And like I said, in plain ASCII,
And it’s on one English CD,
So I shan’t want your F-M-E!
Your data, it might need transforming,
Cleaning up, to be conforming.
My data’s simple A-B-C,
And that’s the way it’s meant to be,
Unchanging points that aren’t 3D,
So where’s the role for F-M-E?
Your data you might need to fix,
To join it up to Informix,
My data’s all one table, see?
It doesn’t have a foreign key,
An outer join’s no use to me,
So I won’t need your F-M-E!
Your data could be MID and MIF,
Shape, Idrisi, GeoTIFF!
Oracle or Map 3D,
S-D-F or C-S-V,
N-T-X or Smallworld 4,
Shape, or SAIF or hundreds more!
Our data’s not proprietry,
We use a shareware app for free,
Inter-what-ability?
What’s the point in F-M-E?
But what about the need to train?
Don’t your staff ever complain?
[Ye Gods - this man has half a brain]
My staff all have a PhD,
And they can’t train, they’re too busy,
In fixing data quality,
There’s no time for F-M-E!
What if your data….. oh, but hark!
Is that the trilling of a lark?
No it’s my Blackberry…
A call from our most valued customer…
…you want what?
…when?
But…
That data could be here or there,
That data could be anywhere!
That data could be from Japan,
Spain, China, France, or Vietnam!
That data might be polylines,
paths or rasters, text or splines!
That data, it might need
transforming,
Cleaning up, to be conforming.
That data we might need to fix,
To join it up to Informix,
That data could be MID and MIF,
Shape, Idrisi, GeoTIFF!
Oracle or Map 3D,
S-D-F or C-S-V,
N-T-X or Smallworld 4,
Shape, or SAIF or hundreds more!
And what about the need to train?
My staff are going to complain!
[Ye Gods - I must have half a brain]
mmmmmm…
OK. I will try your F-M-E.
Can I be an evaluee?
Wow!
My F-M-E I’ll never trade,
Now I’ve drunk that cool kool-aid
I’ll use it when my data’s there,
Or when my data’s anywhere
I’ll read that data from Japan,
And Unicode from Vietnam
And if my data’s awkward splines,
Workbench makes me polylines
And when my data won’t conform,
Transformers turn it to the norm
And if my Informix is full,
We’ll join instead to Oracle
Translating some S-57,
Is truly a foretaste of heaven
But if my raster’s SID or SPOT,
F-M-E won’t care a jot
And it’s so easy for to learn,
We’ll finish quick, with time to burn
YES! I love your F-M-E,
Thank you
Thank you
I’m filled with glee!
December 21st, 2007
1) @Tcl2 Temporary Filenames
This is a new ability (build 5147+) with our @Tcl2 function to generate temporary file names within a TCL script. The syntax is FME_TempFilename [] []
While this will be useful to some users, for Safe Software, it allowed development of a new transformer….. the ImageFetcher…
2) ImageFetcher Transformer
This new transformer (build 5147+) - when handed the URL of a TIFF, PNG, GIF or JPEG image - will return that image as a raster feature. The raster feature can then, of course, be further processed or converted in any FME supported manner.
If an error occurs and ‘Continue On Error’ is set to Yes, the output feature will have no geometry and the error message will be logged, but the translation will continue. However, if an error occurs and ‘Continue On Error’ is set to No, the translation will fail.
Above: An image on the Safe website - and as read, written to GeoTIFF and opened in the FME Viewer.
This example can be found on fmepedia at: http://www.fmepedia.com/index.php/ImageFetcher_Example
Again, this will be very useful in its own right, but is really special when combined with another new transformer… the WebCharter…
3) WebCharter Transformer
Just last weekend Google released a new web service through which you can submit numeric data to create a chart - much like you can do with an Excel spreadsheet. So this new transformer (build 5147+) takes a set of data in FME, and turns it into a URL which can be submitted to this Google Chart API (http://code.google.com/apis/chart/)
One URL is created for each feature that enters the transformer. The data for the chart is taken from the list specified, and each element of the list must be numeric. Only one data series is currently supported (i.e. there is no ability to overlay two numeric sets of data on the same chart).
The transformer supports a subset of the Google chart types: Line charts, Bar Charts, and Pie Charts.
How do you submit that URL? With the new ImageFetcher of course!
But also, any format that includes the ability to embed a URL - such as a link on a KML placemark - in effect lets you add a dynamic chart to the data!
Left: Interopolis parks data, converted to a Google Chart URL, read with the ImageFetcher, written to GeoTIFF and opened in the FME Viewer.
This example can be found on fmepedia at: http://www.fmepedia.com/index.php/WebCharter_Example
4) HTTPS Support
On the web theme, a number of users have been asking if they can read data from HTTPS URLs - as opposed to an open HTTP site. The answer is yes …. but for the moment Safe has to be very careful about how and where we ship it.
So if you have a burning need for this, then please email support@safe.com with your request and we’ll work something out.
5) FME on YouTube
I’m sure you all were wondering why we didn’t have it before, so here you are - a YouTube channel for FME users. Since there is a choice of account type - one of which is ‘Guru’ - the group is named the FME Guru.
At the moment, we only have one movie up there - the transformer drag-and-insert demo - but we hope to get some of Ryan Cragg’s excellent movies there soon, the video of GIS day, any small demo movies we do in the future, and perhaps some training exercises (time permitting).
See: http://youtube.com/FMEGuru
6) New GeoConcept Support
The current FME2008 beta version has an updated reader to read GeoConcept data. For those who aren’t familiar with this format, GeoConcept is a French GIS application
The reader updates are based on new functionality provided by GeoConcept. The key benefits are that the reader is a lot faster (3x faster in recent tests) and that you no longer need to have GeoConcept installed to make use of the reader.
The new reader also supports GeoConcept v6, but is (with a few small differences) fully backwards compatible with older versions.
7) AutoCAD_OD Writing
If you thought FME’s AutoCAD Object Data reading was impressive, then wait till you see writing! In a sneak preview of this functionality I was able to see how we can convert attribute data into object data - so easily you wonder what all the fuss is about!!!
Breaking News: In the background I can hear Dr. Lutzenstein saying, “Igor! Throw the switch Igor” which means this should be in a beta by this time next week.
8) Advanced Format Handling
There are a couple of components for handling format reading and writing at an advanced level which are worth mentioning. Both of these emphasize how workspaces might be generated by a developer but used by a lower level user without them having to be aware of the technology used.
The generic writer is a destination writer whose format is not determined until the translation is actually carried out. This allows a developer to set up a generic workspace capable of writing to any format the user decides.
The “Feature Types to Read” parameter allows a user to decide which source feature types will be read during the translation. The benefit is that the user doesn’t have to edit the workspace and disable non-required source layers, they can be defined in the run-time dialog.
Although these items are not particularly new - they were in FME 2007 - they are particularly topical because they come in so useful when deployed using FME Server. And what is new is that they’ve been added to the FME training course in a special section on format translations. So anyone interested in advanced format handling with either of these functions can find screenshots and examples by taking FME 2008 training next year.
9) 3D Elements in the Creator
As of build 5145 the Creator transformer now includes the ability to create 3D related objects such as Boxes, Faces, Triangles etc.
Above: A Creator creating a box feature… [actually the coords in the screenshot look wrong... perhaps a bug where they get reset on reopening the dialog?]
Below: …the same box feature as written to PDF and opened in Adobe reader.
This week’s Evangelism Weekly was written to the tune of…
The Stylistic’s “Can’t Give You Anything”. A great, upbeat, funky Philadelphia soul track.
It’s in the Itunes store, but if you’re too cheap to pay 99c, here’s a YouTube link…
http://www.youtube.com/watch?v=LPTbzPHkEGA
Have feedback?
Email FME Evangelist, Mark Ireland at mark.ireland@safe.com.
December 14th, 2007
1) StatisticsCalculator Transformer Group-By Option
New in FME this week is a group-by option for the StatisticsCalculator. This has been a big user request for a while now, and will let users calculate max, min, mean etc for groups of features, rather than an entire dataset.
There is a demo workspace for this which you can find on fmepedia at: http://www.fmepedia.com/index.php/StatisticsCalculator_Example
It’s just a small demo to find average annual toothpaste expenditure for each postal code in the fictional city of Interopolis.
This is the sort of thing that a census related customer like US Census or UK ONS might find of interest, as well as marketing companies interested in toothpaste; but there are many other instances where it comes in useful for example helping to create feature IDs by finding the max current value, or counting the number of items in feature groups.
User group comments:
“That _is_ cool.”
“Wow, that’s good news for all who are used to SELECT <aggregate_func> FROM … GROUP BY … in SQL”
2) FME Licensing on Linux
For the longest time we didn’t allow the use of floating licenses with Linux (I understand that was for business rather than technical reasons).
However, as of Build 5136 for Linux, FME on Linux can now use a FlexLM license - it can connect to a license server on Windows, Linux, etc, and a license server can be installed on Linux as well.
As Dan Ise said on the user group, “many of you have asked about this in the past, and now it’s here.”
The setup procedure should be the same as for a UNIX license, and the FlexLM packages can be found at ftp://ftp.safe.com/fme/floatingLicense/
3) FME and Python
A number of new Python items this week, some of which you may know about but they’re worth re-iterating.
Misc Python Fixes:
Python paths not logged multiple times (PR#10801)
Fixed python path update (PR#13862 PR#9773)
Fixed population of FME_MacroValues global object (PR#13798)
New Support:
As of build 5142 we now support Python 2.5
New Tutorial:
As you probably saw, Oliver Heimann - a user from Germany, earned himself several gold stars by creating a superb Python Tutorial for FME.
You can find Oliver’s Python Corner on fmepedia at: http://www.fmepedia.com/index.php/Oliver%27s_Python_Corner
It includes info on accessing attributes, using FME functions from within Python, startup and shutdown scripts plus much more.
NB: This is a great example of the sort of knowledge we’d like to be able to extract from our users - so if you are in touch with a customer doing cool stuff beg them to share their methods (on fmepedia or any other way)
4) Transit Ridership
On the subject of great user contributions, the user group had a new message this morning from Graeme Brown, a planning student from Halifax who acts as a consultant to Translink.
He has a great looking output of bus ridership in the Vancouver area - created using FME and the KML writer - that rivals Dmitri’s work in 3d displays.
The description of the workspace can be found on the user group at:
http://groups.google.com/group/fmetalk/browse_thread/thread/df6aec38810e7b15
If anyone in marketing or whatever wants to get in touch with Graeme his email address is: graeme.a.brown@gmail.com
User Group Comment: “That’s sweet… Are you going to be presenting this at the FME UC?”
5) Converting LIDAR to PDF
Having had to do this for a user I thought I’d share this example. The point is that the PDF writer is not only useful for 3d building models, a user can convert a point DEM to a TIN very easily, write it to PDF, and view it in 3D. If and when - as was talked about in yesterday’s lunch and learn - we can add textures to surfaces this would look even better!
6) CSGs
Speaking of the lunch-and-learn, if you didn’t see it you ought to know that we now (build 5137+) have a new factory and function that support CSG (Constructive Solid Geometry) boolean operations between solids. Also, as of build 5141 these CSGs are now supported by the PDF writer and the Geodatabase writer.
For a diagram that highlights this much better than I can explain see the fount of all knowledge, wikipedia…
http://en.wikipedia.org/wiki/Constructive_solid_geometry
FME User Group Comment:
“Ohhh. Does this mean that we can author Quake maps using FME?
I could see some real utility in being able to transform GIS data into online virtual worlds… :) ”
7) SQLExecutor Example
A new item on fmepedia this week is an SQLExecutor example workspace by Robyn.
It’s a good example of reading attribute data and attaching it to a set of existing spatial data - with the twist that it’s using the SQLExecutor rather than the Joiner - and how to use lists when the result of the query is more than a 1:1 relationship.
See: http://www.fmepedia.com/index.php/SQLExecutor_Example
8) Free FME FDO
A final item this week is more for support type dudes - the marketing folk know all about this, but I know some folk at this end of the office missed out on the announcement.
On November 27th Safe announced the release of a free FME FDO Provider for AutoCAD Map 3D 2008 which includes 9 formats.
The product was apparently demo’d at the Autodesk University in Las Vegas, and is available for download from http://www.safe.com/fmefdo
For more info I imagine Jaylene or Fil could help, or see the story at:
http://www.safe.com/company/news/2007/108/index.htm
Have feedback?
Email FME Evangelist, Mark Ireland at mark.ireland@safe.com.
December 7th, 2007
RSS 2.0