1) Historical Markers
2) Complex Annotation Geometries
3) ArcGIS Image Server
4) Persistent SDE Connections
5) Text Size and Point Size
6) Quick Notes
In honour of the 2008 ESRI International User Conference, this post is a special on recent ESRI related updates in FME.
I think a key point of FME is how it easily integrates into spatial data workflows, simplifies processes, and helps people concentrate on using their data (rather than writing scripts or mucking about with text editors!). Therefore, for us at Safe, it’s vital that we keep our format support as up to date as possible; as even the smallest piece of unsupported functionality can force a user to take huge detours with their data.
That’s why it’s a real pleasure to be able to highlight improvements like these below. Browsing our user requests database I saw a number of funky ArcGIS functions - most of which I didn’t even realize existed - that we’d been asked to support. And here they are! Hopefully each one of these newly-implemented features will be a real time-saver.
The one caution is that, because these functions are newly added, they are very much still in beta form and subject to change and improvement as we get user feedback.
Historical markers are part of an archiving process introduced in ArcGIS v9.2 for ESRI’s SDE and Enterprise Geodatabase.
The basic idea is that - once activated - all database edits get stored in a separate archive class, with a date and time stamp. When viewing data you can - by specifying the appropriate time range - read the state of the spatial data as it existed at any particular moment in time. Additionally, an Historical Marker Manager lets you define moments in time with specific names that will obviously have greater meaning than dates (for example “As Designed” and “As Built”).
In FME2009 (as of beta build 5568) we now support querying data on the basis of these historical records. This is achieved by either selecting an historical marker on adding a reader, or entering a query into the appropriate reader parameter in an existing workspace.
Below: When adding a Geodatabase (SDE) reader to the workspace, the settings dialog includes this section. Here you can search out and specify a particular historical marker:
Below: Or you can choose to view data at a particular point in time by date:
Below: Once the reader is added to a workspace you can access historical data by setting the Archive Where Clause parameter:
Below: The Archive Where Clause is edited with this dialog. This example is a “range query” - ie the from and to dates are different.
According to the Readers and Writers Manual, a query can be a ‘moment’ query (the from and to dates are identical) or a ‘range’ query (the from and to dates are different). ESRI documentation tells me it’s also possible to query archives using attribute values (i.e. show me when <attr>=<value>) and Historical Markers, so - although I’ve no evidence - I believe this might also work in this FME parameter.
Complex Annotation Geometries
When, a few years ago, improvements were made to the FME geometry model, it enabled a whole deal of functionality which is still being unlocked. Support for complex ArcGIS annotation features is one of these.
By “complex” annotation geometries, we mean any text that is not a single piece of straight-line text, that originates at a specific point; i.e. text that is aligned to curves, arcs, polygons, multipoints, etc.
So, as of build 5555 we now support reading and writing of annotation with complex geometry.
Below: Bonus points for spotting the language of this curved annotation string.
Key things to remember:
- You MUST have the Geometry Handling parameter (shown below) set to “Rich”. You won’t be able to write complex annotation geometries (and maybe not read) if it is set to “Classic”.
- The data inspected in the FME Viewer will not show curved text - that’s because it isn’t supported (i.e. the Viewer is only meant for data inspection, not visualization)
- Splines and curves will be stroked into line features; we do hope to properly support these geometries in the near future.
Above: The Geometry Handling Mode must be Rich (not Classic)
ArcGIS Image Server
As of build 5550 Safe has added a new reader to FME: ArcGIS Image Server.
ArcGIS Image Server (or as of 9.3, ArcGIS Server Image) serves up file-based raster data to other applications. There are a number of applications with interfaces to use this data, including ArcGIS, AutoCAD and MicroStation; but the data can also be delivered as a web service, and this is where the FME reader might come into the picture.
Using FME on top of ArcGIS Server Image would firstly extend the format reach, secondly open up the data to all the raster transformation tools in FME, and thirdly allow everything to be integrated with all FME ETL functionality (for example, the ability to merge in vector datasources, live feeds, other raster data, text-based data etc etc) all “on-the-fly”.
The ArcGIS Server Image format is a reader only, since the product itself doesn’t have it’s own format (we read from a service). However for writing, it (the ArcGIS Server Image product) supports as source many of the same formats FME supports as a destination, plus I strongly suspect it would also accept a service as input, so you could create an “on-the-fly” FME writer using an FME Server service as an ArcGIS source.
NB: To get access to this format requires installation of the ArcGIS Image Server Client Core.
Persistent SDE Connections
Whenever FME connects to an SDE database it carries out the require processing and then shuts down the connection to save resources. However, when multiple tasks need to be carried out, creating a new connection each time is an expensive operation.
So that’s where a new keyword for FME2009 comes in. The PERSISTENT_CONNECTION keyword specifies whether to leave database connections open once their task is complete. That way other operations can use the same connection and avoid re-connection overheads.
Below: Persistent Connections are defined via a Reader Parameter:
This keyword isn’t intended for normal standalone translations, but for when the workspace is used in “super batch” mode or on FME Server; in these cases the reported improvement in connection time is a huge 50-60%. But just remember that an open connection is, effectively, an intentional one-time memory leak, so if you are not using SDE access in all translations it may not be worth the cost in system resources.
Text Size and Point Size
A curious discussion occurred here recently about whether “text size” and “font size” were the same thing, or different. It appears (from what I understand) that they are different; text size is the size of a text feature in ground (coordinate system) units, whereas font size is the size of the font to use in “point” units.
Therefore users were disappointed to find out that setting the format attribute geodb_text_size had no effect whatsoever on the size of the output font; and because font size is unrelated to geometry, the attribute geodb_text_ref_scale had no effect either.
The solution was for us to create a new format attribute: geodb_font_size
geodb_font_size defines the size of the font used to display the text string, in point units. However, there are some important points to note:
- Once more, you MUST have the Geometry Handling parameter set to “Rich”. The attribute would be ignored if the geometry handling mode is set to “Classic”.
- For the sake of backwards compatibility, geodb_text_size will overrule geodb_font_size if both are set. So remove any reference to geodb_text_size if you are wanting to set a text font size.
This fix was so important for a number of users we backported it to an updated FME2008 version. For more information see the Post-CD Fixes page on fmepedia.
- The “Split Multi Part Annotations” option available on the Geodatabase_MDB reader has also been exposed on the SDE Geodatabase and File Geodatabase readers as of build 5564.
- An equivalent format attribute for the ArcGIS parameter “Character Width” has been created as of build 5531. It is called geodb_text_character_width
- The personal geodatabase metadata attribute, fme_num_entries, is also exposed in the file geodatabase reader/writer as of build 5529
- Did you catch Don Murray and Mark Stoakes presenting a pre-conference seminar on Data Interoperability? If not, don’t worry. Safe-hosted ArcGIS Data Interoperability Training will take place in Denver, Houston and Washington DC in the coming months. See the Safe training calendar for more information.
This Edition of the FME Evangelist…
…was written to the tune of “Greetings to the New Brunette” by Billy Bragg.
One of Britain’s best singer-songwriters. This is a slightly bizarre video, but you can’t argue with lines like:
“And if you hadn’t noticed yet, I’m more impressionable when my cement is wet.”
“How can you lie there and think of England, when you don’t even know who’s in the team?”
August 6th, 2008