November 30th, 2009
This is a fascinating story about the world of 3D data. It begins…
“Once upon a time, a geo-nerd was converting spatial data into a 3D model. He (or she) wanted to record the coordinate transformation parameters so the data could get home again. Unfortunately, a trail of breadcrumbs didn’t work, nor did a WLD file. So that’s when a fairy godmother from the core team at Safe Software waved a magic wand, and came up with a solution!”
What our geo-nerd found was that output to a 3D model may need to be compressed to a small area around an origin of 0,0. This is because some end-applications can’t handle large coordinate values; they run out of precision and end up collapsing geometries into single points.
So when FME writes to certain formats we (optionally) squash the data between the coordinates [-0.5, -0.5, -0.5] and [0.5, 0.5, 0.5]
We also refer to this as model space or, (because we like long words) a normalized coordinate system space.
Above: Imagine your entire dataset squashed (normalized) into a small area where x,y and z all have a minimum value of -0.5, and a maximum value of 0.5.
Storing the Georeferencing
To the geo-nerd, logic suggested that when he normalized a dataset into a local grid, he should store georeferencing information so that it can revert back into its true spatial position.
A traditional folk tale geo-nerd would no doubt record the information as a binary pattern of breadcrumbs, however this one was more technologically advanced and tried to use a World (WLD) file.
You’re probably aware a world file stores georeferencing information. According to my research there are two varieties: the first (which Wikipedia explains) are used purely for raster data; the second are often used for CAD datasets such as DWG and DGN files. If you have a spatial dataset with a world file then FME honours that georeferencing when reading the data (as do other applications).
The problem our hero faced is that world files are only capable of storing 2D transformation points.
This is a problem because if you can only record the XY of the model into unit space, not the Z, then you have no georeferencing for elevations. But if you don’t correct the data you end up with a model bounding box like [-.5, -.5, 0] and [.5,.5, 1000]. You can imagine what that looks like when you visualize the data: extremely tall, narrow buildings.
So, instead of writing world files, for 3D georeferencing information FME now waves its magic wand and stores the information in FWT files.
FWT files contain a 3 by 4 transformation matrix which defines the parameters required to swap data between its true spatial location and a generalized model space bounded by [-0.5, -0.5, -0.5] and [0.5, 0.5, 0.5].
For the mathematically minded, the matrix is an Affine transformation in row-major format, and if you were to open up the file in a text editor, you would see something like:
4.00 0.00 0.00 500 0.00 4.00 0.00 500 0.00 0.00 4.00 1000
This corresponds to the parameters of a 3 by 4 transformation matrix, in the terminology of the 3DAffiner transformer:
A B C D E F G H I J K L
If you are interested then this article explains all you might want to know about 3D matrices. In particular it explains the structure of our FWT files (the section titled Extending to Infinity), and talks about matrices in nice ways such as comparing the relationship between someone standing up, and someone lying down!
Are FWT Files a Recognized Standard?
Let’s be clear: FWT files are not a recognized standard for spatial data. Only FME will use them.
We don’t really like creating ad-hoc standards, but no existing solution seemed to fit the bill. The important part is that we can unambiguously describe the transformation. Formally speaking, an FWT file has enough degrees of freedom to uniquely describe a 3D transformation.
Anyway, perhaps in creating a solution, we’ve defined a new standard for others to use?! The structure is simple enough and a matrix is a mathematical standard. Personally, my favourite part is the complete absence of XML!
What About Coordinate Systems?
Good question! What WLD/FWT files do is describe a transformation independent of coordinate system.
Storing Coordinate System information in a prj file by itself is not enough, because it only records what system the data was held in. It doesn’t specify how the data was transformed to get it into its local position. So what you need is the FWT file and a coordinate system reference (such as a PRJ file).
What would work is a coordinate system which describes the local grid. Then the data could be reprojected back and forth. The first problem is that this would need to be a custom system which would differ for every translation. The more serious issue is that this would require true 3D coordinate systems, and very few applications support this. Not even FME does this (yet).
Using FWT Files in FME
I think all this sounds more complicated than it actually is. In practice FME makes the whole process very simple. To demonstrate this in action, let’s take this small workspace:
What it does is read some line features, turn them into polygons, and extrude the polygons in blocks of random height. It’s meant to generate a random city skyline. The data is written to Collada format.
The Collada writer has a parameter which determines whether I want to compress my data down into a local coordinate system. So, for the purposes of this example, I said yes.
Now when I run the workspace, I not only get the Collada model, but two other files:
The prj file tells me what the original coordinate system was, and the fwt file contains the mathematical matrix by which the data was compressed into local coordinate system space. In case you’re interested, it looks like this:
7204.75 0 0 3127915.458333 0 7204.75 0 10096319.875 0 0 7204.75 499.5
When I import the data into SketchUp I can see that it is nicely arranged around the 0,0 coordinate (ie it is now in a local coordinate space)
However, when I want to read it back with FME (say into the new FME Data Inspector) then I have the option to move it back into true geographic space
…meaning that FME will use the prj and fwt files to ensure coordinates are all in their original coordinate system
So that’s how FME helps users of 3D data models live happily ever after!
I hope you enjoyed this story. Many thanks to Brittany and Kevin in our core team for explaining the parts which didn’t really make sense to me the first time around.