FME Blog

How to Choose the Right FME Transformer for your Data Challenge

Stephanie Warner

July 24, 2018•6 min

Let’s talk about jam for a minute. In 2000, a pair of psychologists conducted a study where they arranged a display table of 24 kinds of jam, and the next...

Let’s talk about jam for a minute. In 2000, a pair of psychologists conducted a study where they arranged a display table of 24 kinds of jam, and the next day a similar set-up with only six varieties of jam. While the large display attracted more attention, the small display produced more purchases. This led to the conclusion that too many choices can leave people paralyzed by decision—and sometimes, less satisfied with the result.

However, more choices empower us to achieve better outcomes—if you have the choice between 100 different hotels, you’re more likely to find the one that suits your needs and budget.

We know FME has a lot of transformers. With over 400 to choose from, starting with that blank FME Workbench canvas can be paralyzing. But as you narrow down the purpose of each transformer, the choice reduces to a nice-sized jam table, empowering you to devour any data challenge that comes your way.

The trick to knowing which FME transformer to use comes from hands-on experience and understanding what your output needs to contain—for example, cleaned-up data, dynamic feature classes, or multiple PDF pages. When searching for a transformer, you can type the desired output, like “filter” or “clip”, on the FME Workbench canvas or in the FME Transformer Gallery and get a list of transformers that can help you accomplish your goal. The Transformer Reference Guide is a great way to learn about what transformers are best suited to a particular task.

Tip: Typing on the FME Workbench canvas automatically searches the FME Transformer Gallery and FME Hub.

For our recent webinar, we invited users to submit their scenarios. Dale (Safe co-founder) and Iris (FME Server web developer) then walked through the process of solving each problem in FME and how to decide which FME transformers to use. Here are some interesting problems they solved:

Scenario 1: Geospatial PDFs: How to create a multi-page PDF containing map data

The challenge: To produce a multi-page PDF. Each page needs to contain a photo, a title and signature, and descriptive info containing the location of a building footprint.

Key transformers used to solve it:

Counter: Provided a page attribute that allowed the PDF Writer to know what page to put features on. This was the trick to writing to multiple PDF pages.
SpatialFilter: Filters out the undesired data features (CAD building footprints), giving the building polygons and attributes from each point.
SpatialRelator: Used to find the raster orthophoto footprint that the building is in.
PDFPageFormatter: To layout the PDF page in the desired output format.

Scenario 2: Creating Dynamic FME Workspaces

The challenge: A SQL Server database has tables containing multiple versions of the same dataset that are then written out as Shapefiles. The goal is to produce a dynamic workspace that can handle revisions in source data, like adding more tables, and can offer different outputs for historical and current data.

How we solved it:

To allow for dynamic source data, Reader properties were set to “Merge Feature Type.”
Tester transformer checked the table name to decide whether to write the feature to the Current or Historical directory.

Scenario 3: How to Generate DEMs and Hillshades from LiDAR files

The challenge: Automating the process of generating 4000 LAS files into DEMs and hillshades.

The solution came in the form of dynamic workflows and transformers like:

NumericRasterizer: Reduced the data volume by rasterizing the point cloud to a 10×10 resolution.
RasterHillshader: Generated the hillshades.
RasterDEMGenerator: Generated the DEMs.
RasterMosaicker: Prepared the output by mosaicking all the raster features.

Scenario 4: How to convert Area feature class to Line feature class

The challenge: To replace path polygons with lines.

Transformers used to solve it:

CenterLineReplacer: To do exactly that. In FME 2019 it will have a new mode called “Approximate Centerline”, which is substantially faster and more stable.
Dissolver: Cleaned up the geometry.

Scenario 5: Analyzing Radar Images

The challenge: To analyze raster images of historical radar, to compare the value in the radar image with a linear feature of an electrical line, and to know how often the linear feature intersected with a certain level of radar activity.

How we solved it:

The solution came from keeping the data in raster form as long as possible (to reduce data volume) and with the help of several essential transformers:

RasterBandNodataSetter: To clean up the empty values in the input rasters.
Sorter: Used the numeric component of the images to read them in order.
3DForcer: Before turning the linear feature to a raster, give it an elevation (so the linear features show up in the raster).
NumericRasterizer: Turned the vector features into rasters.
RasterExpressionEvaluator: Found the intersecting cells between two rasters.
StatisticsCalculator: Calculated the average interval.

Scenario 6: Bulk Reading and Writing CSV Files

The challenge: To read as many as 13 different CSV files (all with the same name), decide which feature type each CSV file refers to, and to write records for each CSV file to the appropriate set of fields in a database.

How we solved it:

Reader parameters set to “Single Merged Feature type” for all CSV files, so the workspace reads the union of all possible fields.
TestFilter: To filter data to differentiate each CSV file.
AttributeKeeper: For each record type, keep only the attributes that apply to a specific CSV.

Our webinars are a great place to see our transformers in action. Watch some of the other scenarios we solved:

Are you stuck on a scenario? Share your problem with our community of users and experts on the FME Knowledge Center. To learn the basics of FME transformers, try one of our free online training courses and get hands-on experience.