What’s Great 2014 – FME Desktop

2014 promises to be an exciting year. I’m not just talking about the scheduled test launch of NASA’s Orion, or the 22nd Winter Olympics, or even the fact that it’s the International Year of Crystallography (who knew?!). It’s also the year of the FME International User Conference 2014 – the first FME UC since 2009, and projected to be bigger and more exciting than ever.

Of course, no New Year would be complete without the launch of the best FME to date.

“What’s Great” is a technical summary of all the new and updated applications, functionality, and formats in a particular version of FME. Expand the sections below to explore What’s Great in FME Desktop 2014, and go here for FME Server 2014.

The new FME 2014 Workbench splashscreen

Click the section headings to expand them.

[accordions title=”” disabled=”false” active=”” autoheight=”false” collapsible=”true”]

[accordion title=”General”]

NULL support

• Improved the way FME reads, writes, transforms, and inspects nulls. Read about it at fme.ly/nulls. This includes a list of affected formats and transformers.

Point clouds

• Added support for arbitrary component data types and user-defined components. FME 2014 has a completely overhauled Point Cloud core that features a flexible data model. In FME 2013 and earlier, the Point Cloud data model in FME was “fixed”. In FME 2014, every point cloud can define its own set of components (measures per point), and each measure can have its own type (int, float, etc.). With this update comes new and updated transformers for manipulating this data model (see the Transformers section below).

Python

• Improved exception logging: if there is no access to a logfile, Python exception information is printed to stderr.
• Upgraded to 2.7.5.

Cross-platform compatibility

• FME Desktop is now available as a technical preview on Mac and Linux operating systems.

[/accordion]

[/accordions]

FME Workbench underwent a major visual overhaul this release. Be sure to check out fme.ly/workbenchTop10 to read more about the top 10 most exciting Workbench improvements, and explore The Hidden Treasures of FME 2014 to learn about enhancements to the FME Desktop interface.

[accordions title=”” disabled=”false” active=”” autoheight=”false” collapsible=”true”]

[accordion title=”Workbench”]

• If a transformer only has one Input port named “Input”, that port is now hidden.
• Changed transformer port names from ALL_CAPS_FORMAT to CamelCaseFormat (for example Clipper rather than CLIPPER):

• Added decorations to transformers that operate outside the normal pipeline. Transformers that produce features have a left-side decoration. Transformers that consume features have a right-side decoration.
• Changed ‘…’ parameter buttons to gears.

• Improved annotations: rich text editor, resizable, color choice, word wrap, hyperlinks.
• Improved bookmark behaviour – magnets are off by default; objects are deemed to be within a bookmark even if they only overlap and are not fully contained.

• Added ability to right-click to cancel pan/zoom mode.
• Added ability for selection tool to pick up all objects it overlaps (rather than objects it encompasses fully). This can be done by holding the Alt key while selecting, or by checking a box in FME Options.

• Added buttons on autoconnect to allow undoing/changing the connection.

• Added ‘help’ option to all right-click menus.
• Added ability to edit dataset path by right-clicking a feature type node. This is also seen in the new ‘reader parameters’ and ‘writer parameters’ option in the right-click menu for readers/writers.

• Added concept of “Dynamic Input Ports” to InlineQuerier, MaptextLabeller, MapNikRasterizer.

• Added ability to click anywhere on connection line to move a connection.

• Changed how constants are set and added the ability to change attribute values based on an expression.

• Updated settings boxes to automatically add network authentication parameters and network proxy parameters if dataset is using http, https, or ftp protocol.
• Enabled sorting by clicking column header, and added sorting to choice dialogs.
• Added ability to move selected table row(s) to top or to bottom.
• Added collapsible parameter panels to dialogs.

• Changed Transformer Description tab to “Help”, and display help for all canvas objects.
  • Within Help pane, shortcut keys Alt+Left Arrow and Alt+Right Arrow can be used for navigating between previously viewed items.

  

• Added version support for transformers in FME Store.
• Added prompt for option to embed/link all instances of a given transformer.
• Added encryption for passwords saved in workspaces.
• Added password verification when password-protecting workspaces.

Custom transformers

• Added ‘Attribute References’ option when creating a custom transformer to allow output schema management. User can choose whether to create published parameters.

• Added ability to show attributes on input and output nodes of custom transformers.

• Added version support for custom transformers.

• Improved error logging so all missing linked custom transformers are identified on startup.
• Added support for embedding a password-protected custom transformer.

[/accordion]

[/accordions]

The FME Data Inspector gained some highly useful functionality, providing the most detailed and interactive data inspection yet.

[accordions title=”” disabled=”false” active=”” autoheight=”false” collapsible=”true”]

[accordion title=”Data Inspector”]

• Added MapBox and MapQuest background map formats.

• Added “Copy Ground Coordinates” to the context menu for 2D data.

• Added “Save Data As…” and “Save Selected Data As…” functionality.

• Open in 3D mode automatically for 3D-only or Point Cloud-only formats, as well as for datasets containing only 3D geometries.

[/accordion]

[/accordions]

[accordions title=”” disabled=”false” active=”” autoheight=”false” collapsible=”true”]

[accordion title=”Quick Translator”]

• Added support for private parameters.
• Added start page.

[/accordion]

[/accordions]

FME 2014 gained a multitude of formats and transformers for vector, raster, point cloud, database, and web data. Read about the top 10 transformer updates over at the Safe Blog, and visit safe.com/transformers to learn more about each of the new and updated transformers listed below. To learn about the new and improved readers/writers, visit safe.com/formats.

[accordions title=”” disabled=”false” active=”” autoheight=”false” collapsible=”true”]

[accordion title=”Transformers”]

New transformers

• DatabaseUpdater
• DatabaseDeleter
• GeographicBufferer (was previously available in the FME Store)
• JSONFragmenter (renamed from JSONExploder)
• MapnikRasterizer

• NullAttributeMapper (part of NULL changes)
• PointCloudComponentAdder
• PointCloudComponentCopier
• PointCloudComponentKeeper
• PointCloudComponentRemover
• PointCloudComponentRenamer
• PointCloudComponentTypeCoercer

• PointCloudExtractor
• PointCloudReplacer
• RasterAspectCalculator
• RasterSlopeCalculator

• S3Uploader
• S3Downloader
• SNSSender
• SQSReceiver
• SQSSender
• VertexCreator
• WebSocketSender
• WebSocketReceiver

Deprecated transformers

• AttributeAccumulator (merged into Aggregator)
• NullAttributeReplacer (replaced by NullAttributeMapper)
• NullAttributeRemover (replaced by NullAttributeMapper)
• PointCloudColorScaler (replaced by PointCloudComponentTypeCoercer)
• 2DPointAdder, 2DPointReplacer, 3DPointAdder, 3DPointReplacer (replaced by VertexCreator)

Updated transformers

• AreaBuilder
  • Changed name of port UnusedLine to Incomplete.
• AttributeCopier
  • Added “default value” for if the attribute being copied doesn’t exist.
• CenterLineReplacer
  • Removed Classic mode.
• Clipper
  • Performance improvements when clipping large polygon features (large areas with many holes).
• CSGEvaluator
  • Added Rejected port.
• DateFormatter
  • Updated to let you define source date format, as well as destination

  

• Dissolver
  • Performance improvements when dissolving large polygon features (large areas with many holes).
• FeatureMerger
  • Added “Suppliers First” option. When the supplier data is small, and the requestor data is large, this allows the FeatureMerger to avoid waiting for and buffering all the requestor features.
  • Added column to Join On table to allow selecting String or Numeric comparison for each Supplier/Requestor pair.
  • Added option to prefix supplier attributes.

• ImageFetcher
  • Added an “Auto-Detect” option to the “Image Type” parameter.
• ListBasedFeatureMerger
  • Added Comparison Mode parameter (string or numeric).
• FME Server Transformers
  • Added support for published parameters in the wizard for connecting to FME Server.
• GeometryFilter
  • Ports are now user-selectable. By default, only <Unfiltered> port is provided, requiring port selection. During workspace generation when GeometryFilter transformers are added, only connected ports are shown.

• GeometryPropertySetter
  • Added support for UTF-8 encoded characters, so non-English characters can be used in geometry names.
• GeometryReplacer
  • Added option to remove attribute after use.
  • Allow geometry source to be something other than an attribute, i.e. text or parameter.

  

• GeometryValidator
  • Added option to repair holes that are touching the boundary at a single point (self-touching polygons). This was added as a parameter for the Self-Intersections in 2D option.
  • Faster 3D self-intersection check.
• Intersector
  • Added caching support for clipping case to rich path; removed unused SelfIntersect parameter.
• JSONExploder
  • Renamed to JSONFragmenter.
• JSONExtractor
  • Added multiple query support. Updated transformer GUI to include a table, so multiple JSON queries may be executed in one transformer.
• KMLPropertySetter
  • Added “Additional Files” parameter to allow a KML feature to list the files required for its description to display correctly: images, javascript, css, etc.
• Labeller
  • Added support for expressions instead of only attributes.
• LabelPointReplacer
  • Added support for expressions instead of only attributes.

• ListRenamer
  • Allows renaming of complex lists.
• MapTextLabeller
  • Added “Unlabeled” port to output unlabelled features.
• Matcher
  • Added Match Count Attribute on SingleMatched Output parameter, which records how many features matched for that group of features.
• MeasureSetter
  • Allows the LINE flavour of GET_MEASURES and SET_MEASURES to work on arcs and paths.
• NeighborFinder
  • Automatically sorts candidate lists by distance.
• Offsetter
  • Added support for offsetting point cloud components.
• PointCloudFilter
  • Added option specifying whether to output unfiltered points.
• RasterReplacer
  • Added option to remove attribute after use (like GeometryReplacer).
• RasterToPolygonCoercer
  • Added option to extract nodata values.
• RubberSheeter
  • Added Constraint input port.
• Scaler
  • Added support for scaling point cloud components.
• SectorGenerator
  • Allows Max Distance Between Site Points to come from an attribute or an arithmetic expression.
• ShortestPathFinder
  • If there are no network features for a given group, now outputs all From-To lines as Unused.
• Snapper
  • Added new mode for segment snapping.

• Sorter
  • Allows publishing of sort attribute.
• SpatialFilter
  • Renamed Base port to Filter.
  • Allows predicates to come from attributes.
  • Allows spaces in output attribute names.
• SpatialRelator
  • Renamed Base and Candidate ports to Requestor and Supplier.
• SQLExecutor
  • Added support for multiple result sets.
• StringSearcher
  • Allows searching an expression rather than only an attribute value.
• SurfaceDraper
  • Added new Existing Elevation option to offset Z values.
• SurfaceModeller
  • Added new Existing Elevation option to offset Z values.
• WebMapTiler
  • Added attribute specifying the total number of tiles created from the input raster.

[/accordion]

[/accordions]

[accordions title=”” disabled=”false” active=”” autoheight=”false” collapsible=”true”]

[accordion title=”Format readers/writers”]

New format readers/writers

• Aeronautical Information Exchange Model (AIXM) 5.1 writer
• Amazon DynamoDB reader/writer
• Amazon Redshift reader/writer (includes adding this format to SQL + DB transformers)
• Autodesk Revit reader
  • Including the FME Revit Exporter for Autodesk Revit.

• CARIS Spatial Archive (CSAR) reader/writer
• Esri ArcGIS Online Feature Service reader
• Esri Geodatabase (ArcSDE Geodatabase Raster Catalog) reader
• Esri Geodatabase (File Geodatabase Raster Catalog) reader
• Google Cloud SQL Non-spatial reader/writer
• Google Cloud SQL Spatial reader/writer
• Google Maps Engine (GME) Tables reader/writer
• Google WebP reader/writer
• INSPIRE GML writer
• Landmark Z-Map Grid reader
• Landmark Z-Map Vector reader
• OGP P1/90 Post Plot Positioning Data (P190) reader/writer
• SeabedML (GML) reader/writer
• Trimble SketchUp reader/writer
• Virtual Reality Modeling Language (VRML97) reader
• Windows Azure SQL Database Non-spatial reader/writer
• Windows Azure SQL Database Spatial reader/writer

Updated format readers/writers

• Esri ArcGIS Layer reader
  • Enhanced to read symbology/styling attributes from a layer.
  • Added fme_color and fme_fill_color support for ‘color ramped’ and ‘unique value’ symbology.
• Autodesk AutoCAD DWG/DXF and RealDWG reader/writers
  • Added unicode character encoding support.
  • Added support for reading in multiple layouts from the paper space.
  • Added raster reading and writing support.
  • Updated to 2014 libraries.
  • Added reading of AutoCAD drawing spatial reference system as FME coordinate system.
  • Added xref reading and writing support.
  • Added leader entity writing support.
  • Added xref reading and writing support.
• CityGML writer
  • Added option to choose the preferred output format for textures.
• CSV reader
  • Default setting for “File Has Field Names” parameter changed to Yes, so the first row is skipped on read.
  • New sort and filter options on reader parameters let users sort the data and use regular expression to thin the data as it’s read. Both result in performance improvements for reading CSV data.
  • General performance improvements when reading CSV with a large number of columns.
• IBM DB2
  • Added to 64-bit Windows build of FME.
• Bentley MicroStation Design (V8)
  • Added support for element priority for 2D draw order.
  • Added support for writing features to multiple models available in the seed file.
• ERDAS writer
  • Added support for writing multiple string palettes. The writer now supports bands with an RGBA32 palette, optionally followed by any number of string palettes.
• FDO reader/writer
  • Added support for 64-bit integers (int64).
• Esri Geodatabase (File Geodatabase API)
  • Updated API to version 1.3.
• Esri Geodatabase
  • Added support for GUID column types.
• GML reader/writer
  • Moved most parameters to Advanced.
  • Renamed “Read Predefined Properties” parameter to “Map Predefined Properties”.
  • Added ability to write generic GML and use a custom XSD/schema. The “GML Version” writer parameter allows writing a dataset conforming to an existing GML application schema.

• Added warning for orphan features, those with parent ids pointing to parent features that never arrived.
• Changed log details to log XSD parsing only if  “Log debugging information” is checked in Workbench, or if the XSD_VERBOSE option is set on FME_DEBUG.
• Google Fusion Tables reader/writer
  • Enabled ability to retrieve system proxy settings if specified.
• IBM Informix
  • Added to 64-bit Windows build of FME.
• Industry Foundation Class STEP Files (IFC) reader
  • Allows the attribute_id rule to be used as a numeric_expression. Allows parsing of the IFC4 schema file.
• INGR reader
  • Now reads world files.
• INSPIRE GML reader
  • Simplified settings.
• JPEG (Joint Photographic Experts Group) reader
  • Exposed more exif format attributes:
    • jpeg_exif_contrast
    • jpeg_exif_exposureprogram
    • jpeg_exif_focallengthin35mmfilm
    • jpeg_exif_lightsource
    • jpeg_exif_saturation
    • jpeg_exif_sharpness
    • jpeg_exif_software
• Google Earth KML writer
  • Added support for including additional files (images, javascript, css, etc) in output KMZ. (See KMLPropertySetter transformer update.)
• ASPRS Lidar Data Exchange Format (LAS)
  • Added support to delete a source file when the populator is destroyed.
• MapInfo TAB (MFAL/MAPINFO)
  • Renamed format long name from ‘MapInfo TAB (MFAL)’ to ‘MapInfo TAB (MAPINFO)’.
  • Big performance improvements when reading or writing large polygon features (large areas with many holes).
• MapInfo TAB (MITAB)
  • Big performance improvements when reading or writing large polygon features (large areas with many holes).
• Netezza
  • Now detects misconfigured ODBC DSNs and fails instead of corrupting data.
• NMEA Automatic Identification System (AIS) reader
  • Added support for TAG or COMMENT blocks.
• Presagis .flt (OPENFLIGHT) writer
  • Added an option to automatically repair missing vertex normals.
• Oracle
  • Hides versioning metadata tables/views from table list.
  • Added support for mixed-case table names when writing to non-spatial tables.
  • Expanded support for special characters like ‘&’, ‘#’, and ‘.’ in table and column names.
  • Added support for XML columns.
  • Added “Show System Tables” option to reader settings.
  • Added index support to non-spatial columns.
  • Added ability for users to choose whether attribute names are usable without quotes.
• PostGIS, PostgreSQL readers/writers
  • Added support for Amazon RDS hosting.
  • Added support for begin and end SQL statements.

• SGI Image writer
  • Added option to allow specifying an output file extension.
• SQLite
  • Added Int64 support.
• Golden Software Surfer Binary Grid reader/writer
  • Upgraded to version 7 (now supports both versions 6 and 7).
• Landmark Z-Map readers/writers
  • Landmark Z-Map (ASCII) reader deprecated in favour of new Landmark Z-Map Grid reader. There is now a Landmark Z-Map Grid reader/writer and a Landmark Z-Map Vector reader/writer.
  • Added support for writing contour data.
  • Added support for Simple/Print variants (Vector reader).

[/accordion]

[/accordions]

For the developers out there, FME Objects gained some modules and methods for more flexible integration.

[accordions title=”” disabled=”false” active=”” autoheight=”false” collapsible=”true”]

[accordion title=”APIs”]

• FME Objects Python API
  • Added FMECoordSysManager module. This class enables handling coordinate systems in plugin development.
• FME Objects .NET SDK
  • Added Has*() methods to IFMEOAppearance, IFMEOTexture, IFMEOSurface interfaces and subclasses.

[/accordion]

[/accordions]

Lastly, be sure to check out FME 2014’s new coordinate systems and reprojection updates.

[accordions title=”” disabled=”false” active=”” autoheight=”false” collapsible=”true”]

[accordion title=”Coordinate systems / reprojection”]

New coordinate systems

• Germany: Added high precision DHDN grids for Hessen and Rheinland-Pfalz.
• Belgium: Added Belge2008_FME coordinate system.
• Added Antarctic Polar Stereographic coordinate system.

Updates

• Upgraded to CS-Map 14+.
• Added support for reprojectable attributes.

[/accordion]

[/accordions]