Styled Layer Descriptor and Metro Extracts

In WhirlyGlobe-Maply 2.5 we added support for the Styled Layer Descriptor format.  We also added an example in the AutoTester and a tutorial.

Belfast, it seems.

Let's start with the data.

Metro Extracts

Mapzen provides regular extracts of OpenStreetMap data.  These are available in a variety of formats; useful if you just want a little OSM data.

Courtesy Mapzen

We chose Belfast in GeoJSON for our own example.  This is just data, though, so how do we make it visual?

Styled Layer Descriptor

The Styled Layer Descriptor format is an OGC standard used to style data into map images.  It's a bit crusty but, you know, standard.  We edit it in QGIS and export it for use in our toolkit.

WhirlyGlobe-Maply reads SLD and uses it to style Shapefiles, GeoJSON, or Mapbox Vector Tiles.  Any vector data, basically.  So let's put it all together.

Example & Tutorial

Rather than blather about it here, we put a nice example in the AutoTester app for iOS.  In Swift!

Ranen put together a tutorial that walks you through the process.  You can start with a Metro Extract and modify the SLD styles to your heart's content.

Up Next

We're picking up SLD users for the toolkit at a steady clip.  There's a lot more to SLD we could support, so let us know how you like it.

We'd also like to circle back and improve the Mapbox GL Style Sheet support and implement Mapzen's Tangram format.  But we need customers for those.  Speak up if you're interested.

Part 2: Future 3D Support in WhirlyGlobe-Maply

Last time we looked at WhirlyGlobe-Maply's existing 3D support.  It's considerable, but often hard to use.

FlyQ EFB and NATS Airspace Explorer

Now we'll talk about where we're heading.  It's all about the data sources.

Putting the 3D's in Data

Most of our users have fairly limited data processing capabilities.  Say you want to add elevation to an app.  We'd say:

• Gather all the elevation data you need.  USGS is a good source.
• Stitch it all into a single giant image in Spherical Mercator
• Chop that image up into tiles using this random tool we have on github
• Load the resulting sqlite file onto the device.

Easy enough, right?

Yeah, not so much.  That's why you see so few WG-Maply apps with elevation.

It's not just us, you know.  OpenStreetMap data is much easier to use when you move from Planet Files to Metro Extracts and Vector Tiles.

Structured 3D Data Sources

What developers really want is web services that they can plug into their apps.  Also pizza that lifts itself into your mouth.  Pizza robot.  Patent pending.

People want to point their app at an elevation, 3D building, or vector tile service and stuff just happens.  So let's look at some of those services and the formats they support.

Two New Styles of Elevation

Elevation tiles are an obvious thing.  We already support Cesium's elevation, but there are others out there now.

Mapzen came out with a straight up elevation tile service a few months ago.  There are a few minor technical hurdles, but it could be made to work well with the toolkit.

Mapzen Elevation Example

Mapbox came out with their own elevation tile service shortly thereafter.  It's a different approach, with similar problems in 3D but could also be made to work.

Courtesy Mapbox terrain-rgb post

So there you go.  We'd like to support Mapzen & Mapbox elevation data for 3D.

Buildings with Height

OpenStreetMap buildings have height (usually, mostly).  Mapbox has been showing this off lately in their APIs and Mapzen has been doing it for a while in Tangram.

Mapbox Data w/ Mapbox GL

We'd like to support height in vector tile data.  Not too difficult, obviously, but cool.

Now for the harder stuff.  Let's start with a detour into data formats.

GL Transmission Format (glTF)

The 3D file format we support right now, Wavefront OBJ, is old and weird.  We need to move to something new and... less weird.

glTF is backed by the Khronos  Group with lots of work from the Cesium folks.  It's a new, relatively simple optimized model transmission format.  Easy to read, fast to transmit, it solves a lot of problems for models like these.

And it enables some even crazier stuff.

3D Tiles

The Cesium folks have a 3D Tiles proposal out that looks great.  It's open, they've got example implementations, and they're trying to make it an OGC standard.

3D Tile data displayed with CesiumJS

That approach is well suited for mobile.  We can't display as much as a desktop can, but we can display a lot.

Naturally, we'd love to support this in WhirlyGlobe-Maply.

Point Clouds

We already support point cloud display for things like LIDAR.  It's restricted to a Sqlite based format read on the device.  Hey, it's what the customer needed at the time.

San Simeon LIDAR

We'd love to extend this to services like Greyhound, an open source streaming data server.  We'd also like to add support for the GeoPackage point cloud extension.

Summary

There's a ton of explicit 3D support in WhirlyGlobe-Maply for interaction, model construction, and motion.  We'll continue to push in these areas and others we haven't talked about, like 3D weather.  But that stuff is driven by high end client needs.

NATS Airspace Explorer

What we'd really like to do is support these newer services for our regular users.  If you have other services you'd like to see or, better yet, some budget, please let us know.

Mapzen Vector Tile Service

We're going to support Mapzen's Vector Tile Service.  Currently that means we read their vector tiles.  In the future we'll support their style sheets.

Courtesy Mapzen

Let's do a little background and then get to the details.

Vector What Now?

We've been slinging around vectors for years in WhirlyGlobe-Maply so vector tiles weren't a huge diversion.  We even had our own format for a while.

These days vector tiles means Mapbox Vector Tile format, but it implies two other things:

• A curated version of OpenStreetMap suitable for web and mobile map display
• One or more style sheets to describe the visual representation

You might call this a Vector Tile Service if you were so inclined.  Developers want to shove a map underneath the thing their app is actually doing.

It's a trivial, seamless process with an image tile service.  We're going to make it just as easy for a vector tile service.

Here is our roadmap to get there.

Mapzen Vector Tile Service

We can parse and display Mapzen's vector tiles just fine in either GeoJSON or Mapbox Vector Tile format.  But we use really simple style sheets.

Must be night or something

That looks kind of meh.  For a real map we need better.  There are a number of ways to do style sheets, but I'd like something that requires minimal upkeep.

Mapzen has a Javascript/C++/Java based renderer called Tangram.  It's got it's own style sheet format.  That format is.... very specific to Tangram. 

I've been on the fence about their style format.  Some discussion was in order.

Data Formats & Common Usage

Data formats have crazy parts.  Go look at PDF (better yet, don't).  You have to decide what's common usage and what parts you ignore, at least for a while.

The Mapzen folks were kind enough to host a meeting where we could explore this.  It looks like the common usage among their various style sheets is blunting the weirdness.  They've got professional cartographers using it, in addition to engineers, which is a big plus.

In short, when we see some deep_Tangram_weirdness we can substitute minor_WG-Maply_weirdness and it'll look pretty similar.  Good enough. 

There's Open and Then There's Open

The most attractive thing about the Mapzen Vector Tile Service is the terms of service.  You can use the data in any API and in a variety of ways.  As long as you're not completely rude.

Best of all, you can mix online and offline databases without violating the license.  That makes it perfect to slip into a WhirlyGlobe-Maply app and in our test apps.

That's not a knock on other services.  I get that the API and data are often tightly linked by a license.  That's how you make money.  And speaking of money...

Roadmap and Funding

The vector tile part works just fine in WhirlyGlobe-Maply now.  We still need to implement the Tangram style format.  We're just wrapping up Styled Layer Descriptor support so it shouldn't be all that difficult.

It does require money.

That's the roadmap.  If you're interested in any part of it, speak up.  I get asked about this a lot, so it'll get paid for eventually.

Mapzen Vector Tile Service

Let's take a look at Mapzen's vector tile service.  They've recently announced a developer program and done a lot of work behind the scenes.

Sea levels aren't rising!  Shut up!

Vector Tiles

Mapzen is processing the OpenStreetMap planet file into nicely addressable vector tiles.  They provide the recipes for doing this yourself, but my attitude is better their Amazon account than mine.

Of the formats they offer, WhirlyGlobe-Maply supports two:  GeoJSON and PBF (aka Mapbox Vector Tiles).  I prefer the latter.

PBF (or MVT) contains compressed vector data with general attribution.  To make something visual, you have to match the attributes to a style.  In the past I've used Mapnik XML for this, but this time I've gone with something different.

Mapbox GL Style Sheet

Mapbox has been busily developing (yet another) styling format for their vector tiles.  The new format is nicely tuned to display rather than offline rendering.

The main disadvantage with the new format is there's no way to generate the damn things; you just have to edit the JSON directly.  I imagine we're all hoping they'll update Mapbox Studio one of these days.

OSM loves footpaths

But I like the format and it's clearly where this part of the open source community is going.  So I added a parser to WhirlyGlobe-Maply.

Mapzen Data + Mapbox Style + WhirlyGlobe-Maply

We've got the vector data from Mapzen, a style sheet in the Mapbox format and a big real time rendering toolkit.  How hard is it to put it all together?

Not hard at all.  There's a standard tiling loader (not shown) that will read it on the fly (in multiple threads).  We just have to set up the MapzenSource which involves..

• Reading the MapzenGLStyle file.  It's in the local bundle.
• Setting up the cache directory, so we're not always hitting Mapzen.
• Specifying the base URL and API key for forming the Mapzen URL.
• Picking the layers we want.  All of them in this case, but we could be selective.
• Using PBF (MVT) for the source and Mapbox GL for the style.

Yup, it really is that simple.  I've been messing around with vector tiles for years now.  I've got it down.

This code is in the WhirlyGlobeComponentTester app in the develop branch on github.  I suggest using it in 2D mode.  Things get a little weird on the globe at lower resolutions.

Performance & Next Steps

It's pretty good on the device.  You could use this in a shipping product.  The style is a little bland, but it's fine for a demo.  Rendering performance is decent, but could be optimized.

Needs more labels

Mapzen's set things up so you can fetch individual layers.  I think it would be interesting to use, say the symbol and labels on top of one of NASA's live data sets.  Somebody go do that and let me know how it looks.