Elevation / Synthetic Vision

I’ve never been happy with elevation support in the toolkit.  With the move to 3.0 and advances in mobile hardware, we can do much better.

But before I dive in, a bit about terminology.  Aviation developers tend to refer to a simulated display of where a pilot is and where they're going as Synthetic Vision.  The rest of you might just think of it as elevation support in the toolkit.  It's the same thing.

Mt. Rainier from Mapbox Satellite with elevation

I’m going to try to put together a small group of WhirlyGlobe-Maply users to sponsor a new development effort.  Let’s look at what that would be.

How Loading Used to Work

Level 0 loaded

Elevation loading was something of a hack in the toolkit.  To understand why, we have to look at the way we loaded a tile’s worth of data:

  Can we see enough of this tile to load it?

  Request the imagery & wait for it to return

  Request the elevation & wait for it to return

  Build the geometry

  Slap them all together

The biggest problem was loading time.  The visuals would get super chunky as users waited for things to load.  And it was all serial.  Wait for imagery, then wait for elevation.  Not great.

How Loading Works Now

With the move to the Sampler/Loader architecture in 3.0 things got a lot better.  The tile loading process now looks like this.
  Can we see enough of this tile to load it?
  Rebuild the geometry immediately.
  Slap whatever we currently have on the tile
  Request the image

As new images come in, we update the connection between geometry and textures so things get sharper.  Best of all, the user sees the geometry immediately.  It never gets blocky.  The figures above are level 0 for the Stamen Watercolor tiles (right) even though we’re at level 4 or so for the geometry (left).

Saildrone Forecast uses a hybrid vector map on a globe

We can also do cool stuff like load a whole stack of data sources and merge them together to produce an image and maybe some vector data and the system just manages it all.  And it’s smart enough to request everything it needs at once, rather than feeling its way down from the top.  It’s really nice.

Elevation could be a simple part of that, but I had something much bigger in mind.

How Elevation Should Work

Now we’ve got a pool of imagery tiles controlled by what the system thinks is visible.  It can be laggy, imagery can be missing, we can tolerate sources with varying levels.  It’s much more flexible.  This suggests a way forward for elevation.

But let’s detour briefly into how we draw stuff.  This is what the geometry goes through.

Super complex diagram of the render pipeline

The geometry tiles point at the best imagery tiles (or sub-sections) for their use.    Imagery becomes textures in OpenGL (or Metal) and these are applied in the fragment shader stage.  It’s all well greased for just this use case.

On the left, you can see the tiles that are loaded, including level 0 and level 5(ish) for the given (white) viewing area.  On the right, that’s what it looks like.  To capture this picture, you have to take a video of the system while you zoom out very quickly.  It will catch up and delete the high res tiles very quickly.

The idea here is that geometry tiles will point at the best imagery they can find currently loaded.

For elevation, we actually need to do the work before the fragment shader.  Each piece of geometry needs to get its spatial location before it can move on to the next stage.  We used to do this on the CPU, but now we could do it in a vertex shader, on the GPU.

It’s good to do things on the GPU.  Mostly.  In general.  But specifically, in this case.  If we pick up the elevation right at the beginning of rendering, we can do a lot of interesting stuff with it.

Geometry Tile Points to Elevation and Imagery Pools

Each geometry tile would point to the best elevation data available to it.  That might be the right tile or it might be a subset of a lower resolution tile.  Just like with imagery.

Now we’d have two pools of data, one for imagery and one for elevation.  They’d be loaded independently and they could be different resolutions, loaded at different rates.

Having the elevation living in a separate pool of data opens up another possibility:  We can mess with it.

Elevation Overlays

One big problem with synthetic vision and elevation is airports.  Runways are flat(ish), but the elevation data may not be, particularly at the resolutions available.  What we need is a way to flatten the elevation.

Elevation + Runway = Flattened Elevation

I would propose adding an overlay at a high resolution.  We render the airport outlines into that overlay with a bit of a buffer and ask the geometry vertex shader to do the heavy lifting.  And yes, runways slope, but I’m trying to keep the explanation simple.

The geometry vertex shader would read from the regular elevation and the overlay.  No overlay, it just does the regular thing.  With the overlay, it has a couple options.  Within the runway itself, calculate a specific elevation.  In the buffer zone, interpolate between the target elevation and the elevation database.  That’ll be easy on the eyes.

As far as data representation goes, this is wonderfully simple.  Elevation data is just a set of grids, loosely corresponding to the imagery.  Airports (or runways) are simple polygons.  All we need is a bit of spatial information in a simple database for lookup.

Best of all, it’s pretty easy to update.  But there are some details.

Point Model Placement

There are a number of objects you want to sit on the terrain.  These are things like towers, poles, maybe even buildings.

Objects sitting on top of terrain

Those will also need to be tied to the elevation database and do their position lookups in a vertex shader.  It’s the same process as geometry tiles for the visuals, but it’s a toolkit change and will require a bit of work.

3D Loading Details

Tiles with St. Helens take up more screen space

Another problem is how we load 3D data tiles versus 2D.

In general, we don’t take the elevation into account when loading tiles.  And because we don’t, you end up loading more than you should just in case.

Taking elevation into account requires a fair bit of feedback between the loading system and the data it’s loading.  The short version is, we need the bottom and top of the tile, but we don’t have it until we’ve loaded it.  Fixable, but a bit of work.

Airport/Runway Generator

The toolkit has a facility for generating geometry on the fly, but it doesn’t know anything about runways.

I’d propose building a rudimentary runway generator based on simple data.  Markers, numbers, direction, that sort of thing.  It’ll likely work well enough for most use cases and provide endless opportunities for developers to improve on.

Elevation Database

I’d leave the actual airport databases up to the developers.  But for the elevation, there’s no reason to reinvent the wheel.

I propose to build a new elevation database for CONUS and possibly Alaska.  Worldwide would be a possibility, but I’m guessing it’s less useful.

I’d use standard data sources, available for free (USGS, mostly).  Resolution would be determined by available data, but probably not better than 10m and possibly as low as 30m.

The resulting database would be MBTiles-like Sqlite and lower resolution versions could be easily harvested.  It’s pretty easy simple a handful of SQL calls.

I’ve done this before, so I’ll also update the open source tools I used in the past and provide instructions for doing it again.

Conclusion

With a good set of imagery this would look fantastic.  Even without the imagery, it’ll be pretty useful.  Since the data is fairly lightly processed, you’d be able to mix and match to your own requirements or what customers are willing to pay for.  It’ll work paging over the network or with everything local to the device.

This is where I want to take elevation in WhirlyGlobe-Maply.  What are your priorities and what requirements do you have?  Drop me a line.  And money.  Money is always good.

Vector Features / Mapbox Style Sheet Roadmap

Mapbox Style Sheet support has been ported to Android!  Look for it in 3.1.

But this post is really about vector features and money.  Let's dive in.

Vector Maps

You know what vector maps are.  Take some points, lines, and polygons and turn them into visual maps.

Let's start with source data which is maybe GeoPackage or ShapeFiles, sometimes Mapbox Vector Tiles or even your own weird format.  Then style it.  Maybe you use SLD or Mapbox's Style Spec.  Some people have their own formats and a lot just do it in code.  That's how you get visuals from data.

This requires some basic vector functionality in the WhirlyGlobe-Maply toolkit.  Stuff like labels, wide lines, big polygon features, texture support.  You know the drill if you're using the toolkit.

I'd like to make that all work better.

Who This is For

Are you using WG-Maply in your aviation app?  Then I know you'd like better vector support.  This is for you.

Are you using the Mapbox Styles in your map or weather app?  Then this is definitely for you.  All of these features will benefit the Mapbox style maps.

With WG-Maply 3.0 released and working, now is the time to add a whole bunch of cross-platform vector features.  What follows is a menu of things that can be added to the toolkit.   For money.

Zoom Level

What zoom level is this is one of the fundamental questions in laying out a flat map.  Alas, our maps are not flat and it’s a much more complicated question than it seems.  Check out this spherical mercator map splatted on the globe.

What zoom level are we at?  Well…. it’s complicated.  That particular map might be a zoom level 6.  But if we head north, without changing our elevation we’re quickly going to be at zoom level 5.  Why?  Because the tiles get much smaller.

But a lot of things depend on a continuous “zoom level”.  A color might trigger at zoom 2.5.  Text size might interpolate between zoom levels 1-5 continuously.

To fix this once and for all I’ll implement a continuous zoom level for a given data source.  It’ll be an object you can pass around for reference and query yourself as needed.  I’ll even smooth out the nonlinearities (with some deeply strange code it’s best not to think about).

With a dependable zoom level calculation we use on specific features a lot of other things get easier.

Wide Vectors

Wide Vectors are just linear features with width and proper junctions.  We have a wide vector implementation, but it’s got problems: It’s old and leans on geometry to do things it could do in the shaders and it’s missing a bunch of features.

Some of the basic attributes should vary by zoom level including width, color, opacity and offset.

Offset is a big one we don't have now.  It would let you do insets and borders around polygons.  I know you aviation users want this one, because you keep asking for it.

All the standard junction and cap types need to be supported.  We just do bevel right now.

https://www.w3.org/TR/SVG/painting.html#StrokeLinejoinProperty

We have some facility for dots and dashes, but those need to be updated and tested properly.  Much of this logic just needs to go into a more intelligent shader.

Polygons

Big polygonal features mostly work just fine.  All that's really missing is control of color, opacity and enabling based on zoom level.

Fill patterns with textures could use some work as well.

Text

Text support gets tricky, particular with two platforms.  It does work, but we need even more.  Ignoring layout for the moment there are other features we need.

As with lines and polygons, we should be able to vary color, opacity and size continuously on zoom level.

Right now outlines are kind of blurry.  Ignoring the low level details of why, this is fixable.

And here's a specific Mapbox Style Spec feature:  Support for their wacky font packs.  I'm not going to switch entirely over to those, but for one of their maps it would be nice to just use them.

Layout

Here's the really big feature for layout.  Making labels follow lines.  That one would be huge.  It'll also be a lot of work.  Again, looking at the aviation users here.  I know you want this.

But general Mapbox Style Sheet users would like this too.

Screen Objects

We have markers in the toolkit and we've got text.  So if you want to make a highway shield or something you can just stick one on top of the other, right?

Yeah, kinda.  It'll work until you see those features intersecting each other and then it looks weird.  This happens a lot with aviation symbols.

One solution is to make custom markers for everything, which people do, but it tends to get messy and complex to adjudicate.

Screen Objects fix that problem.  You just tell the system you'd like a highway symbol overlaid with a bit of text and it'll make that happen for you.  And, oh yeah, that'll work will with part of the Mapbox Style Spec.

Mapbox Specific Features

Up to this point we're looking at general purpose WG-Maply features.  Things you can use without committing to their style spec.  But there a few specifics that'd be great if you are using their style sheets.

Sprite Sheets

There's some support for this in the toolkit, specifically on iOS.  It just needs to be hooked up and tested properly.

Mapbox Font Madness

I don't love how they implemented fonts.  It seems a bit extravagant to me.  But if you want it and you're willing to pay for it, then sure.  It would simplify loading the standard Maptiler styles quite a lot.

Layout Specifics

How they do layout and how I do layout are pretty different.  There are lots of niggling little specifics in their spec that I'm likely to just ignore.  If you want to get closer to pixel perfection with their maps, I'm game.  You know.  For money.

Hillshades, Heatmaps, Fake 3D & Rasters

If you're using some of this crazy stuff in the style spec, you're probably just using MapboxGL (whatever it's called now).  I have other ways of doing it which are more code based.  But if you want to, then sure.  Again.  For money.

Conclusion

Odds are you're reading this because I sent you here with the request "I'd like some money to add several of these features I know you want".  So, you know, let's do that.  There's efficiency in scale and the more time I spend on vector maps, the better they will get.

For everyone else, this is where I'd like to take the vector map support in WG-Maply.  Your feedback is welcome too.