Data Paging Layers

We load a lot of data sources in WhirlyGlobe-Maply, usually over the network.  Images are relatively easy, but how do you deal with things like vector tiles?

Ye olde vector tile example

That would be the job of the Paging Layer.

Quadtree Structured Data Paging

It all starts with a quadtree, a simple enough data structure.  It's here that we decide what's visible and what needs to be loaded.

Quadtree courtesy wikipedia

The quadtree for a data source can be in any proj.4 supported coordinate system, but most people use Spherical Mercator which is... what they use.  No judgement.

In deciding what to load, we start at the top node (0: 0,0), projecting a box into the view and then recursing if the box is visible.  It's not all that expensive, because quadtree.  But does get a bit tricky when using the globe or multiple coordinate systems.

For image layers, the toolkit has its own complex logic for loading image tiles and frames and handling failure and so on.  We provide similar logic for vector tiles, but for anything else, the logic is you.

Quad Paging Layer

You've got a data source, sqlite file full of points, let's say.  You want to turn them into markers, but there's too many to load at once.

Here's where the QuadPagingLayer (Android) or MaplyQuadPagingLayer (iOS) comes in.  Set up one of these babies with the coordinate system and extents of your data source and it'll figure out what to load.

You have to create the visible objects and that's where the PagingInterface (Android) and MaplyPagingDelegate (iOS) come in.  The paging layer calls you back when it's time to load a tile's worth of data.

Mt. St. Helens LIDAR

We love our threading in WhirlyGlobe-Maply and it was ever so hard to implement.  So use it.  Try a dispatch queue on iOS or use a worker thread on Android.  But don't hold up the paging layer.  It has other things to do.

When you make a visible object (marker, label, etc) tell the paging layer what you created with the addData (iOS, Android) method and call tileDidLoad (Android, iOS) when you're finished.

That's enough information for the paging layer to do the rest, which includes...

Cleanup on Node (12: 129,1304)

One of the trickier bits here is deciding when to delete something.  Luckily, you don't have to.

You told the paging layer what you created and it'll delete it for you.  There's even a callback if you really, really want to know it did that.

Paging Layers are Awesome

There it is, one of the key pieces of WhirlyGlobe-Maply: the paging layer.  We use it all over the place and if we don't support your data type, you'll use it too.

Map Projections

Let's talk map features.  How about custom projections?

Does this projection make my poles look big?

Spherical Mercator (web mercator) is the most common map projection you'll see.  It's a fairly simple unraveling of the earth, but its main purpose is to annoy cartographers.  Did you know the EPSG code for Web Mercator stands for "DIAF".  True fact.  Look it up.

Our users mostly stick with web mercator, but we now let you specify your own custom projection.

Proj.4 CoordinateSystem

Like all the cool kids we use Proj.4 to represent coordinate systems.  In theory this allowed us to use a wide variety of systems.  In practice, we just used Spherical Mercator, Plate Carree and a very weak geocentric.

Recently we exposed Proj.4 directly with a MaplyProj4CoordSystem.  It's simple enough, if you're familiar with Proj.4.  Here's an example for British National Grid.

Full disclosure, there's actually a datum grid shift that goes with that for proper BNG.  We've shortened it for the example.

You can get these Proj.4 strings from spatialreference.org, among other places.  But odds are if you have the need, you already have the string.

British National Grid

The folks at Ordnance Survey were kind enough to help us sort this out.  Let's look specifically at the British National Grid, starting with a data source in BNG.

BNG tile source on spherical mercator

That's a 2D map with a base layer in web mercator (Stamen Watercolor) overlaid with a dummy tile source in BNG.  As far as these things go, that's the easy case.  Let's look at a more difficult one where the map itself in BNG.

BNG tile source on BNG map

In this one the map is in BNG with a base layer in web mercator and an overlaid dummy tile source in BNG.

That's pretty cool!  Web mercator stretches far outside the boundaries of the BNG system, so we convert the tiles to BNG and toss out any that don't overlap.  And it works!  But there's more.

BNG tile source on globe

Okay, not all that difficult as these things go but it looks cool.  We're just overlaying a dummy BNG source on the globe.

You can find all of these examples in the AutoTester app for iOS.  And there's more...

Android

This is something I hope to say a lot more in the future:  Custom map projections are also available on Android in the develop_3_0 branch!

A triptych of British National Grid

You can find these examples in the AutoTester app for Android.  I'll explain AutoTester in more detail soon, but it's already been a great help in development.

Next Up

The hard part here was tracking down the weird corner cases and systems which just don't play well with each other.  There are likely to be a few more like that, so let me know what you find.

We've got more map features in the queue, including some vector data display.  Stay tuned!