Thursday, April 12, 2012

MapBox Tiles Paging

The whole point of supporting MapBox Tiles is to load them selectively in to memory as needed.  That is, to page them.  And now we can.  The video explains it best.

That's the Blue Marble January data set and it contains levels 0 through 8.  Level 0 covers almost the entire globe except for the poles.  Children subdivide by 4 each time, so you can do the math.  Or not, but I'm feeling lazy.

New Data Layer

I've added a new data layer to the WhirlyGlobe library, WhirlyGlobeMBTileLayer.  It acts like a regular SphericalEarthLayer with a few differences.

  • It takes a WhirlyGlobeMBTiles object on startup.  It'll check the metadata and then page textures from that data set.
  • maxTiles controls how many individual tiles are loaded in at once.
  • minTileArea controls how deep its willing to page for a zoom level.  This is screen resolution dependent, which is good.
  • viewUpdatePeriod tells the system how often to update based on view changes.
Kick it off and it'll go do its thing within the layer thread.  It serializes tile updates to avoid overwhelming the rendering system and generally tries to keep a low profile.

QuadTree Based Tile Paging

A tile data set organized in a quad tree is great, but if the data is in a different coordinate system from the display, things get interesting.  Here, we're projecting that quad tree, in Spherical Mercator, on to a sphere.  Stuff gets warped.

There's two ways of dealing with that:  You either don't, or you do.  I chose the latter.  It's easier to just show.

WhirlyGlobe Toolkit Changes

In addition to the new data layer and MapBox Tiles objects, I added support for view based updates.

WhirlyGlobeView updates can be overwhelming; there are modes where the matrix is continuously changing, so you might be called at each and every frame.  That's problematic if you're doing any sort of serious calculation for your updates.  To help out, I broke the view update into two pieces.

  • A watcher delegate to capture every view update
  • A layer view watcher that filters the updates based on frequency.
A data layer plugs into this is by registering with the layer view watcher for a maximum update frequency.  The layer is then called no more often than that frequency.  And yes, I take care of the dangling update condition.

This will likely change a bit in WhirlyGlobe 2.0.  Consult me if you want to use this mechanism directly.

How's It Working?

It's working pretty well.  You could slip this in to a few existing WhirlyGlobe apps without much trouble.  There are a few caveats, though.
  • This only works with static local databases.  Paging over the network will come eventually, but it'll be more work.
  • At present the layer really, really wants a full database between the levels you specify.  No missing tiles.
  • Edges aren't correct between neighboring tiles of differing resolutions.  If you pay attention, you can see lines.  It's a classic problem, I know how to fix it.  Just takes time.
  • There's at least one quad tree bug out there.  You might not notice it, but I do.
  • Tiles should really fade in and out rather than pop.
  • Textures need to be downloaded in the background to avoid overwhelming the rendering engine.
Most of these you and your users may never notice.  If you need the paging functionality, grab a copy of the trunk, after I've checked in, and give it a try.  If you notice the problems, let's talk.

Future Work

I'll be doing some minor cleanup and documentation over the next day and then checking this all in.  That will be the last big thing I do for WhirlyGlobe 1.3.  At some later date, hopefully after someone's used it, I'll bundle it all up as the WhirlyGlobe 1.3 distribution in its own framework.

I've got clients (that is, paying customers) who want more complicated things.  Those I'm doing in WhirlyGlobe 2.0.  Some of that is paging related, much of it isn't.  It should all be neat and, yes, I'll be releasing it open source.

If you've got an app coming up in the next few months, use the trunk and/or WhirlyGlobe 1.3 and let me know what you're up to.


It's all checked in to the trunk.  Enjoy.


  1. I've just built the trunk from Google Code. The WhirlyGlobeMBTiles looks very, very nice. At first I thought it would be a way to stack additional textures onto the SphericalEarthLayer, but rather it's a replacement, right?

    What I'm looking for, is a way to stack tiles (containing ozone data) onto the earth layer. Is that possible? My tiles can be semi-transparent.

    The data, projected onto a 2D map (might not work in Safari, does work in Firefox):

    So, is there a class I can use for this?

  2. Indeed, that is a replacement for the the spherical earth layer.

    If you're using the google code trunk then you're working with WhirlyGlobe 1.2+. I haven't tried to stack image data layers with that version, but it might be possible. In theory, you just add the two layers in order and give the second one a larger drawOffset. Honestly though, I forget if I added draw offset in that version or if it's only in 2.0.

    Now in WhirlyGlobe 2.0 the possibilities for quad paging data layers are much broader. You can have an MBTiles layer with one or more other layers overlaid on top. These layers could just be image tiles that you hand back as a delegate. You can control the rendering order with drawOffset or turn off z buffering entirely and just use drawPriority. The latter will get you a cleaner look and work well in more map-like cases.

    If you're skilled enough to compile WhirlyGlobe from the trunk, 2.0 wouldn't present much of a problem:

    On the other hand, it might be overkill.