Thursday, April 12, 2012

MapBox Tiles Paging

The whole point of supporting MapBox Tiles is to load them selectively in to memory as needed.  That is, to page them.  And now we can.  The video explains it best.

That's the Blue Marble January data set and it contains levels 0 through 8.  Level 0 covers almost the entire globe except for the poles.  Children subdivide by 4 each time, so you can do the math.  Or not, but I'm feeling lazy.

New Data Layer

I've added a new data layer to the WhirlyGlobe library, WhirlyGlobeMBTileLayer.  It acts like a regular SphericalEarthLayer with a few differences.

  • It takes a WhirlyGlobeMBTiles object on startup.  It'll check the metadata and then page textures from that data set.
  • maxTiles controls how many individual tiles are loaded in at once.
  • minTileArea controls how deep its willing to page for a zoom level.  This is screen resolution dependent, which is good.
  • viewUpdatePeriod tells the system how often to update based on view changes.
Kick it off and it'll go do its thing within the layer thread.  It serializes tile updates to avoid overwhelming the rendering system and generally tries to keep a low profile.

QuadTree Based Tile Paging

A tile data set organized in a quad tree is great, but if the data is in a different coordinate system from the display, things get interesting.  Here, we're projecting that quad tree, in Spherical Mercator, on to a sphere.  Stuff gets warped.

There's two ways of dealing with that:  You either don't, or you do.  I chose the latter.  It's easier to just show.

WhirlyGlobe Toolkit Changes

In addition to the new data layer and MapBox Tiles objects, I added support for view based updates.

WhirlyGlobeView updates can be overwhelming; there are modes where the matrix is continuously changing, so you might be called at each and every frame.  That's problematic if you're doing any sort of serious calculation for your updates.  To help out, I broke the view update into two pieces.

  • A watcher delegate to capture every view update
  • A layer view watcher that filters the updates based on frequency.
A data layer plugs into this is by registering with the layer view watcher for a maximum update frequency.  The layer is then called no more often than that frequency.  And yes, I take care of the dangling update condition.

This will likely change a bit in WhirlyGlobe 2.0.  Consult me if you want to use this mechanism directly.

How's It Working?

It's working pretty well.  You could slip this in to a few existing WhirlyGlobe apps without much trouble.  There are a few caveats, though.
  • This only works with static local databases.  Paging over the network will come eventually, but it'll be more work.
  • At present the layer really, really wants a full database between the levels you specify.  No missing tiles.
  • Edges aren't correct between neighboring tiles of differing resolutions.  If you pay attention, you can see lines.  It's a classic problem, I know how to fix it.  Just takes time.
  • There's at least one quad tree bug out there.  You might not notice it, but I do.
  • Tiles should really fade in and out rather than pop.
  • Textures need to be downloaded in the background to avoid overwhelming the rendering engine.
Most of these you and your users may never notice.  If you need the paging functionality, grab a copy of the trunk, after I've checked in, and give it a try.  If you notice the problems, let's talk.

Future Work

I'll be doing some minor cleanup and documentation over the next day and then checking this all in.  That will be the last big thing I do for WhirlyGlobe 1.3.  At some later date, hopefully after someone's used it, I'll bundle it all up as the WhirlyGlobe 1.3 distribution in its own framework.

I've got clients (that is, paying customers) who want more complicated things.  Those I'm doing in WhirlyGlobe 2.0.  Some of that is paging related, much of it isn't.  It should all be neat and, yes, I'll be releasing it open source.

If you've got an app coming up in the next few months, use the trunk and/or WhirlyGlobe 1.3 and let me know what you're up to.


It's all checked in to the trunk.  Enjoy.