New Post: Version 3.2

 Psst!  Hey you!  Yes, you!  The mousebird consulting blog moved.  And there's a new version of the toolkit.

New Blog: Location Tracking For Android

 We've moved the mousebird consulting blog.  Check out this post on Location Tracking for Android.

An Offer to the Mobile Mapping Community

The short version:  We're offering to fix a key problem with the open source version of the Mapbox mobile display toolkit.  But it'll cost money.

To emphasize, I'm talking about the native mobile display SDK here, not the web.

The Situation

A lot of discussion is happening around Mapbox closing their Javascript API.  For good reason.  It's the most flexible geospatial display API on the web that focuses on performance.

Less noise has been made about Mapbox doing something similar to their mobile APIs back in July.  Same problem, different platform.

Mapbox Did What Now?

It's a little convoluted, but it goes like this.  They changed the license on the mobile APIs to make it more restrictive, sure, but they also did something more worrying.  They scooped out the rendering engine for Android and iOS and placed it in a binary blob.  So it's no longer open source, or even visible.

Obviously that makes the SDK no longer open.  In a big, big way.  So you can just use the old version, right?  Well, there's a wrinkle.

Apple, OpenGL ES and Metal

The old Mapbox mobile SDKs used OpenGL ES for rendering.  That's fine on Android, but Apple has deprecated OpenGL in favor of its own rendering toolkit, Metal.  The how's and why's and wailing and such is for another day.  They can do it.  They did it.  We live with it.

So that's a big problem using the old (open) version of the Mapbox mobile display SDKs.  You're going to fall off the cliff at some point without a Metal renderer.

So Why Not Switch to WhirlyGlobe-Maply

Honestly, we'd prefer you just took a look at our toolkit.  It has a good open source lineage and we didn't close it when we added Metal support.

But yeah, we know people will want to stick with MapboxGL, even if they can't have the new version.  So what if you could drop in a new Metal renderer?

Who Do We Know With a Metal Renderer?

Well, we have a Metal renderer in WhirlyGlobe-Maply.  It's sitting there in our toolkit purring away.  Tested in shipping software, uses indirect rendering, the more GPU friendly complicated option, and already rendering Mapbox Style Sheets.

So You Just Drop It In Then?

Ever tried to drop a new engine in your car?  Yeah, it's a bit more complex, but it can be done.

Our low level renderer is meant for internal use.  We interact with it through a set of Managers which turn things like roads into polygons and shader controls.  The renderer itself has modules for OpenGL and Metal.

But the good news is that low level piece (WhirlyKit, we call it) is separable.  It's one nice C++ blob that could be copied into another project.

Lawyers, Drugs & Money

I've never looked inside MapboxGL.  I don't know how they implement their shaders, I don't know if they have a true rendering engine or some tile based thing.  I've never looked and I wasn't planning on it.

No disrespect to their work, I just manage our Intellectual Property carefully.  Our solutions will look different from theirs, have a provable audit trail in git and be more likely to stand up in court.

This project would pierce that veil and I'd want to have a good attorney lay out the ground rules.  Good attorneys aren't cheap.

And as for the engineering, this isn't a trivial task and it would require money.  We already give away our toolkit for free and maintaining it uses up our "free" time.  Someone needs to pay for this.

So Maybe Someone Will Just Do This For Free?

Maybe.  It's a good learning experience.  But there's a reason Mapbox buried its rendering engine.  Doing it is hard.  Doing it well is much, much harder.

Not many geospatial developers understand real time rendering.  If you want to use game developers, you need to teach them geospatial.  Been down that road.  It's fine, but it's a long road.

Summary

We know what we're doing here.  Obviously.  We could do this, replacing their OpenGL renderer with our likely faster Metal renderer.  All it takes is money.

I'm not delusional.  A lot of things have to happen quickly.  Any sufficiently large player can find another way to pressure Mapbox and probably already has.  That's just business.

But if you have the cash and the need, we're open to fixing this problem for the community.

Closing of the Mapbox APIs

We make WhirlyGlobe-Maply, a mobile map toolkit that's open source and really fast.  We'd like money to make it better.

The web?  Sure, we'd love to make a Javascript/WebGL version.  Pay us!

And while you're here, do you have any ideas for free marketing?

MapTiler Streets in WhirlyGlobe-Maply

Mapbox & Competition Lost

Let's be honest, Mapbox makes the biggest and best maps SDKs for developers.  Google's map is better, but their SDK is meh.  Apple Maps is pretty good in both ways.  Bing is... also around.  As are lots of others.  But MapboxGL was always our competition.

My vector toolkit predated theirs.  Really!  And I had a few map app developers because of that.  But when they turned on the Softbank money hose, my little toolkit never had a chance.  I lost most of those customers in the ensuing years.  That wasn't fun.

Lucky me, I could see it coming.  I steered the company into weather and aviation.  Unlike a lot of competitors, we survived.

Closed vs. Open Source

Mapbox has changed the license on its mobile and now Javascript APIs to something that is not open source.  Is it free?  I can't really tell.  But it's certainly not open.

I've been expecting this for years.  It's the standard Silicon Valley Venture Capital play.  Make a thing cheap, destroy the competition, enjoy your market.  They executed it well.

So now we're going to get all those customers back, right?  Ha!

Lock In / New Users

The truth is those developers won't ever come back.  The cost of switching is too high and they're off doing other things.  Anyway, if you're using Mapbox services, it's fine.  That strategy totally works.

We have picked up a few open source map extremists in recent years.  They've contributed money to our Vector Tiles and Style Sheet support.  It's getting really good.

So what's next?

We Are Open Source

Let me reiterate that the WhirlyGlobe-Maply toolkit is licensed under Apache 2.0, a very friendly license for doing all sorts of commercial work.

Open source powers our small business and we're committed to it.  It's a contract between us and our community.  One we've honored through several major upgrades.

Javascript / WebGL

And hey, we're open to making a Javascript version of the toolkit.  And not just a cross-compiled version, a real SDK for JS developers.

I've actually proposed this to weather users a few times.  Just the parts they need, overlaid on a web map.  Not the whole thing, but the core rendering is the same.

We'd open source it, because that's who we are.  If you're interested, reach out.  Preferably with money.

Marketing on the Cheap

With less competition, you'd think we'd be set, right?  Not so much.  Mapbox remade the market in its image and we have to live in that world.

But there are opportunities.  Projects that have to switch, projects that are just starting out, competitors who don't want to write their own.  That sort of thing.

So if you have ideas for marketing to those people, I'd love to hear them.  Free ideas, preferably.  Because open source. 

WhirlyGlobe-Maply 3.0 Integration & Housekeeping

There are currently two github repos for WhirlyGlobe-Maply. The main one and the one for version 3.0. We're merging them. 

 Version 3.0 was developed in private for sponsors and then released publicly.  That caused exactly as much trouble as I expected.  We did get Metal support on iOS and a really nice Android version.  Worth it! 

 WhirlyGlobe-Maply Versions

Not everyone wants to upgrade to 3.x right now, so we got you covered on 2.x.

The current 2.x version is 2.6.6.  The tag is v2.6.6 and you can use that directly in Cocoapods or whatever your favorite system may be.  There may be additional v2.6.x versions.  Just look at the tags.

We're going to tag v3.0 on the main repo as well, but you probably want to use 3.1 which we'll be releasing shortly onto master and develop branches.

Going forward, master will be the stable one and use develop if we tell you to (or if you're feeling daring).

Tutorials

I almost forgot.  The tutorials have been updated for 3.0!

Android is new.  Lots of stuff changed on the Android side of things.  iOS has been mostly updated.  If you find problems with either, let us know.

We Can Display Your Mapbox Style Map

When people think of us, most think of WhirlyGlobe.  It's the iconic interface in weather apps like DarkSky and tons of aviation apps you'd only know if you're a pilot.

Saildrone Forecast

But did you know we also make a good 2D slippy map?  That's Maply and it's completely open source.

Our support for Mapbox style maps has gotten really good this year, but we'd like to make it better!

Mapbox Kinda Maps

Vector Tiles and Mapbox Style Sheets are a potent combo and they've won the geospatial format wars.  If you want to make your own vector slippy map, that's likely how you're doing it.

Maply has supported Vector Tiles for years, but the Mapbox Style Sheet support lagged.... until this year.

MapTiler Streets

With the recent move to a unified code base we've got one single Mapbox Style Sheet implementation for iOS and Android.... and wherever else we port to.  On iOS we upgraded our shaders to react to zoom level changes and it looks fantastic.

But there's a lot more to do.  The spec if vast, and the fiddly bits are so very fiddly.  We need money for more.

Who Are You Again?

mousebird consulting is a little company that makes WhirlyGlobe-Maply.  We specialize in weather, aviation, and GIS mobile app development based on the toolkit.

We've been around with this thing for nearly a decade.  It actually predates most of the competition... and it really is open source.

How Much Money We Talking?

Mapbox Style Sheet support is vast and will absorb a lot of development money.  Two platforms with two graphics APIs (Metal and OpenGL ES) make it more complex.  But architecturally, things are good and money will translate directly to features.

As for amounts, $150k means I can hire someone just for this.  $30k means we spend a couple of sprints on new features.  Obviously more is better, but even $10k would move things along.

I Dunno, We're Kind of Invested In...

Oh, I know you are.  Maybe you'll consider switching someday, but that's not what this is about.  

Some very large companies are committed to this technology.  Toss us a little cash and we'll be there as your backup plan.

And for the rest of you, our toolkit has gotten really good and we've outlived most of our competitors.  We're stable and we're not going anywhere.  If you're committed to open source, check us out.

Elevation / Synthetic Vision

I’ve never been happy with elevation support in the toolkit.  With the move to 3.0 and advances in mobile hardware, we can do much better.

But before I dive in, a bit about terminology.  Aviation developers tend to refer to a simulated display of where a pilot is and where they're going as Synthetic Vision.  The rest of you might just think of it as elevation support in the toolkit.  It's the same thing.

Mt. Rainier from Mapbox Satellite with elevation

I’m going to try to put together a small group of WhirlyGlobe-Maply users to sponsor a new development effort.  Let’s look at what that would be.

How Loading Used to Work

Level 0 loaded

Elevation loading was something of a hack in the toolkit.  To understand why, we have to look at the way we loaded a tile’s worth of data:

  Can we see enough of this tile to load it?

  Request the imagery & wait for it to return

  Request the elevation & wait for it to return

  Build the geometry

  Slap them all together

The biggest problem was loading time.  The visuals would get super chunky as users waited for things to load.  And it was all serial.  Wait for imagery, then wait for elevation.  Not great.

How Loading Works Now

With the move to the Sampler/Loader architecture in 3.0 things got a lot better.  The tile loading process now looks like this.
  Can we see enough of this tile to load it?
  Rebuild the geometry immediately.
  Slap whatever we currently have on the tile
  Request the image

As new images come in, we update the connection between geometry and textures so things get sharper.  Best of all, the user sees the geometry immediately.  It never gets blocky.  The figures above are level 0 for the Stamen Watercolor tiles (right) even though we’re at level 4 or so for the geometry (left).

Saildrone Forecast uses a hybrid vector map on a globe

We can also do cool stuff like load a whole stack of data sources and merge them together to produce an image and maybe some vector data and the system just manages it all.  And it’s smart enough to request everything it needs at once, rather than feeling its way down from the top.  It’s really nice.

Elevation could be a simple part of that, but I had something much bigger in mind.

How Elevation Should Work

Now we’ve got a pool of imagery tiles controlled by what the system thinks is visible.  It can be laggy, imagery can be missing, we can tolerate sources with varying levels.  It’s much more flexible.  This suggests a way forward for elevation.

But let’s detour briefly into how we draw stuff.  This is what the geometry goes through.

Super complex diagram of the render pipeline

The geometry tiles point at the best imagery tiles (or sub-sections) for their use.    Imagery becomes textures in OpenGL (or Metal) and these are applied in the fragment shader stage.  It’s all well greased for just this use case.

On the left, you can see the tiles that are loaded, including level 0 and level 5(ish) for the given (white) viewing area.  On the right, that’s what it looks like.  To capture this picture, you have to take a video of the system while you zoom out very quickly.  It will catch up and delete the high res tiles very quickly.

The idea here is that geometry tiles will point at the best imagery they can find currently loaded.

For elevation, we actually need to do the work before the fragment shader.  Each piece of geometry needs to get its spatial location before it can move on to the next stage.  We used to do this on the CPU, but now we could do it in a vertex shader, on the GPU.

It’s good to do things on the GPU.  Mostly.  In general.  But specifically, in this case.  If we pick up the elevation right at the beginning of rendering, we can do a lot of interesting stuff with it.

Geometry Tile Points to Elevation and Imagery Pools

Each geometry tile would point to the best elevation data available to it.  That might be the right tile or it might be a subset of a lower resolution tile.  Just like with imagery.

Now we’d have two pools of data, one for imagery and one for elevation.  They’d be loaded independently and they could be different resolutions, loaded at different rates.

Having the elevation living in a separate pool of data opens up another possibility:  We can mess with it.

Elevation Overlays

One big problem with synthetic vision and elevation is airports.  Runways are flat(ish), but the elevation data may not be, particularly at the resolutions available.  What we need is a way to flatten the elevation.

Elevation + Runway = Flattened Elevation

I would propose adding an overlay at a high resolution.  We render the airport outlines into that overlay with a bit of a buffer and ask the geometry vertex shader to do the heavy lifting.  And yes, runways slope, but I’m trying to keep the explanation simple.

The geometry vertex shader would read from the regular elevation and the overlay.  No overlay, it just does the regular thing.  With the overlay, it has a couple options.  Within the runway itself, calculate a specific elevation.  In the buffer zone, interpolate between the target elevation and the elevation database.  That’ll be easy on the eyes.

As far as data representation goes, this is wonderfully simple.  Elevation data is just a set of grids, loosely corresponding to the imagery.  Airports (or runways) are simple polygons.  All we need is a bit of spatial information in a simple database for lookup.

Best of all, it’s pretty easy to update.  But there are some details.

Point Model Placement

There are a number of objects you want to sit on the terrain.  These are things like towers, poles, maybe even buildings.

Objects sitting on top of terrain

Those will also need to be tied to the elevation database and do their position lookups in a vertex shader.  It’s the same process as geometry tiles for the visuals, but it’s a toolkit change and will require a bit of work.

3D Loading Details

Tiles with St. Helens take up more screen space

Another problem is how we load 3D data tiles versus 2D.

In general, we don’t take the elevation into account when loading tiles.  And because we don’t, you end up loading more than you should just in case.

Taking elevation into account requires a fair bit of feedback between the loading system and the data it’s loading.  The short version is, we need the bottom and top of the tile, but we don’t have it until we’ve loaded it.  Fixable, but a bit of work.

Airport/Runway Generator

The toolkit has a facility for generating geometry on the fly, but it doesn’t know anything about runways.

I’d propose building a rudimentary runway generator based on simple data.  Markers, numbers, direction, that sort of thing.  It’ll likely work well enough for most use cases and provide endless opportunities for developers to improve on.

Elevation Database

I’d leave the actual airport databases up to the developers.  But for the elevation, there’s no reason to reinvent the wheel.

I propose to build a new elevation database for CONUS and possibly Alaska.  Worldwide would be a possibility, but I’m guessing it’s less useful.

I’d use standard data sources, available for free (USGS, mostly).  Resolution would be determined by available data, but probably not better than 10m and possibly as low as 30m.

The resulting database would be MBTiles-like Sqlite and lower resolution versions could be easily harvested.  It’s pretty easy simple a handful of SQL calls.

I’ve done this before, so I’ll also update the open source tools I used in the past and provide instructions for doing it again.

Conclusion

With a good set of imagery this would look fantastic.  Even without the imagery, it’ll be pretty useful.  Since the data is fairly lightly processed, you’d be able to mix and match to your own requirements or what customers are willing to pay for.  It’ll work paging over the network or with everything local to the device.

This is where I want to take elevation in WhirlyGlobe-Maply.  What are your priorities and what requirements do you have?  Drop me a line.  And money.  Money is always good.

Vector Features / Mapbox Style Sheet Roadmap

Mapbox Style Sheet support has been ported to Android!  Look for it in 3.1.

But this post is really about vector features and money.  Let's dive in.

Vector Maps

You know what vector maps are.  Take some points, lines, and polygons and turn them into visual maps.

Let's start with source data which is maybe GeoPackage or ShapeFiles, sometimes Mapbox Vector Tiles or even your own weird format.  Then style it.  Maybe you use SLD or Mapbox's Style Spec.  Some people have their own formats and a lot just do it in code.  That's how you get visuals from data.

This requires some basic vector functionality in the WhirlyGlobe-Maply toolkit.  Stuff like labels, wide lines, big polygon features, texture support.  You know the drill if you're using the toolkit.

I'd like to make that all work better.

Who This is For

Are you using WG-Maply in your aviation app?  Then I know you'd like better vector support.  This is for you.

Are you using the Mapbox Styles in your map or weather app?  Then this is definitely for you.  All of these features will benefit the Mapbox style maps.

With WG-Maply 3.0 released and working, now is the time to add a whole bunch of cross-platform vector features.  What follows is a menu of things that can be added to the toolkit.   For money.

Zoom Level

What zoom level is this is one of the fundamental questions in laying out a flat map.  Alas, our maps are not flat and it’s a much more complicated question than it seems.  Check out this spherical mercator map splatted on the globe.

What zoom level are we at?  Well…. it’s complicated.  That particular map might be a zoom level 6.  But if we head north, without changing our elevation we’re quickly going to be at zoom level 5.  Why?  Because the tiles get much smaller.

But a lot of things depend on a continuous “zoom level”.  A color might trigger at zoom 2.5.  Text size might interpolate between zoom levels 1-5 continuously.

To fix this once and for all I’ll implement a continuous zoom level for a given data source.  It’ll be an object you can pass around for reference and query yourself as needed.  I’ll even smooth out the nonlinearities (with some deeply strange code it’s best not to think about).

With a dependable zoom level calculation we use on specific features a lot of other things get easier.

Wide Vectors

Wide Vectors are just linear features with width and proper junctions.  We have a wide vector implementation, but it’s got problems: It’s old and leans on geometry to do things it could do in the shaders and it’s missing a bunch of features.

Some of the basic attributes should vary by zoom level including width, color, opacity and offset.

Offset is a big one we don't have now.  It would let you do insets and borders around polygons.  I know you aviation users want this one, because you keep asking for it.

All the standard junction and cap types need to be supported.  We just do bevel right now.

https://www.w3.org/TR/SVG/painting.html#StrokeLinejoinProperty

We have some facility for dots and dashes, but those need to be updated and tested properly.  Much of this logic just needs to go into a more intelligent shader.

Polygons

Big polygonal features mostly work just fine.  All that's really missing is control of color, opacity and enabling based on zoom level.

Fill patterns with textures could use some work as well.

Text

Text support gets tricky, particular with two platforms.  It does work, but we need even more.  Ignoring layout for the moment there are other features we need.

As with lines and polygons, we should be able to vary color, opacity and size continuously on zoom level.

Right now outlines are kind of blurry.  Ignoring the low level details of why, this is fixable.

And here's a specific Mapbox Style Spec feature:  Support for their wacky font packs.  I'm not going to switch entirely over to those, but for one of their maps it would be nice to just use them.

Layout

Here's the really big feature for layout.  Making labels follow lines.  That one would be huge.  It'll also be a lot of work.  Again, looking at the aviation users here.  I know you want this.

But general Mapbox Style Sheet users would like this too.

Screen Objects

We have markers in the toolkit and we've got text.  So if you want to make a highway shield or something you can just stick one on top of the other, right?

Yeah, kinda.  It'll work until you see those features intersecting each other and then it looks weird.  This happens a lot with aviation symbols.

One solution is to make custom markers for everything, which people do, but it tends to get messy and complex to adjudicate.

Screen Objects fix that problem.  You just tell the system you'd like a highway symbol overlaid with a bit of text and it'll make that happen for you.  And, oh yeah, that'll work will with part of the Mapbox Style Spec.

Mapbox Specific Features

Up to this point we're looking at general purpose WG-Maply features.  Things you can use without committing to their style spec.  But there a few specifics that'd be great if you are using their style sheets.

Sprite Sheets

There's some support for this in the toolkit, specifically on iOS.  It just needs to be hooked up and tested properly.

Mapbox Font Madness

I don't love how they implemented fonts.  It seems a bit extravagant to me.  But if you want it and you're willing to pay for it, then sure.  It would simplify loading the standard Maptiler styles quite a lot.

Layout Specifics

How they do layout and how I do layout are pretty different.  There are lots of niggling little specifics in their spec that I'm likely to just ignore.  If you want to get closer to pixel perfection with their maps, I'm game.  You know.  For money.

Hillshades, Heatmaps, Fake 3D & Rasters

If you're using some of this crazy stuff in the style spec, you're probably just using MapboxGL (whatever it's called now).  I have other ways of doing it which are more code based.  But if you want to, then sure.  Again.  For money.

Conclusion

Odds are you're reading this because I sent you here with the request "I'd like some money to add several of these features I know you want".  So, you know, let's do that.  There's efficiency in scale and the more time I spend on vector maps, the better they will get.

For everyone else, this is where I'd like to take the vector map support in WG-Maply.  Your feedback is welcome too.

Mapbox Style Support

Here's a short note on Mapbox Style Sheet support in 2.6.5:  It's waaaay better.

MapTiler Basic

I did some work for a client on MapTiler and Mapbox map sources.  You get to benefit.  Open source!

MapTiler & Mapbox

We were focused on Map Tiler primarily, so those work best.  You can find examples for their Basic, Hybrid Satellite, and Streets styles in the AutoTester app.

MapTiler Streets

As for Mapbox, we were trying out their Satellite offerings so you'll see examples for Satellite and Hybrid Satellite.  Streets will probably kind of work, but it wasn't my priority.

Mapbox Satellite Streets

You'll need to add in your own Map Tiler or Mapbox tokens.  I'm not that much of a sucker.

Mapbox Kinda Map

Mapbox-style maps lean more toward the web side of things: Load a lot of random junk before you're ready.  That's always been hard on my users.

Look for a new Swift object called MapboxKindaMap in the AutoTester app.  Just copy it into your app.  For various reasons it's hard to add an actual Swift module to the old version (2.6) of the toolkit.

At its simplest, give the thing a URL for the style and it'll figure out the rest.  Consult the MapTiler and Mapbox Test Cases for details.

Caveats

It's working fine for my client, but their needs were pretty simple.  If you want highway shields and fades per zoom level and that good stuff, it's not there.  I'll be bugging my users for money to add all that and make it work on Android.... soonish.

MapTiler has pretty flexible usage, but Mapbox does not.  So work it out with them before you ship anything.

WhirlyGlobe-Maply 3.0 Is Feature Complete

WhirlyGlobe-Maply 3.0 is ready, it's working and you can have it in on January 1 2020!

If you're a sponsor, you can have it whenever you like.  That's why you're a sponsor.

I just fixed the last missing piece which was.... the Layout Manager.  It's always the Layout Manager.

What 3.0 Means To You

My bigger clients paid me money for 3.0.  Being completely funded from the start was nice.  Really nice.

The point of 3.0 was Metal for iOS & a better Android version.  And sure, I threw in a few minor goals of my own.  I'm very pleased with the results.

If you're a sponsor, this means it's time to think about upgrading.  The new Android version is very comfortable and the Metal support is just WOW!

But if you're not a sponsor, it's cool.  The toolkit has a future.  Use 2.6 on iOS knowing you won't fall off the OpenGL cliff.  Use it on Android knowing it's going to get so much better.

Performance & Optimization

I like to make graphics run fast.  This is all an excuse to feed my peculiar hobby.  The Metal version is faster than OpenGL, but there's another level beyond.

So I've saved dessert for last.  After I upgrade a few sponsor apps, I'm using that data to make the Metal version even better.  And, okay sure, there could be some improvements on the Android side.

The Island of Lost Features

A few features didn't make it to 3.0, either for Android or both versions.

I used to have a custom LIDAR format.  The only people who used it don't seem to any more.  I'd build this differently now, so they can stick with 2.6.

Elevation is used by a few people, but mostly in a way that's better served by something else.  So it's out for now.  I'll come back in a better way, I hope.

A bit of the model support, like the Geometry Builder, Shape instancing, that kind of thing didn't make it to Android.  Aviation apps use it, but they're not as big on Android.

What's Next

Most of you get 3.0 at the beginning of 2020.  There's that.  I'll be continuing to maintain and support 2.6 on both platforms, so don't worry.

I'm impressed with what I'm seeing devs doing already!  You'll see some sponsor apps come out with better graphics and lower power use on iOS.  On Android, you'll see more ambitious use of the toolkit.

Long term, I'd like to circle back and improve the 3D data support.  A native implementation of 3D tiles, for instance.  But not until there's money for it.  No customers, no money, not gonna do it for free.

WhirlyGlobe-Maply 3.0 Status Report #4

Metal development has begun!

As a reminder, this is about the WhirlyGlobe-Maply 3.0 effort: A better version of Android and a port to the Metal rendering toolkit on iOS.

Quarantining OpenGL ES

WhirlyGlobe-Maply was designed around OpenGL ES.  Version 1 if you can believe it!  It's been upgraded since then, but OpenGL permeates its design.  Like fish in a microwave.

Multithreading and state management are particularly bad in OpenGL ES and my toolkit works around this in a variety of bad ways.

I've just finished shoving all of this logic into its own separate modules.  Twenty-something of them.  The top level interface doesn't change.  This was my own private pain I'm sharing with you!

Very little of the system now knows about the low level rendering which leads us to the...

Journey Into Metal

Sure, it sounds like a Heavy Metal Parking Lot sequel, but way less cool.  Behold the first of the new Metal code for setting up a texture.



And let's not forget the shaders.  Shaders that Xcode can syntax check!  And compile at compile time!  [Okay, fine there's probably a way to do that with an OpenGL extension, but it's guaranteed annoying.]



It's glorious!  In a boring way!  But there's a ways to go and nothing interesting to show yet.

Summary

If OpenGL development is Napoleon's Russia Campaign, Metal is like taking light rail to the mall.  Which is to say it's going well and there's no reason to suspect it won't continue doing so.

Ideally, I should designate 3.0 finished sometime in June.  If you're a sponsor, you get it then (or now, if you're bold).  For the rest of you, that means next January.

WhirlyGlobe-Maply 3.0 Status Report #3

I'm in the middle of the 3.0 upgrade to WhirlyGlobe-Maply.  As a reminder, this is bringing the Android version up to speed with iOS and then implementing Metal support for iOS.  On the way, I'm cleaning up a ton of old technical debt.

So where are we now?

The Android Port

This month was all about Android: Start with the test cases from iOS, trim the silly or pointless ones, move the rest to Android.

As boring as it sounds.  Let's look at a picture.

So yes, it's working.  This is the new data logic (Sampler/Loader) and it's just a ton faster and cleaner.

Even though the pyramid image tile loading is what most people use, there's a lot more to the system.  I'm steadily working my way through it.  Behold the glory of my spreadsheet!

The green is done, but may have a bug or two.  The mustard is in process and white hasn't been addressed yet.  There's about a week to go.

Android Internals & Development Flow

The original Android port was a janky affair.  It was built using Eclipse and the last generation of Native SDK build tools.  For my part the Java Native Interface logic was.... uneven.

Version 3.0 was an opportunity to clean all that up.  The switch to Android Studio and CMake (in 2.6) was a huge improvement already.

It's really the debugging in Studio that makes it so much more pleasant.    I can reliably set breakpoints on both Java and C++ sides.  And Studio only crashes every few hours.

But the point of it all was a shared C++ core.  After The Port No One Wants, I had that and I've been busily wiring it up on the Android side.  So if you see a MapboxVectorTileParser_iOS wrapper object, you can find an equivalent MapboxVectorTileParser_Android for talking to Java.  And they mostly work the same.

I've also simplified much of the Java/C++ interface.  There are fewer Java callbacks holding fewer Java objects on the C++ side.  Simple data marshaling, like conversion to arrays, is handled on the Java side.  Stuff like that cuts way down on possible bugs. 

As a result, I can debug on one platform and expect the fix to pop up on the other.  Multiple times I've fixed something on Android and then verified it on iOS.  That's new.

Wither Kotlin

Did I say Kotlin?  I didn't, but sure.  Am I rewriting everything in Kotlin and using the Native Kotlin transpiler to rewrite the C++ side and then using that on iOS?  No.  Seriously.  No.  Calm down.

I am writing new AutoTester MaplyTestCase modules in Kotlin, though.  It's pretty nice!

What's Next & Thanks

After a round of conference & vacation, iOS Metal is up next.  I'll start real development in a couple weeks and should wrap up June-ish.

Metal started the 3.0 effort so a big thanks to Apple for threatening to casually break everything.  That kicked the party off.

But more seriously, a big thanks to the sponsors of the 3.0 effort.  We've all worked together a while now and I appreciate your faith in me, and more importantly, your ability to pay my invoices.  WhirlyGlobe-Maply 3.0 is going to do great things for your apps!

WhirlyGlobe-Maply 3.0 Status Report #2

I'm two months into the 3.0 effort.  It's going well!

I covered all of this last time, but here's the plan.  We have an Android version and an iOS version.  The iOS version is always better and it's been hard to update the Android version.  To fix this, I need a unified C++ core used on both platforms.

So the first step was a separate C++ core on iOS.  And it's done!

The Port No One Wants

The newly merged C++ core is the Port No One Wants on iOS.  It does nothing new and it's buggy. But it's an important step on the way to a unified toolkit.  And there are bug fixes from Android that can benefit iOS, even it mostly goes the other way.

The big news is it works!  WhirlyGlobe-Maply 3.0 for iOS is now 50% common C++ core.  I was hoping for more, but damn those Mapbox style parsers are wordy.  And I want to leave that in ObjC so users can modify it themselves.

It wasn't all C++ core, though.  I've moved the QuadPaging logic, what you use to load things other than images, over to the new Sampler/Loader architecture.  Man, that fixes a lot of problems.

To Android and Beyond!

Now I'm working on the Android version.  The common C++ core compiled easily and it's on to the JNI interface.  If you're not familiar, this is the C++ interface to Java and it is pretty "meh".

I'm slogging through that logic now and it's slow going.  Thankfully Android Studio has gotten better with native code.  But honestly, it was a pretty low bar.

The goal is feature parity between the platforms and I think we'll get pretty close.

Unwinding Technical Debt

You accrete a bunch of technical debt in 8 years (!) and now is the time to clean it up.  Here's a very boring list of some of it.

• MaplyShader and the low level OpenGL ES programs now unified into one representation.
• Texture loading for tiles now consolidated in one object which works in one place.
• Elevation callbacks removed to be replaced with something more logical.
• The entire RemoteSource infrastructure removed in favor of something simpler and more flexible.  Yeah, this will cause problems.  See the next section.
• Cut the AutoTester app back to just the essentials.  Much easier for users to read.
• Got rid of the DynamicDrawableAtlas.  Android never liked it.
• Gestures now live at the Component level with no duplicates lower down.
• ActiveModels only exist at one level in the toolkit.
• Generators are now gone, replaced with ActiveModels where needed.
• Component Objects are now a concept in the core toolkit, not just the high level.
• Paging controllers handle arbitrary visual objects as well as images.  Really nice for vector maps.
• Rather than passing around View objects, we use ViewState objects.  Fixes some threading problems.
• Managers no longer being fetched by string.  Not sure why I never did that.
• More logic moved into the BaseInfo descriptors.  Render target, that sort of thing.
• Moved locking logic over to C++11 constructs.  Much cleaner.
• Passing around references to CoordSystems, fixing a few recurrent shutdown problems.
• Moved vector subdivision logic into a unified module.

You read this far?  Weirdo.

There was a lot of trivia to clean up in 8 years.  Now it all kind of makes sense.  Except OpenGL ES.  I'll be reorganizing that when I get to Metal.

(Not Going To) Sunset Version 2.6

I'm breaking things in Version 3.0.  The new version will work better, but it won't be 100% compatible with 2.6. 

The big issue is the move from the QuadImages and QuadPaging layers to the new Sampler/Loader approach.  I'll cover that more in its own blog post (with pictures!) but for now let's just say it's better and I'm not going to make it backward compatible.

In that spirit, I'm going to continue supporting 2.6 for both iOS and Android into the future.  This will include bug fixes and updates for new platform versions.

Android, Android, and more Android

The next few weeks is all about Android.  I'll continue hooking up C++ through JNI to the existing Java.  Then I'll start on the new interfaces and loading logic that Android never had.

Lastly, it's on to testing and feature parity.  I honestly forget what's in the Android version and I'm kind of excited to have a full feature spreadsheet comparing the two.

WhirlyGlobe-Maply 3.0 Status Report

The 3.0 effort's been going all month.  So how are things?

Things are boring.  Boring, but good.

Collateralized Technical Debt Obligations

The Android port of the toolkit was supposed to share C++ code with the iOS version.  It never really did.  This is the fundamental problem between iOS and Android.  It hurts the Android version and it's holding back other ports.

On Android we have a hard line between Java and C++ (JNI).  When I ported things the first time, I made a copy for Android and then hacked as needed.  Naturally, the versions diverged between platforms.

I've merged the C++ cores back together and introduced that hard line between Obj-C and C++ on iOS.  This should let me fix bugs for both platforms at the same time.  I'm excited.  In a boring way.

The Port No One Wants

All of this gives me an iOS version based on a solid C++ core.  No new functionality, not going to be faster or slower.  And it's going to have bugs for a while.

Which is to say, it's not a port anyone's excited about, but it's vital.  Once that's done, I get back to the good stuff.

Good Stuff Part I

Android's behind on new functionality.  The whole Sample/Loader approach, which is working so well in 2.6, is only on iOS.  Before I port it, I need to clean up some old things (QuadPagingLayer), remove stuff I've replaced (QuadImagesLayer) and make sure those deprecations can be supported.

That's more fun than it sounds, I swear.  The QuadPagingLayer is used for loading all sorts of data objects and its replacement, based on the new Sampler/Loader, will fix a host of problems.

On the more exciting front, there's elevation support.  It's always been a mess on iOS and never worked on Android.  It'll be fun to fix that definitively.

Other good stuff will follow, including but not limited to, Metal for iOS.

Saildrone Forecast

Is there anything cooler than shipping an app you've spent a year and a half working on?
It was a rhetorical question.  There is not.

Saildrone Forecast

Hint: It's a weather app.

Saildrone Forecast

Yup, Saildrone Forecast is a weather app.  Just watch the video.

Go download it.  I'll wait.

Technical Bits

There's a ton of stuff in Saildrone Forecast that's unique.  But there are a couple bits of interest in WhirlyGlobe-Maply itself, like the new sampler/loader architecture and offscreen render targets.  Go watch the video.

Could you make your own weather app out of the box now?  Ha ha.  Oh god no.  There's so much other stuff to do.  Not to mention the data.  But there are cool things to do with the vector maps and render targets.

Acknowledgements

I didn't do this all alone.  Heck, I didn't even write the whole app. 

Credit goes to Logical Animal for the rest of the UI.  Big thanks to the meteorology/modeling team at Saildrone for the data and putting up with stupid questions.  Thanks to the platform developers and ops for processing data and keeping it running.  And, of course, management and finance upon whom I can bestow the greatest contracting honor:  They always pay early.

Technical Advisory Committee

I'm forming a technical advisory committee.  We'll meet a few times and discuss the iOS Metal port and rendering architecture in the toolkit.

This post is for developers I'm trying to recruit to the committee.  Let's start with the basics.

What is WhirlyGlobe-Maply?

National Geographic World Atlas (RIP)

WhirlyGlobe-Maply is a real time geospatial display toolkit for iOS and Android.  It's used in a whole bunch of apps, like FlyQ EFB and a few big weather apps like Dark Sky.

Dark Sky

The toolkit's been around since 2011 and it's carved out a crazy little niche in aviation and weather.  It's not a wildly profitable business, but it doesn't lose money either.  How many can say that?

Morecast

Oh, and it's open source.  So that's fun.

WhirlyGlobe-Maply 3.0

The last version I shipped was 2.6 a few weeks.  A huge improvement for data loading on iOS and a nice upgrade to the Android build system.  I.... need to do a post on that.  But no time.

3.0 is the big upgrade.  I'm going to add iOS Metal support, redo the rendering internals and upgrade the Android port with a better shared C++ core and newer versions of OpenGL ES.

Technical Advisory Committee

So here's the deal.  I'm adding a new renderer for Metal and replumbing the internals to match.  I'd really love to discuss it with a few experts along the way.

This is an open source project so, you know, not a ton of resources.  Here's what I'm offering:

• Free lunch in a pleasant conference room at my coworking space in San Francisco
• An invitation to pontificate on real time rendering without having to do any work
• A cool entry on your resume

You do not need to know anything about geospatial or weather or aviation.  I got that covered.  Here's what I need from you:

• Work in San Francisco or nearby.  No budget for travel, sorry.
• Know something about real time rendering, iOS Metal or Android OpenGL 3.x
• Willingness to chat about real time rendering

It's your real time rendering chops I'm interested in.  So game developers most welcome!

Contact me at sjg@mousebirdconsulting.com and we'll see if you're a good fit.

Data Paging Layers

We load a lot of data sources in WhirlyGlobe-Maply, usually over the network.  Images are relatively easy, but how do you deal with things like vector tiles?

Ye olde vector tile example

That would be the job of the Paging Layer.

Quadtree Structured Data Paging

It all starts with a quadtree, a simple enough data structure.  It's here that we decide what's visible and what needs to be loaded.

Quadtree courtesy wikipedia

The quadtree for a data source can be in any proj.4 supported coordinate system, but most people use Spherical Mercator which is... what they use.  No judgement.

In deciding what to load, we start at the top node (0: 0,0), projecting a box into the view and then recursing if the box is visible.  It's not all that expensive, because quadtree.  But does get a bit tricky when using the globe or multiple coordinate systems.

For image layers, the toolkit has its own complex logic for loading image tiles and frames and handling failure and so on.  We provide similar logic for vector tiles, but for anything else, the logic is you.

Quad Paging Layer

You've got a data source, sqlite file full of points, let's say.  You want to turn them into markers, but there's too many to load at once.

Here's where the QuadPagingLayer (Android) or MaplyQuadPagingLayer (iOS) comes in.  Set up one of these babies with the coordinate system and extents of your data source and it'll figure out what to load.

You have to create the visible objects and that's where the PagingInterface (Android) and MaplyPagingDelegate (iOS) come in.  The paging layer calls you back when it's time to load a tile's worth of data.

Mt. St. Helens LIDAR

We love our threading in WhirlyGlobe-Maply and it was ever so hard to implement.  So use it.  Try a dispatch queue on iOS or use a worker thread on Android.  But don't hold up the paging layer.  It has other things to do.

When you make a visible object (marker, label, etc) tell the paging layer what you created with the addData (iOS, Android) method and call tileDidLoad (Android, iOS) when you're finished.

That's enough information for the paging layer to do the rest, which includes...

Cleanup on Node (12: 129,1304)

One of the trickier bits here is deciding when to delete something.  Luckily, you don't have to.

You told the paging layer what you created and it'll delete it for you.  There's even a callback if you really, really want to know it did that.

Paging Layers are Awesome

There it is, one of the key pieces of WhirlyGlobe-Maply: the paging layer.  We use it all over the place and if we don't support your data type, you'll use it too.

Styled Layer Descriptor and Metro Extracts

In WhirlyGlobe-Maply 2.5 we added support for the Styled Layer Descriptor format.  We also added an example in the AutoTester and a tutorial.

Belfast, it seems.

Let's start with the data.

Metro Extracts

Mapzen provides regular extracts of OpenStreetMap data.  These are available in a variety of formats; useful if you just want a little OSM data.

Courtesy Mapzen

We chose Belfast in GeoJSON for our own example.  This is just data, though, so how do we make it visual?

Styled Layer Descriptor

The Styled Layer Descriptor format is an OGC standard used to style data into map images.  It's a bit crusty but, you know, standard.  We edit it in QGIS and export it for use in our toolkit.

WhirlyGlobe-Maply reads SLD and uses it to style Shapefiles, GeoJSON, or Mapbox Vector Tiles.  Any vector data, basically.  So let's put it all together.

Example & Tutorial

Rather than blather about it here, we put a nice example in the AutoTester app for iOS.  In Swift!

Ranen put together a tutorial that walks you through the process.  You can start with a Metro Extract and modify the SLD styles to your heart's content.

Up Next

We're picking up SLD users for the toolkit at a steady clip.  There's a lot more to SLD we could support, so let us know how you like it.

We'd also like to circle back and improve the Mapbox GL Style Sheet support and implement Mapzen's Tangram format.  But we need customers for those.  Speak up if you're interested.

WhirlyGlobe-Maply 2.5 Release

Version 2.5 is out for both iOS and Android.

Location Tracking

You can get v2.5 off of the build page or from the master branch.  We'll update the master Podspec for Cocoapods soonish.

As usual, getting this version out the door took way too long.  But here it is and it's really good.

Big Changes in Version 2.5

Android.  Without a doubt.  We now have a good Android version with a bunch of deployed Android apps.  Millions of people are using WhirlyGlobe-Maply on Android.  Crazy, right?

We started doing automated builds.  I can't over stress how useful those have been.

Both iOS and Android have AutoTester apps; individual test cases presented in a list format.  Useful both for testing and user examples.

We upgraded to Swift 3.0 and kept pace with all the new Xcode versions and UIKit deprecations.

Ripped out AFNetworking and Boost.  Respect to both packages, but AFNetworking was interfering with users' own version and we eliminated Boost with C++11.

Vector (tiled and not) support got fancier with Styled Layer Descriptor format and the Simple Styles approach for building your own.

True 3D interaction got fancier in support of LIDAR.  There's an example in the AutoTester so check it out.

We added user location tracking and display.  A relatively simple change, but a popular one for map users.

Important but Boring Stuff

We made an enormous number of small fixes, efficiency improvements and minor functionality tweaks.  Also a lot of new tests for things that are (no longer) broken.  Here's a partial list in a boring format no one is going to look at.

Maps now have ViewState animations like the globe.  Feature selection is smarter with individual and group features.  Widened vectors is faster, smaller, and better.  We've got a 3D model builder for things like airstrips.  Elevation support is better and more flexible.  Stars and atmosphere work better.  Vector tiles kept up with new versions and added more style options.  Particle systems work better.  Internal coordinates are now double everywhere for getting really close to things.  3D support includes the ability to "grab" objects via the IntersectionManager.  Removed Mapbox GL Style Sheet support and then put it back.  Single level loading works on the globe now for both vector and image layers.  Screen Label layout works correctly for all modes.  Great circles work in 2D across the date line.  And much, much more.

What's Next

We're going to make some internal changes to the source tree, like unify the Android and iOS branches.  Then we'll move to git-flow methodology to hopefully speed up minor releases.

In terms of functionality, I've been rolling out statements of intent for things like Windows, True 3D support, MapKit, and Mapbox support.  We've also got the Imagery Pro toolkit for weather apps and another one in the pipeline.  Much of this requires money, so let me know if you have some.