Thursday, January 5, 2017

Part 2: Future 3D Support in WhirlyGlobe-Maply

Last time we looked at WhirlyGlobe-Maply's existing 3D support.  It's considerable, but often hard to use.

FlyQ EFB and NATS Airspace Explorer

Now we'll talk about where we're heading.  It's all about the data sources.

Putting the 3D's in Data

Most of our users have fairly limited data processing capabilities.  Say you want to add elevation to an app.  We'd say:

  • Gather all the elevation data you need.  USGS is a good source.
  • Stitch it all into a single giant image in Spherical Mercator
  • Chop that image up into tiles using this random tool we have on github
  • Load the resulting sqlite file onto the device.

Easy enough, right?

Yeah, not so much.  That's why you see so few WG-Maply apps with elevation.

It's not just us, you know.  OpenStreetMap data is much easier to use when you move from Planet Files to Metro Extracts and Vector Tiles.

Structured 3D Data Sources

What developers really want is web services that they can plug into their apps.  Also pizza that lifts itself into your mouth.  Pizza robot.  Patent pending.

People want to point their app at an elevation, 3D building, or vector tile service and stuff just happens.  So let's look at some of those services and the formats they support.

Two New Styles of Elevation

Elevation tiles are an obvious thing.  We already support Cesium's elevation, but there are others out there now.

Mapzen came out with a straight up elevation tile service a few months ago.  There are a few minor technical hurdles, but it could be made to work well with the toolkit.

Mapzen Elevation Example

Mapbox came out with their own elevation tile service shortly thereafter.  It's a different approach, with similar problems in 3D but could also be made to work.

Courtesy Mapbox terrain-rgb post

So there you go.  We'd like to support Mapzen & Mapbox elevation data for 3D.

Buildings with Height

OpenStreetMap buildings have height (usually, mostly).  Mapbox has been showing this off lately in their APIs and Mapzen has been doing it for a while in Tangram.

Mapbox Data w/ Mapbox GL

We'd like to support height in vector tile data.  Not too difficult, obviously, but cool.

Now for the harder stuff.  Let's start with a detour into data formats.

GL Transmission Format (glTF)

The 3D file format we support right now, Wavefront OBJ, is old and weird.  We need to move to something new and... less weird.

glTF is backed by the Khronos  Group with lots of work from the Cesium folks.  It's a new, relatively simple optimized model transmission format.  Easy to read, fast to transmit, it solves a lot of problems for models like these.

And it enables some even crazier stuff.

3D Tiles

The Cesium folks have a 3D Tiles proposal out that looks great.  It's open, they've got example implementations, and they're trying to make it an OGC standard.

3D Tile data displayed with CesiumJS

That approach is well suited for mobile.  We can't display as much as a desktop can, but we can display a lot.

Naturally, we'd love to support this in WhirlyGlobe-Maply.

Point Clouds

We already support point cloud display for things like LIDAR.  It's restricted to a Sqlite based format read on the device.  Hey, it's what the customer needed at the time.

San Simeon LIDAR

We'd love to extend this to services like Greyhound, an open source streaming data server.  We'd also like to add support for the GeoPackage point cloud extension.


There's a ton of explicit 3D support in WhirlyGlobe-Maply for interaction, model construction, and motion.  We'll continue to push in these areas and others we haven't talked about, like 3D weather.  But that stuff is driven by high end client needs.

NATS Airspace Explorer

What we'd really like to do is support these newer services for our regular users.  If you have other services you'd like to see or, better yet, some budget, please let us know.

Monday, December 5, 2016

Part 1: Existing 3D Support in WhirlyGlobe-Maply

We've got a two parter for you.  First up we'll look at the existing 3D support in WhirlyGlobe-Maply.  Next time we'll lay out the roadmap for 3D in the future.

So let's start with the question:  Isn't WhirlyGlobe-Maply already 3D?

3D Minus 1D Equals 2D

Everything WhirlyGlobe-Maply does is 3D, but we hide it most of the time.  Look at some of the more prominent apps, even ones that use the globe.

Dark Sky, Gaia GPS, NatGeo World Atlas

The giveaway is the gestures.  Can you tilt?  Nope.  And we're not hiding some wealth of information.  If you did tilt toward the horizon you'd see.... nothing much.

To be truly 3D an app needs to turn on the 3D gestures and it needs data with real 3D structure.  There are a few that do it and we provide a lot of functionality to support them.

LIDAR Point Clouds

Did you know WhirlyGlobe-Maply supports LIDAR in a pageable sqlite format?  It does and it's pretty cool.

Available on both iOS and Android in 2.5 this is an essential use of 3D.  You couldn't see all that much in 2D... well okay you can, but it's better in 3D.

Truly 3D Terrain

We've been able to do true 3D terrain for years.  Aviation apps use it.

FlyQ EFB Synthetic Vision Mode

Why don't you see that more?  Well, it's useful for aviation and.... um, games maybe?  Users tend to be specialized.

Models, Markers, Billboards, Labels & Lines

The toolkit can display 3D models in Wavefront OBJ, a data format old enough to vote, but it works well enough.

We can also do billboards, which are like a 3D version of markers and, of course, labels.  Here's what you'd use that for.

NATS Airspace Explorer

Inspired to make your own 3D traffic app?  Of course you are, but it's a lot of work to get the display right and the data is expensive.

General 3D Geometry

Lots of our users make stuff up on the fly.  Why?  Because if there was a pre-packaged version of it, they wouldn't be making an app.

Fake airstrips are fake.  Do not try to land.

For stuff like this we've got a model generator.  You build up your special snowflake of a model polygon by text by line as needed.

If you've already got 3D geometry, you can hand the triangles over to the renderer.  Hard core, but useful in a few cases.

Lofted Polygons and Extrusions

On the easier side are a couple of features that let you think in 2D and tack on a height.

Downright candy colored

Lofted polygons have been around a long time and work well on the globe.  They're nice for showing relative height... and they just look cool.

Extrusions are a way of whipping up a model without having to do, like, math.  You can slap together an arrow, for example, to point at things.

Where We're Heading

WhirlyGlobe-Maply has extensive 3D support and real apps (mostly aviation) have been using it for years.  It's all rather do-it-yourself though, particularly on the data side.

There are a number of good data sources for 3D coming on line recently.  In the next post we'll discuss how we'd like to use them and where we'd like to go.

Monday, November 21, 2016

The Return of Mapbox Streets

A few weeks ago we withdrew support for Mapbox GL Style Sheets.  You can read about our reasoning if you like.

Courtesy Mapbox

That's all changed!  Mapbox is opening up their mobile license for WhirlyGlobe-Maply users and offering competitive mobile pricing.  We're responding by making it easier to use their data.

License and Pricing

WhirlyGlobe-Maply developers will be treated just like Mapbox GL devs for tracking and billing purposes.  You should email to get an amended license that will look a bit like this.

If you want to use Mapbox vector tiles on more than 50,000 devices or you're doing device tracking, talk to them.  That's all standard stuff in their agreement.  If you want the mobile pricing, talk to the sales department.

Telemetry & Offline Use

We're being treated as an equal to Mapbox GL here and that is great, but there are responsibilities. Mapbox GL sends back telemetry to improve OpenStreetMap and do traffic stuff.  We'll have to do the same.

The standalone telemetry library isn't ready and we're proceeding without it.  When it's ready, we'll integrate it with WhirlyGlobe-Maply.  It'll be on only for Mapbox data sources.

As for offline Mapbox Streets use, you're subject to the same rules as Mapbox GL.  If you ask how to work around that, I'll be happy to refer you to Mapbox's terms of service.

Who's This For?

WhirlyGlobe-Maply occupies a niche in real time data display.  We're big in weather and aviation apps, with a smattering of other specialized users.  And developers who just want a globe, obviously.

I see the biggest potential in two kinds of base maps.  First up, Mapbox Satellite Streets.  This fills in a nice hole in apps that want an optional satellite mode with a sprinkling of transportation.

Also Mapbox

Next up, a stripped down variant of Mapbox Streets suitable for data overlay.  They have a few starting points here and we can tweak a bit.

Schedule & Shout Out

WhirlyGlobe-Maply 2.5 is getting close to release so we're not going to hold it up for this.  We'll put the Mapbox GL Style Sheet support back in 2.5.1 and do a bit of testing.

Lastly, a shout out to the folks at Mapbox.  We thought you'd gone all Mapquest on us, but you haven't!  Cool.

Wednesday, September 14, 2016

MapKit Compatibility Roadmap

We get asked this one a lot:  "Why can't you be more like MapKit?"  It's a good question.

MapKit, obviously.

Internally the toolkit won't ever be more like MapKit.  But we can pretend in a couple of ways:  Overlays and Interfaces.

MapKit Overlay

MapKit provides overlay options for everything from markers to tiled image sources.  Annotations are super flexible, but very slow beyond a handful.  And don't even think about animated tile sources.  Ouch.

For some data sources it's going to be easier to render in WhirlyGlobe-Maply and slap the results on top of MapKit's UIView.

It's not topical, I just like MapKit's stations.

That's the solution.  We'll track what MapKit is up to, at least in flat map mode, and overlay our own rendering to match.  Think weather apps.

MapKit Interfaces

MapKit has a nice collection of interfaces for adding annotations, great circles, and all sorts of other good stuff.  We'll try to be more compatible with those.

Portland in MapKit.  Why not?

This means methods that take MKMapPoint or CLLocationCoordinate2D objects.  It means init calls that are similar to the MapKit equivalents.  And yes, it means working in (shudder) degrees.

The goal here is a simpler conversion from a MapKit app to a WhirlyGlobe-Maply app.

UIView vs UIViewController

This one really bugs a handful of developers.  Our main interfaces are derived from UIViewController rather than UIView.

View Controllers are nice because they have life cycle events.  You have to wire all that up yourself in a UIView.  But it's possible and we'll do it because we know you want it.

Roadmap Not Actual Size

That's the plan and this all goes in the base toolkit so it'll be free.  But we need money to do it.  Always with the money.

As with all our plans, it'll get done eventually.  Pieces will go into proposals, clients will step forward, and some of it we'll just do ourselves.

Got feedback?  Let us know.

Wednesday, September 7, 2016

Android Tutorials

We have tutorials!  For Android!  Yay!

Screen shot of a web page?  I'm not proud.

A big thanks to Nicholas for putting these together.  And to José for doing the binary AAR examples.

Getting Set Up

The first few tutorials focus on setting up a really basic project and getting the WhirlyGlobe-Maply library included.  We now have three ways of doing this from least Android-y to most Android-y.

Yes, we're doing nightly builds for Android and we have the JCenter/Maven thing set up.  Once you've got the toolkit included, you can peruse the tutorials on actually doing stuff.

Enough With the Typing

There's nothing more to say.  Go check out the tutorials if you're doing Android development with WhirlyGlobe-Maply.

If you'd like more tutorials on iOS or Android, open up an Issue.  That's how we communicate.

We can thank the Support Customers for this since it came out of their budget.  More support contracts means a bigger budget.  It's all spent on stuff like this.

Tuesday, September 6, 2016

Mapzen Vector Tile Service

We're going to support Mapzen's Vector Tile Service.  Currently that means we read their vector tiles.  In the future we'll support their style sheets.

Courtesy Mapzen

Let's do a little background and then get to the details.

Vector What Now?

We've been slinging around vectors for years in WhirlyGlobe-Maply so vector tiles weren't a huge diversion.  We even had our own format for a while.

These days vector tiles means Mapbox Vector Tile format, but it implies two other things:
  • A curated version of OpenStreetMap suitable for web and mobile map display
  • One or more style sheets to describe the visual representation

You might call this a Vector Tile Service if you were so inclined.  Developers want to shove a map underneath the thing their app is actually doing.

It's a trivial, seamless process with an image tile service.  We're going to make it just as easy for a vector tile service.

Here is our roadmap to get there.

Mapzen Vector Tile Service

We can parse and display Mapzen's vector tiles just fine in either GeoJSON or Mapbox Vector Tile format.  But we use really simple style sheets.

Must be night or something

That looks kind of meh.  For a real map we need better.  There are a number of ways to do style sheets, but I'd like something that requires minimal upkeep.

Mapzen has a Javascript/C++/Java based renderer called Tangram.  It's got it's own style sheet format.  That format is.... very specific to Tangram. 

I've been on the fence about their style format.  Some discussion was in order.

Data Formats & Common Usage

Data formats have crazy parts.  Go look at PDF (better yet, don't).  You have to decide what's common usage and what parts you ignore, at least for a while.

The Mapzen folks were kind enough to host a meeting where we could explore this.  It looks like the common usage among their various style sheets is blunting the weirdness.  They've got professional cartographers using it, in addition to engineers, which is a big plus.

In short, when we see some deep_Tangram_weirdness we can substitute minor_WG-Maply_weirdness and it'll look pretty similar.  Good enough. 

There's Open and Then There's Open

The most attractive thing about the Mapzen Vector Tile Service is the terms of service.  You can use the data in any API and in a variety of ways.  As long as you're not completely rude.

Best of all, you can mix online and offline databases without violating the license.  That makes it perfect to slip into a WhirlyGlobe-Maply app and in our test apps.

That's not a knock on other services.  I get that the API and data are often tightly linked by a license.  That's how you make money.  And speaking of money...

Roadmap and Funding

The vector tile part works just fine in WhirlyGlobe-Maply now.  We still need to implement the Tangram style format.  We're just wrapping up Styled Layer Descriptor support so it shouldn't be all that difficult.

It does require money.

That's the roadmap.  If you're interested in any part of it, speak up.  I get asked about this a lot, so it'll get paid for eventually.

Tuesday, August 30, 2016

State Of the Map - Brussels

I'll be at State Of the Map this year in Brussels.  SOTM is running from September 23 through the 25th.

Apparently purple is the thing this year.

This is something of a convenient detour.  We're heading to the Meteorological Technology World Expo that next week in Madrid.

I've got a table and I'll be showing off the usual stuff.  Given the crowd, I'll lean more on the map apps.

If you're going to be there, drop by and say hello!