Tuesday, March 29, 2016

Map Features in WhirlyGlobe-Maply

I'm going to discuss map related features of the WhirlyGlobe-Maply toolkit. The SDK is an open source, mobile centric geospatial display toolkit. It’s used in a variety of weather, aviation, geography and map apps.

Two of the more prominent examples are Dark Sky, a weather app and National Geographic World Atlas, which is exactly what it sounds like.

NatGeo World Atlas & Dark Sky

The globe apps are best known, but WhirlyGlobe-Maply has huge support for traditional 2D map apps too.  We're going to talk about some of those features, but let's start with map apps in general and one in particular.

Map Apps

Mapping is one of the more popular categories in the app stores. Both Apple and Google have their own map offerings and toolkits, but they're poor if you have your own data? That’s where toolkits like WhirlyGlobe-Maply come in and we’ve built a number of features specifically for those sorts of apps.

One of the more capable map apps that uses the toolkit is Gaia GPS.

Fact: 30% of hiking app screenshots are Yosemite.

Gaia GPS displays a variety of image and vector based data sources. It’s popular among the backpacking and off roading communities.  It can work without a network and is very careful with battery.

Let’s move on to one of the key features for nearly every map app: Image Tiles.

Image Tiles

A basic feature required for map display is loading and rendering of image tiles. We’ve moved well beyond Web Map Service on mobile devices and we now expect tiles. This might be a Web Map Tile Service (WMTS), but is more likely to be a fixed quad tree of tiles like that coming from Google Maps or OpenStreetMap.

Efficient image tile loading is a tricky problem. Mobile devices have network constraints and we want to make careful use of data caching. WG-Maply can handle this for both globe and map, but there’s an interesting variant for the map case.

In the example we’re loading three levels of tiles based on the current window. We start with an extremely low resolution, then overlay that with slightly higher resolution and finally toss in the proper resolution. It gives the user something to look at while the system loads the good stuff.

Tricks like these are important to efficient map display on mobile, but let’s take a quick detour through some really basic functionality.

Vectors, Labels and Markers

Any credible map toolkit should be able to display some basic features and WhirlyGlobe-Maply is no exception. These include vectors for areal and linear features, labels for points and of course markers.

All very simple to us.  Vectors, for example can be consumed from a variety of data sources, like GeoJSON or (ESRI Shapefile). Labels and markers are 2D features and can interact through an adaptive layout engine.  All of them provide selection feedback too.

It’s what the developers do with these features that gets interesting. That leads us to a complex modern feature: vector tiles.

Vector Tiles

Mapbox has popularized vector tiles with their OpenStreetMap data set. Theirs is a Google Protobuf based format, which is as complex as it sounds. The fundamental ideas are simpler.

Vector tiles are to image tiles as vectors are to images. They are just chopped up geometry representing what’s in each map tile. You can use them much more flexibly than images.

A very custom example of vector tiles.
Believe it or not, National Geographic World Atlas uses vector tiles hosted by Mapbox.  The rest of it is completely custom.  And, okay, it's technically a globe.  But if National Geographic doesn’t have some cartographic street cred, nothing does.

A more normal use of vector tiles.

The Gaia GPS developers use vector tiles they generate themselves to draw contour lines. The data is styled using Mapnik XML which is associated with the Mapnik map renderer.  Gaia GPS also has a street layer based on the same approach.

WhirlyGlobe-Maply has vector tile support down solid, but style support is still in flux and likely to be for a while. We can support older style formats like Mapnik XML and developers can always go off on their own. Newer formats like Mapbox GL Style Sheets are not fully supported yet. We also hope to add in Styled Layer Descriptor since it's an actual standard.

Vector Tiles are a bit trippy so let’s go back to some more basic map features.

Wide Vectors

Widened vectors are one of those obvious things that require some work to implement.  OpenGL ES has a basic line implementation, but it looks kind of... meh.  If you want lines that have smooth edges and neat little joins, you need something more.

WhirlyGlobe-Maply's implementation involves a custom OpenGL ES shader and a bit of math.  Line intersections are calculated on the fly (sort of) and width can be varied as needed.  Typically this just looks like constant width lines as the user moves in and out.

Look deep into my shader
Oh, and you can do dashed lines and other effects.  Wide vectors work both on the flat map and the globe, but they're primarily a map feature, just like the next one: clustering.

Marker Clustering

Clustering markers is a pretty simple idea. When you have a lot of markers on your map, it can look bad. Best to gather them up into groups and display the groups.

Simple enough, but our maps move so we have to do something more. As the user zooms in, the groups come apart and as zoom move out, the groups form. It’s tricky to implement, but looks quite nice.

In truth, clustering can be used on both the globe and the map even it’s nominally a map feature. Let’s look at something purely from the 2D map.

Infinite Scrolling

For certain types of map projections, the extreme west and east extents represent the same point (really, line). This means the user expects to move the map and have it wrap around.

Bonus question: What about wrapping top to bottom?

Infinite scrolling is simple in concept, but tricky to implement. The rendering isn’t bad, but when you add in feature selection and overlaid data it gets interesting. You know what else is interesting? Map projections.

Map Projections

The real test of a map toolkit is whether it handles explicit map projections. Actually, that’s not true. Plenty of map toolkits only work in a simple map projection, known as web mercator. But if you want to hold your head high in a room full of cartographers, you should have map projection support.

Does this projection make my poles look big?

Web mercator, shown here, is the most common projection in use on mobile. It has its problems, particularly near the poles. We can thank Google Maps’ dominance for this affront to cartography.  There's data in plenty of other projections and good reason to use them.

From left to right we’ve got:
  • A data source in British National Grid overlaid on a web mercator map.
  • The same data source overlaid on a British National Grid map. The web mercator data source is being reprojected.
  • The same thing on a globe. Because it looks cool.
Though the vast majority of our users will always use web mercator, WhirlyGlobe-Maply can do more. It uses the Proj.4 coordinate system package internally and can handle a variety of useful map projections.


Map toolkits for mobile devices are hot right now. There are a number of excellent ones, though most are tied to specific data services like Apple or Google or are proprietary or both.  A few are open source, like WhirlyGlobe-Maply.

We’ve discussed some basic features and a few advanced ones we consider essential for map display on mobile. If you’re looking to build an app, we hope you’ll look beyond the proprietary services to something like WhirlyGlobe-Maply.

Special thanks to Stamen for the Stamen Watercolor map tiles, derived from OpenStreetMap data, used in many of these examples.

Monday, March 14, 2016

Map Projections

Let's talk map features.  How about custom projections?

Does this projection make my poles look big?

Spherical Mercator (web mercator) is the most common map projection you'll see.  It's a fairly simple unraveling of the earth, but its main purpose is to annoy cartographers.  Did you know the EPSG code for Web Mercator stands for "DIAF".  True fact.  Look it up.

Our users mostly stick with web mercator, but we now let you specify your own custom projection.

Proj.4 CoordinateSystem

Like all the cool kids we use Proj.4 to represent coordinate systems.  In theory this allowed us to use a wide variety of systems.  In practice, we just used Spherical Mercator, Plate Carree and a very weak geocentric.

Recently we exposed Proj.4 directly with a MaplyProj4CoordSystem.  It's simple enough, if you're familiar with Proj.4.  Here's an example for British National Grid.

Full disclosure, there's actually a datum grid shift that goes with that for proper BNG.  We've shortened it for the example.

You can get these Proj.4 strings from spatialreference.org, among other places.  But odds are if you have the need, you already have the string.

British National Grid

The folks at Ordnance Survey were kind enough to help us sort this out.  Let's look specifically at the British National Grid, starting with a data source in BNG.

BNG tile source on spherical mercator

That's a 2D map with a base layer in web mercator (Stamen Watercolor) overlaid with a dummy tile source in BNG.  As far as these things go, that's the easy case.  Let's look at a more difficult one where the map itself in BNG.

BNG tile source on BNG map

In this one the map is in BNG with a base layer in web mercator and an overlaid dummy tile source in BNG.

That's pretty cool!  Web mercator stretches far outside the boundaries of the BNG system, so we convert the tiles to BNG and toss out any that don't overlap.  And it works!  But there's more.

BNG tile source on globe

Okay, not all that difficult as these things go but it looks cool.  We're just overlaying a dummy BNG source on the globe.

You can find all of these examples in the AutoTester app for iOS.  And there's more...


This is something I hope to say a lot more in the future:  Custom map projections are also available on Android in the develop_3_0 branch!

A triptych of British National Grid

You can find these examples in the AutoTester app for Android.  I'll explain AutoTester in more detail soon, but it's already been a great help in development.

Next Up

The hard part here was tracking down the weird corner cases and systems which just don't play well with each other.  There are likely to be a few more like that, so let me know what you find.

We've got more map features in the queue, including some vector data display.  Stay tuned!

Friday, March 4, 2016

CartoType GL

In cooperation with CartoType Ltd, I am proud to announce CartoType GL for iOS and Android.

Parks are green!

CartoType GL is the next generation of the CartoType map toolkit.  It's got all the flexibility of the existing library for map representation, routing and address searching, but we've add a zippy OpenGL ES renderer.

What is CartoType?

CartoType is an offline map library that works with limited resources.  This makes it popular on smartphones and other situations where the user is not connected to a network.  It can handle a variety of data inputs, making it easy to use your own data, but can also deal with the usual OpenStreetMap cases.

I wonder where this is?

The existing CartoType library does a really nice job representing complicated maps in a very small space.  It's primarily a static map rendering library, so when you use it on a smartphone it'll render the map when you stop moving.  What we've done is upgrade the renderer.

What is CartoType GL?

CartoType was already doing a fine job representing maps.  What it needed was to draw them faster.  CartoType GL is just what it sounds like, an OpenGL ES based version of CartoType that solves the speed problem.

CartoType GL marshals the resources of the existing library to convert data to internal vector tiles.  Those are are processed into OpenGL ES compatible data and rendered at a nice zippy, interactive speed.  Basically just what you'd expect in a modern map toolkit.

Online vs. Offline

There are some excellent cloud based map solutions out there for map representation, routing, address searching and so forth.  CartoType GL is not that; it's an offline solution.

I have no idea where this is.

Why offline?  Well, it's actually a different problem.  You can cobble together offline solutions from the online technology, but they're not great.  They're too big, they don't do routing well and they're focused on display.  If you want to do a good offline map toolkit it's a different beast.  That's what CartoType is.

Cost & Availability

CartoType GL is, logically enough, being sold through the CartoType company.  They are the ultimate source of details on this effort.  Contact them for a license.

We hope to get the iOS version out in April.  Android should follow shortly after that.

Boring Technical Details

WhirlyGlobe-Maply is the open source toolkit we (mousebird consulting inc) make and it has two levels.  The core is a C++ rendering and object management engine.  Above that is an Objective-C interface on iOS and a Java interface on Android.

CartoType GL uses the core C++ renderer directly.  Rather than expose high level WG-Maply functionality, we interface with the GL renderer.  This makes CartoType GL a much simpler map centered toolkit.

WhirlyGlobe-Maply is and will continue to be open source.  This arrangement is just like one of the commercial apps that we work on.  It's even contributing functionality back, making the toolkit better for everyone.

Wednesday, March 2, 2016

WhirlyGlobe-Maply 2.4

Version 2.4 is officially out!  It's been a while in the making.  Too long, really.

Version 2.4 is pretty.

For a list of features, go check out the list of features blog post.

If you're building a regular 'ol app using the toolkit, just grab the master branch or use the binary version, take a look at the documentation and get started.

Coming Up Next

The Cocoapod will be updated soon.  I'll edit the post.  But honestly if you're that excited to use the pod, just use the one we have in the repo.
[Edit: We've got the Podspec for version 2.4 up and running.]

We're already working on version 2.4.1.  Have been for months, really.