Maply - First Look

I've been busily working on an a sister toolkit to WhirlyGlobe called Maply.  It's a flat map based on the same technology.

It's working well enough to show.  So with minimal fanfare, here it is.

That video is recorded live off an iPad using AirPlay.  It results in a very clear picture, but it's not doing justice to the interactivity and you can't see where I tap.  Still, it gets the point across.

Features

The video covers the details pretty well, but here's a quick list of features Maply supports.

• Image tile sets - Maply can page image data from a local repository (e.g. MapBox Tiles) or from a remote one (e.g. OpenStreetMap).
• Vectors - A developer can overlay their own data set, such as the country one shown above.
• Markers - Marker images can be overlaid in both screen space and 3D.
• Labels - Same for label text.
• Selection - All vectors, markers, and labels are selectable.  Developers can make of that what they will.
• UIView overlay - Maply can associate a UIView with a geolocation and track it around accordingly.  This is nice for selection boxes or something custom enough to require a UIView.

Technical Details

I imply that Maply is separate toolkit from WhirlyGlobe.  In fact, it's not.  There are a few unique objects for each, but the vast majority of code is shared.  Far more than I was expecting, actually.

It's all checked in to github in the develop branch, so if you're extremely daring... though I wouldn't recommend it.  I'll do a beta release for the Maply Component in the next few weeks.  Wait for that.

Future Stuff

Maply is likely to be the far more popular option so I'll surreptitiously rebrand a lot of the low level Whirly objects as Maply.  WhirlyGlobe will remain, though most of it can now handle both WhirlyGlobe and Maply display systems.

With the early releases of WhirlyGlobe I focused on the toolkit and only later came out with the WhirlyGlobe Component.  This time I'm reversing that.  The focus is on the Maply Component with the low level API being secondary.

What's next is tuning.  The gesture recognizers could use some work, I haven't even hooked up the most interesting feature (It's 3D, you don't need to look straight down), and there are always a thousand little issues that turn up when you use it in a real app.

I'd like a real client project to drive that and "real" means money.  If you have one, let me know.  Maply is ready to to be the custom map engine for your app.

How low can WhirlyGlobe go?

Since I've started demoing the OpenStreetMap support I've gotten this question a lot:  Just how far down can WhirlyGlobe go?

You know, for driving directions between London and San Diego.

It can go down pretty far.  Yup.

The gesture recognizers get goofy down this far, honestly.  The whole "spinning" thing is kind of stupid and the momentum calculations are kind of bogus.

If this kind of thing becomes common, let me know and I'll make the gestures a little more intelligent. Or you could always, ya know, pay me.

WhirlyGlobe - The App

The WhirlyGlobe App update has been approved on the app store.  Now the money'll start rolling in.

Oh wait, it's free.  Damn.

Anyway, it's up there.  It's just the WhirlyGlobeApp example in the source tree.  You can compile it for yourself if you like and now you can download it.  Again.  How exciting.

Knight Foundation Proposal

I recently put together a proposal for the Knight News Challenge on mobile.  I'll detail it here for any interested parties.  It's a good sketch of where I'm thinking of taking WhirlyGlobe and its sister toolkit.

I've got a link in the proposal pointing here, so I may fill it in with a few more pictures and details.  Check back.

Some Background

I'm a full time iOS consultant.  I build iPhone and iPad apps for other people.  I write code, sometimes I manage, and I plan.  Consulting is nice, but I like a little focus in my work, so I developed WhirlyGlobe on my own dime.  That's an interactive 3D toolkit for iPad and iPhone that lets you add a globe to your app.  You can overlay all sorts of interesting data, interact with it and so on.

After the first few releases I realized it was too complicated, so I made it much, much simpler.  Check out the video for the WhirlyGlobe Component, which is the easy to use version.

Introducing Maply

The next step is a 2D(ish) map based on the WhirlyGlobe technology.  I'm developing it now and a preliminary version should be out within a month.  I expect it to be more popular than WhirlyGlobe as it has a broader appeal.

Like WhirlyGlobe, Maply is based on OpenGL ES which lets me do some interesting things.  Also like WhirlyGlobe it will be focused on performance and fidelity.  In other words it'll be fast and pretty, same as its sibling toolkit.

The Problem

I have a lot of WhirlyGlobe users now, most of them are fairly inexperienced with Geographic Information Systems (GIS) or Cartographic data processing.  They can handle the imagery, particularly if they point WhirlyGlobe at an existing data source, but they're lost on vector data.

There are many free and open sources of vector data.  The US government is a wonderful source of data, as are OpenStreetMap and Natural Earth for more focused data sets.  Processing it all is confusing, though, and my users have a difficult time.  Most just grab a simple static data set for vectors if they do anything at all.

Getting ahold of good data is just the first step, alas.  It needs to be processed into smaller units which can be paged at appropriate levels for the user to see.  Consistent size is important for both display and fetching over the network.  A user needs to process their data set, if it's large, into a form appropriate for the use.

Now there are some great proprietary tool chains for doing just this.  However, they're big complicated tools and I'd like to see more in the free domain.

The Goal

I want to make that process easier.  On the image tile side I'm reasonably happy with what's there.  On the vector side, it's much harder in the open source realm.

The goal is to first, add consistent support to WhirlyGlobe and Maply for fetching large vector data sets from a remote server.  Now I've done similar things in the past, but they're all very specific to a particular client.  This would be open and applicable to a wider range of data sets.  This is where most of the software development would occur and I understand the problems well enough.

Then the goal is to build out an open tool chain for processing large vector data sets into a compatible form.  Much of this would be identifying the tools, trying things out and then documenting how we did it.  This problem has been solved innumerable times in a proprietary way.  The goal here is to make it simple and make it open.


To that end, I'd like to use open source tools like QGIS for editing, pull from OpenStreetMap where appropriate and publish out to services like CartoDB.  None of that is particularly hard from an engineering perspective, but it's tricky for users.  Documenting that process is going to be key.

Lastly, I'd want to process a few example data sets and make them available for reuse.  OpenStreetMap data is an obvious target for this and I've contemplated a couple of small projects there.  One might be creating a clean base map without labels.  A second would be processing the building outlines into a vector data set in this new form.

WhirlyGlobe 2.0: New Features

The WhirlyGlobe API is now officially on version 2.0.  Looking back, I'm impressed with what got done since 1.2.  Yay me.

The WhirlyGlobe Component is what I'm pushing for new users.  It's much easier to use and I'm liking the early feedback.  There's still a complex toolkit underneath it and that's what this is.

You can get WhirlyGlobe 2.0 from github.

Of iPad Melting & Sales Offices

I have two main sets of users:  The folks who want to toss a few markers on a spinny globe (sales offices) and the hard core map display folks (iPad melting).  The Component is for the first group, but WhirlyGlobe 2.0 is for the second.

A lot of 2.0 features were developed for specific clients.  They're just kind enough to let me release it all for free!  Ha, as if.  I have a big complicated contract and a lawyer with a specialty in intellectual property.

Anyway, what clients want can be summed up like so:  MORE!  BIGGER!  So that's what 2.0 is about.  I'll break the updates down by popularity.

Some Interesting Random Features

You're going to get bored the further down you read.  I'm already getting bored.  So let's start with some useful features.

On Demand Rendering

WhirlyGlobe is now keeping track of whether anything moved or faded or changed.  If it didn't, then the toolkit stops rendering.  It'll start again when something moves or changes.  Really, it should have already been doing that, so let's never speak of this again.

One of these iPads is busy melting a hole in your lap.  Can you tell which?

Oh and it doesn't work with particle systems, because... well that's kind of obvious.  Just turn it off if you're using particle systems.

Vector Layer Enhancements

Vector lines can now support width and be filled.  Under the right circumstances this looks really cool.  All my cool examples are proprietary.  So... yeah.  It works, trust me.

We can now read GeoJSON for vector features.  It's slow (woo, text) but easy to understand and a bit more web-y than Shapefiles.

UIView Overlays

You can now tie a UIView to a geographic location and have WhirlyGlobe move it around for you.  It'll even disappear when it's behind the globe.

That screen label is a UIView.  Trust me.

This has to be the #1 feature that people were pissed off wasn't already there.  Use it sparingly.  If you use this for more than a handful of UIViews it will be slow.  If you then complain that it's slow I'll set up a video chat so I can stare at you disapprovingly.

Quad Display Layer and Its Ilk

I've discussed these before and there's not much more to say.  They're awesome and incredibly useful for paging image data.

Oh, OpenStreetMap you so... free.

You can also page vector data with them, but you have to set up the data sources yourself.  I realize that can be difficult and that's why I charge by the hour.

Here's a short list of things you can do with the quad display layer:

• Page from a MapBox Tiles database
• Read from a network data source in the standard(ish) Google Tile Format
• Set up data with ImageChopper and page it locally with a SphericalEarthQuadLayer
• ImageChopper has been updated to produce an image pyramid
• Page from your own data source with a TileQuadLoader
• Read data on demand from a properly organized vector data source.

Much of the good stuff has been hooked up to the WhirlyGlobe Component and you can use it there.

Screen Space Objects

You can call them 2D markers, screen space objects or billboards.  Whatever you want to call them, they're now in WhirlyGlobe.

Screen space objects are pretty simple.  Give WhirlyGlobe a location to track and it'll move the label or marker around appropriately.  They'll also disappear if they fall behind the globe.

Both labels and markers support them.  Just set the @screen attribute to YES.  They'll show up correctly in the selection layer too.

Performance Enhancements

WhirlyGlobe sits on top of a real rendering engine.  A small, simple one but a rendering engine nonetheless.  These changes all center on the rendering engine itself.  The first one is, hey, you can now measure performance in the renderer.

Culling is important, but not ideal.  Basically, you have objects in memory that you're not rendering.  That sucks, but it does happen.  The rendering engine now has a culling tree it will sort Drawables in to and then consult before rendering.

OpenGL buffer management is always great fun, if you find annoying and weird things fun.  Luckily, I do.  So now I'm managing buffer and texture IDs so we can reuse them rather than allocate them, which is slow.  So slow.

Usually you want to do Z buffering.  Sometimes you want to kick it old school with priority based rendering.  Actually, in general you don't want to do that, but I do occasionally.  So now that's an option in the renderer.

Things I Changed Just To Mess With You

Some things I changed because I was in a mood to.  The big one was the great renaming and sorting.  Files moved around, I got strict about naming and the WhirlyKit/WhirlyGlobe distinction appeared.  WhirlyMap is also in there, but that's going to change name again.  Just because.

The layer thread teardown has changed a bit.  You can now ask the layer thread to release or delete things for you.  This was actually to fix a few ordering problems, but it's still kind of weird.

Layers changed a bit in general.  There's now a shutdown call and you're expected to respect shutdown and dealloc, but you may not get one before the other.  The number of people writing their own layers is vanishingly small, so I'm not going to sweat it.

Other Things I'll Use More Later

Some things are in WhirlyGlobe 2.0 but not fully formed.  I'm using them here and there, but their full potential has yet to be realized.  So be warned.

On demand paging was a big theme of 2.0.  The quad display layers are pretty tightly coupled with what the viewer is doing, but the UpdateDisplayLayer is not.  It's a fairly casual update based on time and location.  That's useful for... you know... stuff.

Support for random coordinate systems is halfway there in WhirlyGlobe.  I've got Spherical Mercator and a few others, but it's hardly complete.  That will change when the map toolkit comes out.

WhirlyGlobe 1.2 added caching for complex features, such as lofted polygons, but I was never entirely happy with it.  Right now the GeometryLayer exists to throw random polygons into the scene.  At some point in the future, these two will meet... somehow.

Sometimes you need to update geometry every frame.  Not as much as you think you do, so be cautious.  If you really, really do there is now an Active Model construct for doing it.

A Scene Graph is a nice generic way to represent 3D geometry for rendering.  I'm not a big fan, as they can be kind of bloated.  I've added one to be compatible with a file format (not in the toolkit) that thinks in those terms, but I'd prefer if people didn't use it.

Onward To The Future!

WhirlyGlobe is about where I want it to be.  The future holds some performance enhancements (legal, I promise) and a lot more client driven features.  And at some point I'll need to tackle OpenGL ES 2.0.

The next big thing should be a flat map toolkit based on WhirlyGlobe.  Stay tuned.