Contributions and Copyright

I've had clients for WhirlyGlobe-Maply for a couple years now.  Recently I've been getting contributors.  As an open source project, that's very exciting and I'm happy to see them!

They do add a bit of complexity, though, because of the way my project works.

Who Owns The Code?

That's easy, I own the code.  Every line of WhirlyGlobe-Maply is MINE!  But don't worry, you can use it for free.

Copyright is separate from license (I can hear my lawyer frowning).  I hold the copyright on the code, but I release it under an Apache 2.0 license so you can use it.  That license is incredibly friendly to commercial interests so you can make apps, modify the source, and generally go wild while abiding by its pleasant terms.

Sometimes I put WhirlyGlobe-Maply under another license for a client.  Why?  Money, that's why.  This doesn't effect the Apache 2.0 version and you needn't worry about it.  But it does mean I must own every line of WhirlyGlobe-Maply.

This is Open Source?

It is, yes, one of the variants.  It's not GPL, it's not even one of those big techno-hippie projects we all love.  But it is a well known, well tested, well understood variety of open source.

Okay, so if I own it, can I take it away?  I cannot.  Even if I decide to start charging for WhirlyGlobe-Maply 3.0 you still have the 2.2 version released under the Apache 2.0 license.  That can never go away.  [I have no plans to do this, by the way.]

Git Pull and Contributed Source

Now we get to the point of this post.  When contributors send me a pull request, what's the copyright on it?  Legally, it's "All Rights Reserved".

This won't do for either the open source version or for my own.  It could potentially taint the apps of anyone who uses the toolkit.  I can't have that.

If you're sending me a pull request, I need you to resolve the copyright and ownership.

Copyright Options

If you've modified the source, those modifications are yours (or your employer's).  If you write something new, that source is yours (or your employer's).  It's up to you (or your employer) to assign the copyright.  I'll explain some options.

All Rights Reserved

If you reserve copyright I can't touch it.  You can do what you like with it, including putting it up on github, but I'd ask you to put an explicit copyright on it so no one gets confused.

If you modified my source code in the process, you can't change my Apache 2.0 license and the copyright is now ambiguous, in my mind at least.  It would be great if you put a notice to that effect at the top of the files you've changed.  That way no one will accidentally get into trouble.

In any case, I won't be putting any of this code in my github repo.

Retain Ownership & Assign an Open Source License

If you write brand new code, you could retain ownership for yourself, of course.  If you then put an open source license on it, I will be delighted to put it in the Contributed directory.  Please pick a really, really permissive license, if you would.

I don't own source code in the Contributed area of WhirlyGlobe-Maply.  I may maintain it, from time to time, but I make no guarantees.

If you want to modify code in Contributed, feel free.  Just respect the license that's on it.

Transfer Ownership

The last option is transferring ownership to mousebird consulting inc (e.g. me).  In this case I'll pull it back into the toolkit, slap the Apache 2.0 license on it and treat it as my own, because it now is my own.

I'm only going to ask this for small amounts of code and bug fixes.  I know, it seems stupid to ask for bug fixes, but I legally have to.

Let me emphasize: YOU DO NOT HAVE TO DO THIS.  Sending me a pull request does not imply anything.  When I ask you, you can then make your choice and I will respect it.

Working For The Man

If you're an employee or doing work for hire, you must get your employer's permission to assign ownership.  I cannot accept the assignment if I think you can't give it, so I may ask some probing questions.

I know no one does this and I'm sorry to be difficult.  Look, I've been in silicon valley a long time now and I've seen a lot of stupid go by.  Much as I love github, and I really do, there is a lot of stupid around it and copyright.  I won't have it on my project.

"But hey", you may be thinking, "don't you work for The Man as well!"  I do, random interlocutor, but I have a Lawyer and a Contract.  It's a big nasty contract that spells out what I own and what The Man owns.  I trust it works because it's killed a few potential deals.

Standard work for hire contracts or employment agreements do not look like mine.  I very much doubt you have the same rights I do unless you specifically negotiated for them.

Conclusion

If I've sent you here for a pull request, let me just say:  Thank You!  Even if I can't accept it, it's helpful for a few reasons:

• I can see what developers are interested in
• If it's a bug fix, I can see what the problem is
• It gets my stats up on github (don't laugh)
• It was probably useful in your own app

So again, thank you!

Please take a look at your copyright options and pick the one most appropriate.  Picking nothing defaults to "All Rights Reserved".  That's fine too.

Vectors, OpenStreetMap, and Maply

A few months ago I gave a talk at the OpenStreetMap conference in San Francisco.  It was all about using vector tiles for map display on iOS.

Not San Francisco

This was based on new functionality in Maply, the 2D part of WhirlyGlobe-Maply.  Since then I've improved things.

Vector Support in Maply

I've had support for vector data in WhirlyGlobe since 1.0.  With version 2.2 it's gotten much better, specifically for map display.  The important bit was the MaplyQuadPagingLayer and the MaplyPagingDelegate protocol.

It works like this.  You provide an object that supports the MaplyPagingDelegate protocol, start up a MaplyQuadPagingLayer and wait for the callbacks.  When it's time to load a tile, you load your data, convert it to a displayable form and then hand it over.  The system cleans it up for you when it's no longer needed.

Now we can treat vector data like we do image tiles.  If only we had a good data source...

OSM Vectors

A few months ago OpenStreetMap began providing vector data tiles experimentally.  Mike Migurski was the driving force behind it, and he explains it better than I.

Tasteful buildings

The short of it is, we've got vector data chopped up into individual tiles, with some basic cartographic choices made at the various levels.  The server responds much like an image tile server, but it returns GeoJSON.

That's great, but how well does it all work?

Base Map + Vector Overlay

There are two sorts of vector maps we might want to display with Maply.  The first is comprised of vector data overlaid on an image base map.

That's the easier case.  It's a subset of the full vector features, including roads, labels, and buildings.  More importantly, the polygons are fairly simple.  Which leads to the next question:  How about everything?

Pure Vector Map

An entirely vector based map is the goal, at least from a technical standpoint.  With pure vector on one end and pure image on the other, we'd like to dial in the appropriate mix for a given app.  The OSM data is there, so how well does it work?

It works well.  Quite well indeed.  That's the big innovation from a few months ago:  WhirlyGlobe-Maply 2.2 can handle the weird polygons it gets from the server.  The tessellation is working well even for complex polygons with holes.

Room For Improvement

Nothing's perfect, of course; that would be creepy.  There are a few areas that could be better, in no particular order, they are:

• Parsing GeoJSON is just slow.  It needs to be faster.
• A binary vector data cache would make things appear much faster.  For some cases it would be a big improvement.
• The vector data is coming in tiny little chunks.  This is messing with the drawing optimization.  There are ways to fix this.
• Place names are dominating the display in cities.  For a real app, I'd suggest making some cartographic choices.
• It needs more styles.  I've broken out a few of the feature types, but more is better.

Even so, this is usable as is.  Go check out the code, all open source, and give it a try.

What's Next

I build out serious toolkit functionality with serious apps.  On the WhirlyGlobe side I've had some great clients who did some really ambitious things.  There's been some interest on the Maply side, but so far nothing really big.  The toolkit is ready (more than ready) so as soon as I reel those clients in, we should have some interesting results.

In the mean time Maply vectors are pretty stable.  Go give the OSM example a look and feel free to use it in your own apps.

WhirlyGlobe-Maply Feature: Elevation

Terrain support has been a very popular request in WhirlyGlobe-Maply.  Here you go.

Works pretty well, actually.  The data set I'm using there is in Spherical Mercator to match the map tiles coming from MapBox.  There's one 32x32 sample tile for each image and the data is stored locally in an sqlite database.  More about the tools later.

Component Support

If the elevation data is stored in a sqlite database, we obviously have to get it out.  We use a MaplyElevationDatabase to do that.  That object implements the MaplyElevationSourceDelegate protocol, which you can use to query individual tiles.

You can also implement your own elevation database, perhaps one which fetches data remotely.  Just implement the MaplyElevationSourceDelegate protocol.  Don't worry, you're called in another thread so go ahead and block on the network calls.

We hand the elevation database over to a MaplyBaseViewController, which is the parent of both globe and map view controllers.  Though that object doesn't use it, it does pass it on to layers that care about elevation.  At present, that's just the MaplyQuadImageTilesLayer, which now handles all of the local and remote image tile paging.

Here's what that all looks like, from the WhirlyGlobeComponentTester app.

And all that will get you this.

How active could that volcano be?

Elevation Pyramid Generator

Getting terrain support in the toolkit wasn't actually all that difficult.  I know kind of a lot about this stuff.  No, the real problem is how to generate the data.

From past experience I knew I couldn't just throw this functionality out there and expect users to pick it up.  I needed to provide a tool.

This one's just gratuitious

That tool is elev_tile_pyramid and you can find it on github in the develop branch.  It deserves its own blog post, but for now you can look in the README file.  That explains how to compile it and how to run off the Pacific Northwest example.  You can also just download that database.

The Fine Print

It's cool and it works!  But there are some caveats.

Elevation data adds complexity.  We can no longer predict where the surface of the earth is at any given point and, even worse, it moves between levels of detail.  That means features are hard to place.

The settings panel.  Titillating!

That's the settings panel from the WhirlyGlobeComponentTester app.  The base layers are all there, but you might notice all the overlays are missing, as are the vectors.

You can do vectors, labels, markers, and all that good stuff in this mode, but they won't work well.  Markers and labels might, actually, but vectors certainly won't.  Why not?  Because topology.

We've got a different version of the surface for each level of detail, and there are 14 levels in the example.  That means whatever you stick on top of the earth needs to move around as those levels of detail swap in and out.  It's not impossible, but it does require some work.  Work that, frankly, a client can pay for.

The Conclusion

It works pretty well and it's in the develop branch on github.   Go have fun.  I'm sure more features will be forthcoming, but I'm going to see what clients want to do.

WhirlyGlobe-Maply 2.2 Beta

As promised, the WhirlyGlobe-Maply beta is up for version 2.2.  You can get it on github from the master branch.  That's the preferred location.

Gratuitous Image

If you want a binary distribution, I've got one of those up now too.  This includes the framework, now stripped of its debug info, as well as a the sample app and all dependent libraries.

Geospatial Data Display using OpenGL ES (for iOS)

A few weeks ago I gave this talk at the local NSMeetup.  That's a nice venue, primarily for iOS development, with a very strict technical bent.  Most iOS related meetups around here (San Francisco) devolve into meet markets for "technical cofounders".

Anyway, I asked to give a talk and after a bit of back and forth I made it more general than my usual WhirlyGlobe-Maply pitch.  Here it is in its full glory.

Overview

I had about an hour so I couldn't do a full OpenGL ES tutorial.  I tried to distill a few lessons from geodata display, things that you run into when you have more data than you can easily draw.  And of course, how to draw it fast.

There was code!  I honed a few examples related to my favorite problems.  Hit the links below to jump to the code examples in the video.

• Getting Started: Drawing a few boxes
• Speed: Drawing a lot of boxes
• Dynamic Geometry: Adding boxes as you go
• Spheres & Textures: How to make a globe

I tend to think of these as really big problems I throw a lot of code at to fix.  It was interesting to boil them down to their essence in a little code and show the solutions.

The Code

You can get the NSMeetup app source on github.  It has the code examples, as well as links to the code in github gists.

Cubes are like spheres, only less so.

The github gists were an experiment.  I've watched any number of talks with the speaker struggling to use Xcode in less pixels than Xcode would prefer.  And anyway, I do my talks on an iPad because my demos are on an iPad.  So why not show the code on the iPad?

Using gists and a UIWebView let me keep the flow going.  I went back and forth between code and the working example without leaving the app.  I'd definitely do that again.

More Tutorials?

I may do a few more, but just related to WhirlyGlobe-Maply.  This was very interesting, but time consuming and I'm not in the OpenGL ES training business.  If you are in that business, feel free to link to the video if you find it useful.

WhirlyGlobe-Maply 2.2

I've been working on 2.2 for a few months now, ever since 2.1 was released.  There's a lot of new stuff in this version.  So much that I've been actively delaying writing it up.

Let's get on with it!  Here's what's in WhirlyGlobe-Maply 2.2.

Base Map Display

The quad paging layers changed radically in this version.  I talk a bit about the underlying technology elsewhere, suffice it to say that they draw fast.  Real fast.

In addition to that (which is big), I've also exposed a lot more functionality at the Component level.  Now you can do tricky stuff like this.

That's three base maps stacked on top of each other:

• My variant of Geography Class, paged locally.
• USGS Ortho photos coming from their WMS server
• Transparent weather data from the OpenWeatherMap server

Each of those layers is independent, with its own coordinate system, level range and protocol.  You can now stack these things up to your heart's content and control their order.  And it's fast.

In addition to my own low level layers, you can also make your own.   Just create a MaplyQuadEarthTilesLayer and hand it an object implementing the MaplyTileSource protocol.  Have that object return an image for a given tile and off you go.  It even uses dispatch queues.

Vector Paging Layer

Though vector display hasn't changed all that much in 2.2, I have added high level support for paging your own data sets.  We organize base image maps as tiles and we can do the same with vector tiles.

OpenStreetMap vector data

In this example we're paging vector data from the US OpenStreetMap server.  That's an experimental setup where they're serving up GeoJSON tiles, sort of like what we used for images.

You don't have to have GeoJSON or anything specific.  Just create a MaplyQuadPagingLayer and fill in the MaplyPagingDelegate protocol.  You create the visible objects associated with your tile and the Component handles everything else, including cleanup.  It's also multi-threaded.

ZBuffer Control / Draw Priority

This one's a bit low level, but really useful.  You can now control which features respect the zBuffer (or ignore it) and which write to it.  That's incredibly useful for features you want to stick right to the globe.

  

From the picture, here's how the standard defaults play out.

• Geography class is the base layer at priority 100.  It writes to the zbuffer, but doesn't read.
• The USGS Ortho layer is next around priority 101.  Also writes to the zbuffer, but doesn't read.
• Next, we draw the stickers (priority 30,000) which neither read from nor write to the zbuffer.
• Grid lines (in blue) are drawn next (priority 50,000).  They ignore the zbuffer entirely and have a tricky little shader program to deal with the lines behind the globe.
• Lastly, we throw on the shapes, including those spheres and the great circles (with altitude) at about priority 80,000.  They read from the z buffer, but don't write to it.

There's also a little book keeping for skirts.  What are skirts?  Do you see any obvious lines between levels in the base map?  That's what skirts do.

That's all without the dreaded drawOffset; drawPriority now rules the day.

Anyway, that's all with the defaults.  Odds are you'll never need to set any of this yourself, but it's there if you do.

Text Fonts & Layout

We've now got a font glyph manager.  It's not too tricky, but it speeds things up and (potentially) saves a lot of memory.

We used to render each label in Quartz, which worked great, but was kind of slow and a memory hog.  Now when you ask for a label, it renders the appropriate glyphs, saves them to a giant texture atlas and makes up the polygons for your string.

When you delete the string, it may delete the glyphs as well.  So feel free to use weird fonts with impunity.

Labels now support a wider range of features, including an outline color and size.  That's nice for maps with cluttered backgrounds.

The layout engine is a bit more sophisticated as well.  It can handle rotation and does a better job with justification.

Vectors & Lofted Polygons

The toolkit can now tesselate large, weird polygons.  Why?  Because sometimes you get large, weird polygons.

Oh goodie, lakes.

This is useful in a couple of places.  First, the kMaplyFilled attribute works a hole (pun intended) lot better on vectors.  Second, the lofted poly layer is a lot faster.  Tapping Russia is now interactive.

Ha ha.  Red.  I get it.

Thread Control & Active Objects

Now for the obscure stuff.  You can add things on the main thread!  Gasp!  Wait.  What?

That little sphere is moving.  Just keep staring.

WhirlyGlobe-Maply likes its threads, it uses a bunch of them.  We try to do everything on either the layer thread or in custom dispatch queues.

Well great, but what if you want to change something RIGHT NOW.  You know, if you're editing it.  Or maybe animating it.  Now you can.

All the usual add calls (i.e. addScreenLabels:) take a mode: parameter.  Right now the mode is either MaplyThreadAny (the old way) or MaplyThreadCurrent.  For the latter we do the deed right now on the thread we're currently in.

There's also a new MaplyActiveObject, which is wrapper around a callback you get right before the next frame draws.  This is how you animate if you're so inclined.  Use it wisely.

Screen Space Calculation & Tilt

The toolkit spends a lot of its time trying to figure out what to load and when to load it.  It does this by deciding how big things are on the screen.  That works fine if you're staring straight down, it gets trickier if you're not.

Crazy!  Who knew?  Well I knew, I was just avoiding it.

And now I'm not avoiding it.  We can display the sampled curved surface of a globe when looking outward.  That was fun.

You can mess with the tilt in a WhirlyGlobeViewController.  This can get tricky, so check out the setTiltMinHeight:maxHeight:minTilt:maxTilt: method.  That'll muck with the tilt as you get close to the ground in a somewhat intuitive fashion.

Maply (2D) Changes

Support for the 2D/3D map mode is fleshed out a bit more.  I think you can actually use it for apps now, because... well... I have.

Like a Globe, only Flat.

There's a true 2D map mode now, where you can turn off lighting and even tie the map to a UIScrollView.  It's a bit... tricky yet.

The gestures could use a bit of work and I'm getting a better idea of what users might want (hint: looks like MapKit).  There will probably more of this in 2.3.

Component vs. WhirlyGlobe-Maply API

I used to stress the difference between the low level WhirlyGlobe-Maply API and the high level Component.  The API uses a combination of C++, STL, Boost, and apparently MADNESS.  I'm just pushing the WhirlyGlobe-Maply Component these days.

In fact, most of my clients are using the Component, so all the good stuff gets exposed there.  Point being, just use the Component.

Other Stuff

There's a lot of new functionality in WhirlyGlobe-Maply 2.2.  Did I mention custom shaders?  I did not.  The low level rework that'll make Android porting easier?  Didn't really go into that.  How about elevation data?  Yeah, it's there, but you need your own data.  Don't have that... let's wait until 2.3.

And, oh yeah, the WhirlyGlobe-Maply Component Tester app has been heavily reworked to show all this off.  Check that out for examples.

The Future

I'm putting 2.2 into Beta shortly.  It'll go active on the main branch on github, then we'll update the pod specs, the documentation, and the binary distribution.

After 2.2 I'm looking more toward map related features.  The globe is fun, but maps (vector and image) are a bigger user base.  Rather than crazy high end stuff (which my clients like), I need to put some time into making things easier to use.  Annotations are an obvious problem, as is area based update.

Of course, clients pay the bills, so expect more high end craziness as well.

Geospatial data display using OpenGL ES

I'm doing a talk tomorrow night on geospatial display and OpenGL ES for iOS.  Down in SOMA in good old San Francisco.

There's a few examples, a bit of code, and a whole lot of slides.  Should be fun.

Along with the talk is a tutorial illustrating some of the problems you run into doing geospatial data display.  You can find the code on github.