Monday, September 10, 2012

Knight Foundation Proposal

I recently put together a proposal for the Knight News Challenge on mobile.  I'll detail it here for any interested parties.  It's a good sketch of where I'm thinking of taking WhirlyGlobe and its sister toolkit.

I've got a link in the proposal pointing here, so I may fill it in with a few more pictures and details.  Check back.

Some Background

I'm a full time iOS consultant.  I build iPhone and iPad apps for other people.  I write code, sometimes I manage, and I plan.  Consulting is nice, but I like a little focus in my work, so I developed WhirlyGlobe on my own dime.  That's an interactive 3D toolkit for iPad and iPhone that lets you add a globe to your app.  You can overlay all sorts of interesting data, interact with it and so on.

After the first few releases I realized it was too complicated, so I made it much, much simpler.  Check out the video for the WhirlyGlobe Component, which is the easy to use version.

Introducing Maply

The next step is a 2D(ish) map based on the WhirlyGlobe technology.  I'm developing it now and a preliminary version should be out within a month.  I expect it to be more popular than WhirlyGlobe as it has a broader appeal.

Like WhirlyGlobe, Maply is based on OpenGL ES which lets me do some interesting things.  Also like WhirlyGlobe it will be focused on performance and fidelity.  In other words it'll be fast and pretty, same as its sibling toolkit.

The Problem

I have a lot of WhirlyGlobe users now, most of them are fairly inexperienced with Geographic Information Systems (GIS) or Cartographic data processing.  They can handle the imagery, particularly if they point WhirlyGlobe at an existing data source, but they're lost on vector data.

There are many free and open sources of vector data.  The US government is a wonderful source of data, as are OpenStreetMap and Natural Earth for more focused data sets.  Processing it all is confusing, though, and my users have a difficult time.  Most just grab a simple static data set for vectors if they do anything at all.

Getting ahold of good data is just the first step, alas.  It needs to be processed into smaller units which can be paged at appropriate levels for the user to see.  Consistent size is important for both display and fetching over the network.  A user needs to process their data set, if it's large, into a form appropriate for the use.

Now there are some great proprietary tool chains for doing just this.  However, they're big complicated tools and I'd like to see more in the free domain.

The Goal

I want to make that process easier.  On the image tile side I'm reasonably happy with what's there.  On the vector side, it's much harder in the open source realm.

The goal is to first, add consistent support to WhirlyGlobe and Maply for fetching large vector data sets from a remote server.  Now I've done similar things in the past, but they're all very specific to a particular client.  This would be open and applicable to a wider range of data sets.  This is where most of the software development would occur and I understand the problems well enough.

Then the goal is to build out an open tool chain for processing large vector data sets into a compatible form.  Much of this would be identifying the tools, trying things out and then documenting how we did it.  This problem has been solved innumerable times in a proprietary way.  The goal here is to make it simple and make it open.


To that end, I'd like to use open source tools like QGIS for editing, pull from OpenStreetMap where appropriate and publish out to services like CartoDB.  None of that is particularly hard from an engineering perspective, but it's tricky for users.  Documenting that process is going to be key.

Lastly, I'd want to process a few example data sets and make them available for reuse.  OpenStreetMap data is an obvious target for this and I've contemplated a couple of small projects there.  One might be creating a clean base map without labels.  A second would be processing the building outlines into a vector data set in this new form.