Wednesday, September 17, 2014

Cocoapods Support

If you're not aware of Cocoapods for iOS, you should check it out.  It's a dependency management system for source code and it's gotten quite popular.

We now have an up-to-date pod spec for WhirlyGlobe-Maply 2.3.

WhirlyGlobe-Maply Pod


All credit goes to Juan Collas who put together the latest pod (and the old one).  I contributed... absolutely nothing.  Now that's what I call open source!

The pod is named WhirlyGlobe, but Maply is in there too.  If you go looking for it on cocoapods.org you'll see this.


Using the Pod


Pods are pretty easy to use, which is kind of the point.  If you haven't installed the pod gem, go follow their instructions.

I've added a github repo that has the standard test app set up with a Podfile.  To build this, do the following:

  • git clone https://github.com/mousebird/WG-Maply-Pod-Test.git
  • cd WG-Maply-Pod-Test
  • pod install
  • Load WG-Maply-Pod-Test.xcworkspace in Xcode (not the xcodeproj)
  • Build and run

It's that easy.  I'm kind of amazed.

Setting Up Your Own


If you want to set up your own project, include the following in your podfile.

pod 'WhirlyGlobe', '2.3'
pod 'WhirlyGlobeResources'

Consult the Cocoapods site for all the rest of it, or check out the podfile in the example.  If you don't need the resources (e.g. source data), leave out the second one.

That's it.   Thanks again Juan, this is cool!

Monday, September 8, 2014

FOSS4G - Portland, OR

The FOSS4G conference is coming up this week in Portland.  I'll be there.


I've got a booth in the exhibitor section.  You can find it by... well it's a card table and there aren't that many exhibitors.  Drop on by if you'd like to talk geospatial and mobile.  I'll have a bunch of devices to check out demos on.  And a big monitor.  That's my thing.

And I'm giving away.... business cards.  Yay!

Wednesday, September 3, 2014

Tile Loading

WhirlyGlobe-Maply 2.3 saw some significant improvements in tile loading.  Most of this was focused on maps. Let's take a look at how it used to work and what's available now.

Images and tile numbers

Loading From the Top: Globe


WhirlyGlobe-Maply is called that because the globe came first.  It's often used as a whole globe with the user starting zoomed way out.

Mapbox Terrain Style

The image tile loading worked like this.

  • Start at the the lowest resolution tile (e.g. [0: 0,0])
  • While there are tiles to load
    • Load the current most important tile.
    • Did it fail?  Okay we're done here
    • Did it succeed?  Great, add the four children to the list.

Who doesn't love pseudo-code?  Let's just see what that looks like.

By the numbers

That approach had some advantages.  Tile sources can be sparse and it's a little resistant to network failure.  Most important (to me), there are never any holes in the globe.  There's always something to look at, even if it's blurry.

Loading From the Top: Map


One secret of WhirlyGlobe-Maply is that the globe and map share most of their code.  So when I got the map working, I used the same tile loading scheme.

Loading from the Top with Mapbox imagery


That worked okay, but if you start out zoomed in to the map you're waiting a long time to see the right tiles.  There's a whole lot of blurry for far too long.  It gets worse the farther you're zoomed in.  Here's how that looks with tile numbers.

Loading from the top by the numbers

The flashy colors at the beginning are low res tiles we can't see the numbers for.  I could probably argue there are cases where that works well.  But not for a map.

Loading the Current Zoom Level


The obvious thing to do for a map is to load the zoom level we need for the area we're looking at.  If it's a flat (2D) map, that's actually pretty easy.

Single level load with imagery


Simple, but I don't like it.  You're looking at too much blank screen.  For a retina display and a remote tile source this is irritating.

Still, there are cases where this may be appropriate and it's easy enough to activate.  When you set up your MaplyQuadImageTilesLayer set useTargetZoomLevel and singleLevelLoading to YES.

Single level load by the numbers


And again, the tile loading with numbers.  Fine for some cases, like a weather overlay.  But we can do better.

Loading Multiple Zoom Levels


Focusing on a single zoom level is great for performance, but bad for display.  I'd rather have a hybrid that loads a few low res tiles and then fills in the rest.  Like this.

Multi-level load with Mapbox imagery

That's nice.  We've got something reasonable to look at almost immediately, then we get some detail.  When the highest resolution comes in we probably didn't realize it wasn't already sharp.

Multi-level load by the numbers


Pretty good, but the levels you want to load may depend on your tile source.  Luckily, it's configurable.  When you create your MaplyQuadImageTilesLayer set useTargetZoomLevel and singleLevelLoading to YES and then assign an array of numbers to multiLevelLoads.

For example, try @[-4,-2] in multiLevelLoads.  If the user should be looking at level 22 data, then the layer will start out loading level 18, then work on level 20 and then fill in level 22.

Summary


The old tile loading logic could feel out a sparse tile source efficiently.  The new tile logic can do it too, but wow was that complicated.  And perhaps less efficient.  If you've got a normal tile source, it shouldn't be a problem.

I like the multi level loading approach.  But it does cost more memory and rendering time than the single level approach.  Use it thoughtfully and test it in your app.

This is all available in WhirlyGlobe-Maply 2.3.



Tuesday, September 2, 2014

A Cure for the Jiggles

This one's a problem for the big apps, the ones moving gigabytes of imagery and vectors.  You know, the people who pay me.  It looks like this.

I'm... not feeling so good.

What's happening is a floating point precision problem.   If you want to represent the globe all the way from orbit down to nose picking distance, 32 bits of floating point won't do the trick.  Let's look closer.

The Problem


If single precision floating point doesn't crack it for you, why not use 64 bit doubles?  Sure, if OpenGL ES supported them, though it would be kind of a waste.  At some point you have to convert your data to 32 bit floats (or even fixed point, you weirdo).  There's no escape, but there are cheats.

At first blush it's your image tiles and vectors causing the problem.  That's part of it, but don't forget your motion model.  The quaternion controlling your motion can't really run on 32 bit floating point either.

Here's what I did to fix it all.

The Solutions


Let's start with the motion model, what's turning your gestures into movement.  Zoom in too far and things start twitching.  The problem here is the quaternion logic and the matrix math around it.  In WhirlyGlobe-Maply 2.2 it was 32 bit floats.  The fix was to move all this to 64 bit doubles.

Cool picture courtesy: Indiana University



That helped a lot.  Now the motion is nice and smooth.  But things are still jumpy.  It's the data.  As those 32 bit floats get pushed through the OpenGL ES pipeline, they jump around a little.  It's a little different each frame and it adds up to visual insanity.

There's an easy solution to this one, so let's start with the vectors.  For each batch of vectors we define a local origin.  We pass that origin to the renderer as a double precision matrix.  The renderer multiplies the local matrix by the global model/view/projection matrix and then converts down to single precision.  Since the coordinates are local to that origin, nothing gets too big in 32 bit floating point.

Example tiles with spatial origins

You might think we could do the same thing for image tiles.  We almost can, but we don't represent image tiles individually, we batch them together for speed.  Raw, glorious rendering speed.  The solution is similar to vectors, but with a twist.

For a given tile, we look at its size and its center.  We compare it to the big drawables we've already created, looking for a nearby center.  If it's not too far away, we reuse that big drawable, otherwise we create a new one.

Example origins
In that example there may be 100 different tiles of varying resolutions sorted into origin A and origin B drawables simply because the center is close enough.


Conclusion


The center sharing solution works pretty well in practice.  We don't lose too much performance and the jigglies are gone.

50% fewer seizures!

If you're doing closeup work with vectors, be sure to set kMaplyCenter to YES.  The vector tile module does it and the image tiles do their thing by default.  Twitching problems in screen space objects and overlaid views were fixed with similar trickery.

You'll find this all in WhirlyGlobe-Maply 2.3.

Thursday, August 21, 2014

WhirlyGlobe-Maply 2.3

I'm pleased to announce the official release of WhirlyGlobe-Maply 2.3.

Yup, it can do both of those.

The big new feature for 2.3 is maps.  Vector maps.

Where to Get WhirlyGlobe-Maply 2.3


Over at github is the canonical place.  The master branch is now on 2.3.  Compiling from source is a bit slow, but then you've got the source.  Be sure to read the README.  Seriously, read it.

You can also download the binary distribution right there.  Less to set up, faster to compile, but there's a lingering sense that you're not cool.  I don't judge.

Documentation


You can browse the reference documentation right here.

It needs more.... more.  I have a Getting Started guide under development and a rework of the WhirlyGlobe-Maply site to go along with it.  When that's done I'll make more noise about the launch.

Next Up


A few months ago I'd planned to segue into WhirlyGlobe-Maply 3.0, which is the Android/iOS version.  Instead, clients have been paying me to add some big new iOS features.  Damn you paying clients!  So we get at least one more 2.x version.

The develop branch is moving to WhirlyGlobe-Maply 2.4.  I'd stick with the master branch for now.

Thursday, July 31, 2014

Open Academy Internships - Proposal

I've been invited to participate in the Open Academy through LocationTech.  Or to submit a proposal, anyway.  This would mean interns working on WhirlyGlobe-Maply for class credit.  I'm organizing my proposals here along with a bit more detail than I can provide elsewhere.  If you, my users, have an additional ones, add them in the comments.

Requirements


If you're going to work on WhirlyGlobe-Maply, you need an Apple Mac running OS X.  And you will at least need Xcode 5.1 or better.  You can run on the simulator without an account, but if you want to run on hardware (and you should), you may need an Apple developer account.  Please discuss this with your professor first.

Though WG-Maply does have an Android version, it's not yet mature and I'm not willing to support students using it yet.  Perhaps next year.

Possible projects


Here are a few projects that I think would be interesting for a computer science student with some exposure to mobile development.  Most of these focus on open standards and some on open data.

These projects will go into the contributed area of the WG-Maply project.  None of them require internal changes to the toolkit (a good thing) so they can sit outside the main library.

Keyhole Markup Language (KML)


KML has an interesting history.  It's basically the internal format for Google Earth, but due to its ubiquity it's turned into something of an interchange standard.  It's ill suited for this, but you'll find that happens a lot.  Might as well roll with it.

This project would be adding KML support to WG-Maply.  There are several parts to it.

  • Read KML - There's an API, so that's not too bad.  It's C++, though, so you need to know C++.
  • Interpret KML - This is where the C++ comes into play.  The API was written with Design Patterns very much in mind.  It'll be less annoying if you've had some exposure to that (but it will still be annoying).
  • Convert KML to WG-Maply concepts - The last bit isn't too bad.  Convert their markers into WG-Maply markers and so on.  The concepts don't map 1:1 which is what makes it interesting.
There are abundant KML examples out there for testing.  We'll find you a few.

I'd recommend starting with the static KML data.  If you're feeling ambitious, you can move on to the dynamic stuff.


Web Feature Service Support


WFS is one the OpenGeospatial Consortium (OGC) standards.  It's for expressing features, by which they mean points, lines, polygons, that sort of thing.  This is basically map data and can be overlaid on a map or globe in WG-Maply.  Or both.  Up to you, really.  The project is to display WFS in WG-Maply.

Using WFS requires a server.  I'll find you one or set up a GeoServer myself, if need be.  Once you've got an end point to talk to, there are three parts to this:

  • Form the query for what to fetch.  This will be XML, but there are XML libraries for Objective C that work pretty well.  This will take longer than you think it ought to.
  • Parse the Geospatial Markup Language (GML) that comes back.  I don't know of a good Objective-C parser for GML, but we'll look.  In any case it's XML underneath, so you may need to interpret that.
  • Convert the GML concepts into WG-Maply data structures.
The goal here would be to talk to a GeoServer and display the features it sends back for a certain area.  For bonus points, hook this into the MaplyQuadPagingLayer.  We use that for fetching data as we need it, rather than all at once.

MapCSS Support


Turning vector features into a readable map is an ongoing problem.  There are lots of ways to do it, but most of them are at least a little complex.  And for good reason, the problem is complex.  But it would be nice to have some simpler alternatives.

Right now the most common way users style their maps in WG-Maply is with Mapnik XML.  They usually produce this with CartoCSS in TileMill.  There are other approaches supported by other tools, such as MapCSS.

The project is to implement a decent MapCSS parser in Objective-C (or at least a wrapper).  Test it by displaying MapCSS styled maps in WG-Maply.  This would pair nicely with the GeoPackage support below, but I wouldn't recommend taking on both.

TopoJSON Support


There are a couple of newish standards out there that went around (e.g. ignored) the traditional geospatial standards process.  This makes them a little wonky in places, but also a lot easier to implement.  The first one is GeoJSON, which is well supported in WG-Maply.  The second is TopoJSON, which is not.

The goal of this project is to implement TopoJSON support.  This is not as complex as the other projects, but is pretty valuable.  Lots of people use these standards.

You'll need to parse the JSON itself, which is pretty easy on iOS.  Then you'll need to reform the topology and turn it into MaplyVectorObjects to display in WG-Maply.

If you're feeling ambitious, write an editor.  Since TopoJSON preserves topology, moving individual points around can be done without destroying the larger structures they're part of.  If you do this, you'll also want to write the TopoJSON back out.

GeoPackage Support


Let's get back to the official standards.  GeoPackage is a newer one from OGC that's meant to be used on mobile devices.  I don't run into it much yet, so let's see if we can help it along.

GeoPackage is simpler than it sounds.  It's based on SQLite which we use constantly on mobile devices and is well supported.  GeoPackage encodes features (e.g. points, lines, polygons and so forth) and provides for spatial searches.  That last bit is important on mobile devices, as we can only display so much at once.

The project is to support GeoPackage in WG-Maply.  I'd recommend doing this in two steps:
  • Support the vector features it encodes.  You can use the Spatialite toolkit to read them, apparently.
  • Make spatial queries in conjunction with the MaplyQuadPagingLayer.  That's how we load only the data we need to keep from overloading the display.
There are GeoPackage samples out there and tools for looking at the contents.  The goal would be to support as much as possible, with a minimum of getting the traditional point, linear, areal features displayed.  

This project would pair nicely with the MapCSS project or you could try styling the data with Mapnik XML.  Consider that the advanced part.

NASA Geospatial Imagery Browse Service


In an effort to make some of their data sets more accessible NASA has come up with GIBS.  It's a service that converts a few timely data sets into tiled data sets, covering much of the earth.  They provide simplified forms, such as Google's (and OpenStreetMap's) TMS.  That makes it easier for dumber systems (like mine) to display the data.

Actually displaying one of their data sets is too easy.  We already do that with the MaplyQuadImageTileLayer.  This project is to make a browser for the GIBS data sets.  That is, to show the user what is available and let them decide what to display.

In addition to the simple TMS end points, GIBS also provides Web Map Service support.  This is a more complex standard (and how) with a lot more flexibility.  It can tell you a lot more about what's actually available.

The project is to talk to the NASA GIBS service and build a browser for it using WG-Maply.  This would consist of the following steps.
  • Talk to the GIBS service using WMS.  You'll need to make a capabilities request and then parse the results.  There's an existing WMS parser in WG-Maply you can start with.  NASA has added a lot more specific information to their WMS return so you'll need to interpret it.  This will take longer than you think.
  • Build a display for what you get back.  Sort the results into base and overlay layers and then let the user select them.  A couple of UITableViews will work well for this.
This project will look really cool, but it's balanced by the complexity of WMS.

Wrapup


Those are my proposed projects.  Some are trickier than others, but they all require Mac OSX and some iOS hardware.  If any of my users have other project ideas, feel free to chime in.  Remember that the purpose of internships like these is to teach, not to try and get free labor, so write accordingly.

If you're a student considering one of these projects, feel free to ask questions.  I'll be happy to provide more details.

Wednesday, July 9, 2014

WhirlyGlobe-Maply 2.3 Beta 2

It's finally time to release WhirlyGlobe-Maply 2.3 into the wild.  I've been using it for some time and, sure, you could have always switched to the develop branch, but hey, release!

Courtesy Gravitystorm

Thanks go out to the contributors on this version.  This one's seen the biggest set of contributions from people other than me.

Let's get to it.

Where to Get WhirlyGlobe-Maply 2.3


For the source distribution, head over to github, clone the repository and switch to the develop_2_3 branch.  Be sure to update your submodules.

You can also download the binary distribution.

Documentation for 2.3 can be found here.

What's new In WhirlyGlobe-Maply 2.3


It's a huge list of new features.  I'll be working my way through blog posts for the next several weeks.  For now, here's the short version grouped for my own amusement.

Housekeeping


Here's the stuff that doesn't fit well into another theme.
  • 64 bit device support.
  • Xcode 5.1 is supported.
  • OpenGL ES 3.0 is supported.
  • Tessellation bugs are now fixed.
  • EAC/ETC2 texture compression was added.
  • High resolution jiggliness was fixed in images, vectors, annotations and screen objects
  • Resources submodule is smaller
  • Background CPU usage is much lower for paging layers.
  • Screen shapshots are now possible
  • MaplyTexture was added for explicit texture management
  • Animation is available through the startChanges/endChanges methods.

Maps / Maply


How about features that are useful in flat maps?  This version saw a lot of improvements to the Maply side of things.  It's quite the credible map toolkit now. 
  • Wrapping across the date line, for data display, selection, gestures, and overlay.
  • Vector tile support
  • Mapnik vector tile support
  • New gestures for 2D
  • Screen importance logic is short circuited (e.g. faster) for 2D

Gestures & Feedback


This version saw a host of new gestures, particularly for 2D and improvements in the feedback methods for delegates.
  • Heading control
  • North up panning in the globe
  • Double tap, two finger tap, tap hold and drag.
  • Location animation control via delegate.
  • Feedback with the didStartMoving/didStopMoving/willStoopMoving methods
  • Screen to geo and geo to screen calculations
  • Viewing bounds calculations for setting height to show a set of features
  • MaplyAnnotation added.
    • These are nice annotations based on the SMCalloutView library

Quadtree Paging Layers


The paging layers, both for general features and for imagery saw a ton of changes.  These are the things that handle vector tile and image based paging.

  • Single and multi-level loading.
    • This means you don't have to always load from level 0
  • Animation in quad image layer and the multiplex loader.  Think weather.
  • Changing color, enabling/disabling, reset/reload, changing tile sources.
  • Switched to AFNetworking, handles lots of requests better.
  • Broke the remote source from the tile info.
  • WMS improvements
  • Background overhead is much lower.

Wrapup


This was a big release.  Hard to believe I'm also working on 3.0 for Android.  At some point those will be merged, but there's no real hurry for the iOS users.

I'd like to do more documentation for this version.  I've got someone working on getting started examples for a globe and map.  We'll see if those help.

Anyway, 2.3 is pretty well tested.  I'll put it out there for a week or so to see what turns up and then make it official.

Next up, more blog posts describing what's in it.