Monday, October 20, 2014

Dark Sky & Progressive Loading

There's a new Dark Sky for iOS out.  If you're interested in weather apps, I suggest checking it out.

Terrible weather is pretty!

Forecast.io is one of my favorite clients (translation: They pay on time), but whenever they ask for something it's going to be hard.  This was no exception.

Tile Loading


Dark Sky has a ton of custom logic in it, not to mention all the custom data they feed it.  But large parts are using standard WhirlyGlobe-Maply.  The tile loading logic, for instance, is all in there.

Let's review.  For the globe, we start loading at level 0 and work our way down.

I will reuse this at least 5 more times.

The app does something funkier when you're zoomed in, but let's ignore that.  This approach works well for the globe, but for one problem:  Animation means frames.

So let's say you're loading 20 frames of animation to show the weather changing over the course of a day.  That means you have to load each tile 20 times.  Sure, you can do it in parallel... but still.  It takes a while.

Progressive Tile Loading


Adam at Forecast.io wanted something better.  "Couldn't we load individual frames" he said?  "Sure", I said.  "Just let me renew my Xanax prescription and we'll give it a try."

And now WG-Maply can load frames progressively.  That means it'll start at your most important frame and work its way outward.  It'll load depth first, so that first frame looks good everywhere and then work outward to the less important frames.

You can set the frame priorities yourself, change them on the fly, and get feedback on what is properly loaded.  When the user moves, we start loading at the most important frame again, then work on the others.

That Sounds... Complicated


Why yes, voice in my head that may never, ever go away, it is complicated!  It's not all that hard to use, though.  You provide your tile sources as normal and let the loader do its thing.  If you want to tell the user what's going on, there's a callback for that.

Weather apps are the most obvious users here, but anything with animation is fair game.  I'd love to see some data visualization apps using this feature.

Wednesday, October 15, 2014

Facebook Open Academy Interns

We've got interns!  Let's see what they're up to.

Drop shadows with 3D?  Yes, I went there.

Wait.  What?  Interns?


Stanford and Facebook run an internship program for open source projects.  It's course credit for doing a bit of work.

As part of LocationTech, I applied for a few slots.  Last weekend I was holed up at Facebook with five talented students.  They're working on a variety of projects with WhirlyGlobe-Maply.

In a few weeks, we might do some guest posts.  For now here's an overview of what they're up to.

Web Feature Service


WFS is used by a big segment of the open source geo community.  It's for communicating vector data from server to client.  Well, the parts we care about are.  Among other projects, it's used in Geoserver.

The project is to interface WG-Maply to WFS.  Mostly that means reading Geography Markup Language (GML).

Keyhole Markup Language


KML is Google Earth's native format.  It's evolved into an interchange standard for vector and other data.  You can encode vector data and image tile sets, sure.  You can also encode behavior and paging.  Oy, it gets complex.

This project is to read KML into WG-Maply.  Probably just the static features for now.

Marine Traffic Visualization


Did you know you can see real time ship movement data online?  Cool.  But wait, it's also collected by a network of volunteers.  If that doesn't make you want to stick an antenna on your roof then you have no soul.

Anyway, the project is to visualize this data.  Perhaps with history, perhaps in real time.  We shall see.

NASA MODIS Browser


If you haven't played with the NASA (near) real time MODIS data, then go do that.  Come back when you're done.

This project is to put together a similar interface with WhirlyGlobe.  Should make a nice example, plus, NASA imagery.

Summary


This is a pretty neat set of projects.  Indeed.  Any code will be published open source.  Some will make nifty examples, others will be merged into the WhirlyGlobe contributed collection.

Wednesday, September 17, 2014

Cocoapods Support

If you're not aware of Cocoapods for iOS, you should check it out.  It's a dependency management system for source code and it's gotten quite popular.

We now have an up-to-date pod spec for WhirlyGlobe-Maply 2.3.

WhirlyGlobe-Maply Pod


All credit goes to Juan Collas who put together the latest pod (and the old one).  I contributed... absolutely nothing.  Now that's what I call open source!

The pod is named WhirlyGlobe, but Maply is in there too.  If you go looking for it on cocoapods.org you'll see this.


Using the Pod


Pods are pretty easy to use, which is kind of the point.  If you haven't installed the pod gem, go follow their instructions.

I've added a github repo that has the standard test app set up with a Podfile.  To build this, do the following:

  • git clone https://github.com/mousebird/WG-Maply-Pod-Test.git
  • cd WG-Maply-Pod-Test
  • pod install
  • Load WG-Maply-Pod-Test.xcworkspace in Xcode (not the xcodeproj)
  • Build and run

It's that easy.  I'm kind of amazed.

Setting Up Your Own


If you want to set up your own project, include the following in your podfile.

pod 'WhirlyGlobe', '2.3'
pod 'WhirlyGlobeResources'

Consult the Cocoapods site for all the rest of it, or check out the podfile in the example.  If you don't need the resources (e.g. source data), leave out the second one.

That's it.   Thanks again Juan, this is cool!

Monday, September 8, 2014

FOSS4G - Portland, OR

The FOSS4G conference is coming up this week in Portland.  I'll be there.


I've got a booth in the exhibitor section.  You can find it by... well it's a card table and there aren't that many exhibitors.  Drop on by if you'd like to talk geospatial and mobile.  I'll have a bunch of devices to check out demos on.  And a big monitor.  That's my thing.

And I'm giving away.... business cards.  Yay!

Wednesday, September 3, 2014

Tile Loading

WhirlyGlobe-Maply 2.3 saw some significant improvements in tile loading.  Most of this was focused on maps. Let's take a look at how it used to work and what's available now.

Images and tile numbers

Loading From the Top: Globe


WhirlyGlobe-Maply is called that because the globe came first.  It's often used as a whole globe with the user starting zoomed way out.

Mapbox Terrain Style

The image tile loading worked like this.

  • Start at the the lowest resolution tile (e.g. [0: 0,0])
  • While there are tiles to load
    • Load the current most important tile.
    • Did it fail?  Okay we're done here
    • Did it succeed?  Great, add the four children to the list.

Who doesn't love pseudo-code?  Let's just see what that looks like.

By the numbers

That approach had some advantages.  Tile sources can be sparse and it's a little resistant to network failure.  Most important (to me), there are never any holes in the globe.  There's always something to look at, even if it's blurry.

Loading From the Top: Map


One secret of WhirlyGlobe-Maply is that the globe and map share most of their code.  So when I got the map working, I used the same tile loading scheme.

Loading from the Top with Mapbox imagery


That worked okay, but if you start out zoomed in to the map you're waiting a long time to see the right tiles.  There's a whole lot of blurry for far too long.  It gets worse the farther you're zoomed in.  Here's how that looks with tile numbers.

Loading from the top by the numbers

The flashy colors at the beginning are low res tiles we can't see the numbers for.  I could probably argue there are cases where that works well.  But not for a map.

Loading the Current Zoom Level


The obvious thing to do for a map is to load the zoom level we need for the area we're looking at.  If it's a flat (2D) map, that's actually pretty easy.

Single level load with imagery


Simple, but I don't like it.  You're looking at too much blank screen.  For a retina display and a remote tile source this is irritating.

Still, there are cases where this may be appropriate and it's easy enough to activate.  When you set up your MaplyQuadImageTilesLayer set useTargetZoomLevel and singleLevelLoading to YES.

Single level load by the numbers


And again, the tile loading with numbers.  Fine for some cases, like a weather overlay.  But we can do better.

Loading Multiple Zoom Levels


Focusing on a single zoom level is great for performance, but bad for display.  I'd rather have a hybrid that loads a few low res tiles and then fills in the rest.  Like this.

Multi-level load with Mapbox imagery

That's nice.  We've got something reasonable to look at almost immediately, then we get some detail.  When the highest resolution comes in we probably didn't realize it wasn't already sharp.

Multi-level load by the numbers


Pretty good, but the levels you want to load may depend on your tile source.  Luckily, it's configurable.  When you create your MaplyQuadImageTilesLayer set useTargetZoomLevel and singleLevelLoading to YES and then assign an array of numbers to multiLevelLoads.

For example, try @[-4,-2] in multiLevelLoads.  If the user should be looking at level 22 data, then the layer will start out loading level 18, then work on level 20 and then fill in level 22.

Summary


The old tile loading logic could feel out a sparse tile source efficiently.  The new tile logic can do it too, but wow was that complicated.  And perhaps less efficient.  If you've got a normal tile source, it shouldn't be a problem.

I like the multi level loading approach.  But it does cost more memory and rendering time than the single level approach.  Use it thoughtfully and test it in your app.

This is all available in WhirlyGlobe-Maply 2.3.



Tuesday, September 2, 2014

A Cure for the Jiggles

This one's a problem for the big apps, the ones moving gigabytes of imagery and vectors.  You know, the people who pay me.  It looks like this.

I'm... not feeling so good.

What's happening is a floating point precision problem.   If you want to represent the globe all the way from orbit down to nose picking distance, 32 bits of floating point won't do the trick.  Let's look closer.

The Problem


If single precision floating point doesn't crack it for you, why not use 64 bit doubles?  Sure, if OpenGL ES supported them, though it would be kind of a waste.  At some point you have to convert your data to 32 bit floats (or even fixed point, you weirdo).  There's no escape, but there are cheats.

At first blush it's your image tiles and vectors causing the problem.  That's part of it, but don't forget your motion model.  The quaternion controlling your motion can't really run on 32 bit floating point either.

Here's what I did to fix it all.

The Solutions


Let's start with the motion model, what's turning your gestures into movement.  Zoom in too far and things start twitching.  The problem here is the quaternion logic and the matrix math around it.  In WhirlyGlobe-Maply 2.2 it was 32 bit floats.  The fix was to move all this to 64 bit doubles.

Cool picture courtesy: Indiana University



That helped a lot.  Now the motion is nice and smooth.  But things are still jumpy.  It's the data.  As those 32 bit floats get pushed through the OpenGL ES pipeline, they jump around a little.  It's a little different each frame and it adds up to visual insanity.

There's an easy solution to this one, so let's start with the vectors.  For each batch of vectors we define a local origin.  We pass that origin to the renderer as a double precision matrix.  The renderer multiplies the local matrix by the global model/view/projection matrix and then converts down to single precision.  Since the coordinates are local to that origin, nothing gets too big in 32 bit floating point.

Example tiles with spatial origins

You might think we could do the same thing for image tiles.  We almost can, but we don't represent image tiles individually, we batch them together for speed.  Raw, glorious rendering speed.  The solution is similar to vectors, but with a twist.

For a given tile, we look at its size and its center.  We compare it to the big drawables we've already created, looking for a nearby center.  If it's not too far away, we reuse that big drawable, otherwise we create a new one.

Example origins
In that example there may be 100 different tiles of varying resolutions sorted into origin A and origin B drawables simply because the center is close enough.


Conclusion


The center sharing solution works pretty well in practice.  We don't lose too much performance and the jigglies are gone.

50% fewer seizures!

If you're doing closeup work with vectors, be sure to set kMaplyCenter to YES.  The vector tile module does it and the image tiles do their thing by default.  Twitching problems in screen space objects and overlaid views were fixed with similar trickery.

You'll find this all in WhirlyGlobe-Maply 2.3.

Thursday, August 21, 2014

WhirlyGlobe-Maply 2.3

I'm pleased to announce the official release of WhirlyGlobe-Maply 2.3.

Yup, it can do both of those.

The big new feature for 2.3 is maps.  Vector maps.

Where to Get WhirlyGlobe-Maply 2.3


Over at github is the canonical place.  The master branch is now on 2.3.  Compiling from source is a bit slow, but then you've got the source.  Be sure to read the README.  Seriously, read it.

You can also download the binary distribution right there.  Less to set up, faster to compile, but there's a lingering sense that you're not cool.  I don't judge.

Documentation


You can browse the reference documentation right here.

It needs more.... more.  I have a Getting Started guide under development and a rework of the WhirlyGlobe-Maply site to go along with it.  When that's done I'll make more noise about the launch.

Next Up


A few months ago I'd planned to segue into WhirlyGlobe-Maply 3.0, which is the Android/iOS version.  Instead, clients have been paying me to add some big new iOS features.  Damn you paying clients!  So we get at least one more 2.x version.

The develop branch is moving to WhirlyGlobe-Maply 2.4.  I'd stick with the master branch for now.