Saildrone Forecast

Is there anything cooler than shipping an app you've spent a year and a half working on?
It was a rhetorical question.  There is not.

Saildrone Forecast

Hint: It's a weather app.

Saildrone Forecast

Yup, Saildrone Forecast is a weather app.  Just watch the video.

Go download it.  I'll wait.

Technical Bits

There's a ton of stuff in Saildrone Forecast that's unique.  But there are a couple bits of interest in WhirlyGlobe-Maply itself, like the new sampler/loader architecture and offscreen render targets.  Go watch the video.

Could you make your own weather app out of the box now?  Ha ha.  Oh god no.  There's so much other stuff to do.  Not to mention the data.  But there are cool things to do with the vector maps and render targets.

Acknowledgements

I didn't do this all alone.  Heck, I didn't even write the whole app. 

Credit goes to Logical Animal for the rest of the UI.  Big thanks to the meteorology/modeling team at Saildrone for the data and putting up with stupid questions.  Thanks to the platform developers and ops for processing data and keeping it running.  And, of course, management and finance upon whom I can bestow the greatest contracting honor:  They always pay early.

Technical Advisory Committee

I'm forming a technical advisory committee.  We'll meet a few times and discuss the iOS Metal port and rendering architecture in the toolkit.

This post is for developers I'm trying to recruit to the committee.  Let's start with the basics.

What is WhirlyGlobe-Maply?

National Geographic World Atlas (RIP)

WhirlyGlobe-Maply is a real time geospatial display toolkit for iOS and Android.  It's used in a whole bunch of apps, like FlyQ EFB and a few big weather apps like Dark Sky.

Dark Sky

The toolkit's been around since 2011 and it's carved out a crazy little niche in aviation and weather.  It's not a wildly profitable business, but it doesn't lose money either.  How many can say that?

Morecast

Oh, and it's open source.  So that's fun.

WhirlyGlobe-Maply 3.0

The last version I shipped was 2.6 a few weeks.  A huge improvement for data loading on iOS and a nice upgrade to the Android build system.  I.... need to do a post on that.  But no time.

3.0 is the big upgrade.  I'm going to add iOS Metal support, redo the rendering internals and upgrade the Android port with a better shared C++ core and newer versions of OpenGL ES.

Technical Advisory Committee

So here's the deal.  I'm adding a new renderer for Metal and replumbing the internals to match.  I'd really love to discuss it with a few experts along the way.

This is an open source project so, you know, not a ton of resources.  Here's what I'm offering:

• Free lunch in a pleasant conference room at my coworking space in San Francisco
• An invitation to pontificate on real time rendering without having to do any work
• A cool entry on your resume

You do not need to know anything about geospatial or weather or aviation.  I got that covered.  Here's what I need from you:

• Work in San Francisco or nearby.  No budget for travel, sorry.
• Know something about real time rendering, iOS Metal or Android OpenGL 3.x
• Willingness to chat about real time rendering

It's your real time rendering chops I'm interested in.  So game developers most welcome!

Contact me at sjg@mousebirdconsulting.com and we'll see if you're a good fit.

Announcing WhirlyGlobe-Maply Imagery Pro

Today I'm delighted to announce the first option on top of the WhirlyGlobe-Maply toolkit: Imagery Pro.

The red's moving away so that's good.

Up to now everything has gone into the free toolkit or client apps.  This marks the first time we're keeping something to ourselves and, yes, charging for it.

Who's This For?

We love all our users equally.  Okay, maybe we love the ones who pay support a little bit more.  But you're all cool.  Honest.

We've attracted a lot of weather and aviation users.  It makes sense.  They have pretty serious data display needs and only a passing interest in street maps.

Imagery Pro is for them.  We're taking what we've learned and pushing way beyond.

What's In It?

It's all in the marketing, but the basics are like so:

• Efficient data transport
• Fancy shaders
• Getting rid of the !#*$& seams
• Easier particles

I do love my animated gifs

All good stuff if you do weather and/or aviation.

So what about WhirlyGlobe-Maply?

That's right.  My five year plan to give away massive amounts of geospatial code is finally coming to fruition!

But no, the base toolkit is all good.  Imagery Pro is about expansion into a couple of markets that want much more right now and they're willing to pay for it.

We're continuing to expand WhirlyGlobe-Maply with iOS (2.5) and Android releases in the pipeline.  That's not going to change.

Dark Sky for Android

They launched a few weeks ago, but we haven't talked about it yet (here).  Dark Sky for Android!

That's some weather.

Man, that was a lot of work.

Weather Apps Are Popular Apps

The service, the UI, the data, notifications, and the globe are the big pieces of Dark Sky.  We make the globe.

Dark Sky is really popular and has a particularly loyal following.  In just about any group I meet, there's a good percentage who use it regularly.  I'm not reticent about using that fact.

So Much Android So Little Time

The Android port took a while on my end.  I could have done a hacky version, but that wouldn't work for everyone else.  No, we had to port the toolkit in a way Android developers would find friendly.

Totally not a screenshot from Keynote.

The C++ core, which does the rendering and manages the low level objects is (will be) the same on iOS and Android.  But everything above that level had to be new.

Developers expect their toolkits to be familiar on the local platforms.  That means Swift interfaces and dispatch queues for iOS, Runnables and integer colors for Android.  And lots and lots of other things.  This takes time.

Back To Dark Sky

For Dark Sky Android we ported the parts they needed and filled out the underpinnings.  This was by far the biggest app yet released with WhirlyGlobe-Maply on Android.

That's some more weather.

Dark Sky launched on Android with a subscription model.  Apple totally thought of that first and recently announced a similar one.  I'd love to see that take off on both platforms.

In the mean time, go buy it!

Welcome to wetter.com!

I'm proud to welcome the German wetter.com iOS app to the WhirlyGlobe-Maply family.

The wetter.com app presents regional weather, including predictions, on a map.

Details, details, details

It's funny to celebrate a flat map when the toolkit does a globe.  But for a regional app, the globe doesn't make sense.  The flat map simplifies things for the user and lets us optimize the data transport.  All the good stuff we do on the globe works on the map.

The app itself shows current predictions, recent radar, and cloud cover.  As well as some other cool non-map related stuff like weather cams.

Are those labels on top of the weather?  Oh my!

Mobile hardware is really good these days.  Since we're no longer planning for the iPad2 we can do some really great OpenGL ES shader work.  If you zoom in close, you'll see some great spatial and temporal interpolation.

The Inevitable Marketing Push

There are a lot of regional weather apps for iOS and Android.  Not so much in the United States where almost all the data comes from NOAA.

In the rest of the world weather data tends to be less free and more complicated.  Add to that language and local preference and you get a lot of interesting, distinct regional weather apps.  If you make one of those and want to upgrade your graphics, let's talk!

GeoSpatialKit - A suggestion for Apple

I spent last week at WWDC in San Francisco.  Not too much new, but I saw a lot of performance and optimization in iOS9.  I spend a lot of time on those myself, so I approve.

But Apple's geospatial support is still... mediocre, particular given what they must be paying for the map.  I got to thinking:  What would make geospatial display work better on iOS and OS/X?

First, what am I talking about?

Geospatial Is....

Geospatial data display is maps and globes.  A weather app is a geospatial app, as is any sort of map app and so is an Atlas.

Globe, map.  Map, globe.  Same thing.

Geospatial is what I do.   I make a popular toolkit for map and globe data display on iOS (and <cough> Android </cough>).  Presenting geospatial display to developers in a way they can handle is what I do.

And that leads to the important question:  What's wrong with the way Apple does it now?

The Current Situation on iOS (and OS/X)

Let's use weather as an example.  Everyone loves (or hates) their weather app.  In any case, they have an opinion.  So how do you make a serious weather app, with animated radar and such?

You build a serious weather app in one of two ways:  Core Graphics or OpenGL ES.  You can tell which was used immediately.  Is it a little weird and janky (Core Graphics) or smooth and pretty (OpenGL)?

Why that is has a little to do with rendering and a lot to do with data management.  The way you display 20 pins in UIViews is not the way you render 20 layers of animated streaming radar data.

And okay, that's a hard case, but it's also plain old map apps that are kind of 'meh' under Core Graphics.  There's another reason for that.

MapKit Under Glass

The Maps App is a big, capable and (I imagine) very expensive system.  It does a lot and it's pretty good at it.  MapKit looks to be a sort of wrapper on top of it.  That's a problem.

Hands up.  Who's reimplemented that compass?

You can't change the bulk of what MapKit is displaying.  Sure, you've got a few knobs to twiddle and you can toss a little of your data on top, but only grudgingly.   But there's an even bigger issue.  MapKit was obviously (obviously!) written with OpenGL ES, but none of that infrastructure is available to the rest of us.

So here's the question.  If you scooped out MapKit's brains and made neat little piles, what would that (unfortunate metaphor) look like?

CATiledLayer Revisited

I think you'd start with CATiledLayer, the go-to data structure for the pre-OpenGL map era.  As a reminder, CATiledLayer will break the world into little pieces and tell you when to load those pieces.

Something like this

A new CATiledLayer is a good place to start, but it's probably a bit low level and it's missing one giant thing: A Spatial Reference System.

Adding an SRS to a CATiledLayer(-like thing) would solve the first set of problems.  Your data could exist in one system while displaying in another.  I'd also make the levels and loading strategies explicit with budgets for tiles and such.  And threads everywhere.  EVERYWHERE.

One tile loading strategy

That helps with loading, but what do you do once you've got that little image or basket of vectors for a tile.  How do you render it fast?

With Core Graphics, you really don't render it fast.  And the layer based image solutions are kind of weak.  It's time to drill a hole in UIKit and plunge into the icy waters of OpenGL ES.

Images, Vectors, and Shaders - Oh My!

Most of what you want to display on a map is pretty simple:  Image tiles, symbols, text, lines, and polygons.  All pretty easy to display slow, but hard to render fast.

Sometimes a map is just a map

Here's what you do.  You develop a little system for abstracting those object types to hand to a renderer.  Then let the system manage them, batch them and move them around.  Essentially you've moved the developers job off into other threads:  Just load the data and let the system handle it.

This is suspiciously similar to what I do in WhirlyGlobe-Maply.

What About The Globe?

Well, yes, everyone loves the globe; even developers who should know better.  User interaction with a globe is just different from a map and switching between the two makes life harder than it needs to be (looking at you MapKit).

Globe + Weather = Awesome

But the globe is cool, so where do you put it?  SceneKit.  SceneKit is where you put it.

Pull the tile loading logic from CATiledLayer++ above, make a nice globe surface and voila!  You can reuse most of what you build for the 2D maps.  Sure, it's a little more work to set up SceneKit and live in OpenGL-land.  But globe.

Aaaaand back to MapKit

Okay, so you've built up a nice rendering infrastructure for map tiles, vectors, symbols and other such stuff.  It gets along with both UIViews and SceneKit.  But does it use anything from MapKit?  It also begs the question:  Should you rewrite MapKit in this stuff?  

No.  Good grief, no.  Only interns and the criminally insane rewrite working systems for the hell of it.  Instead, you steal data from the map.

Nothing personal, Santa Rosa

Apple Maps has a number of interesting data layers it would be fantastic to reuse in other apps:  Roads, water, symbols, land, parks, and labels to name a few.  Especially labels, which would make one of my dreams come true:  Putting the @%&#@ labels on top of the animated radar.

Map Data Reuse

Mere mortals can't reuse Apple Maps data... but Apple could.  If they started teasing it apart and tossing it into a system like I've described above, it would make maps apps that much more interesting.  And yes, the label thing, which bugs me a lot.  Seriously, that's annoying.

Missouri?  Maybe?

This would also let us mix and match data from varying sources.  There are some excellent map tiles out there and I'm sure Apple could come up with a few more of its own that don't quite belong in what they've got now.

Wrapping it Up

I've skipped over a few details, but here's the basic story.

• Add a Spatial Reference System for selected layers
• Soup up CATiledLayer for some serious tile management
• Drill a hole down to OpenGL ES (and/or Metal) for some decent primitive display.
• Globe meet SceneKit.  SceneKit, globe.
• Start pulling data layers out of the map source.

Do that, Apple, and you've got yourself some seriously reusable geospatial display support.  It would be interesting to see what came of it.

Actually, feel free to not do that.  It would put a crimp in my WhirlyGlobe-Maply business.

WhirlyGlobe For Android

A few months ago I ported the Maply part of WhirlyGlobe-Maply to Android.  It's in shipping apps!  Now it's WhirlyGlobe's turn.

What my desk looks like right now.  Except no plushy thing.

Let's see who's paying for the port.

Bad weather looks awesome.

Dark Sky

Dark Sky is the most prominent app using WhirlyGlobe-Maply today.  They've been good users and great clients (e.g. they pay on time).  I'm pleased to have them taking the lead on this one.

Their users have been clamoring for an Android version, so this is a necessary step.

WhirlyGlobe Android Port

With the Maply rendering engine already there, WhirlyGlobe shouldn't be too bad.

I've already scoped out the hardware, got my ARM and x86 devices and dealt with versions back to 4.0.  I've also experienced the flow of Android development and made my peace with it... by increasing my engineering estimates.

The client comes first on the port, but we're doing this in the open.  Because open source.  You can always grab the develop_3_0 branch and have at it.  After this launches I'll look into making it easier to install.  And, you know, documentation.

What's Next

I've been very busy on two big client apps, the port, and a bunch of other stuff.  Those will launch in the next few months.  As always, I'd love to hear about your apps too.

The new site has gone over really well, particularly the documentation, so more of that is in the works.

WhirlyGlobe-Maply 2.4 is shambling toward release, perhaps in March.  I suspect that'll be the last version before 3.0 and the great Android/iOS unification.  We'll see.

Dark Sky & Progressive Loading

There's a new Dark Sky for iOS out.  If you're interested in weather apps, I suggest checking it out.

Terrible weather is pretty!

Forecast.io is one of my favorite clients (translation: They pay on time), but whenever they ask for something it's going to be hard.  This was no exception.

Tile Loading

Dark Sky has a ton of custom logic in it, not to mention all the custom data they feed it.  But large parts are using standard WhirlyGlobe-Maply.  The tile loading logic, for instance, is all in there.

Let's review.  For the globe, we start loading at level 0 and work our way down.

I will reuse this at least 5 more times.

The app does something funkier when you're zoomed in, but let's ignore that.  This approach works well for the globe, but for one problem:  Animation means frames.

So let's say you're loading 20 frames of animation to show the weather changing over the course of a day.  That means you have to load each tile 20 times.  Sure, you can do it in parallel... but still.  It takes a while.

Progressive Tile Loading

Adam at Forecast.io wanted something better.  "Couldn't we load individual frames" he said?  "Sure", I said.  "Just let me renew my Xanax prescription and we'll give it a try."

And now WG-Maply can load frames progressively.  That means it'll start at your most important frame and work its way outward.  It'll load depth first, so that first frame looks good everywhere and then work outward to the less important frames.

You can set the frame priorities yourself, change them on the fly, and get feedback on what is properly loaded.  When the user moves, we start loading at the most important frame again, then work on the others.

That Sounds... Complicated

Why yes, voice in my head that may never, ever go away, it is complicated!  It's not all that hard to use, though.  You provide your tile sources as normal and let the loader do its thing.  If you want to tell the user what's going on, there's a callback for that.

Weather apps are the most obvious users here, but anything with animation is fair game.  I'd love to see some data visualization apps using this feature.

WhirlyGlobe-Maply 2.3 Beta 2

It's finally time to release WhirlyGlobe-Maply 2.3 into the wild.  I've been using it for some time and, sure, you could have always switched to the develop branch, but hey, release!

Courtesy Gravitystorm

Thanks go out to the contributors on this version.  This one's seen the biggest set of contributions from people other than me.

Let's get to it.

Where to Get WhirlyGlobe-Maply 2.3

For the source distribution, head over to github, clone the repository and switch to the develop_2_3 branch.  Be sure to update your submodules.

You can also download the binary distribution.

Documentation for 2.3 can be found here.

What's new In WhirlyGlobe-Maply 2.3

It's a huge list of new features.  I'll be working my way through blog posts for the next several weeks.  For now, here's the short version grouped for my own amusement.

Housekeeping

Here's the stuff that doesn't fit well into another theme.

• 64 bit device support.
• Xcode 5.1 is supported.
• OpenGL ES 3.0 is supported.
• Tessellation bugs are now fixed.
• EAC/ETC2 texture compression was added.
• High resolution jiggliness was fixed in images, vectors, annotations and screen objects
• Resources submodule is smaller
• Background CPU usage is much lower for paging layers.
• Screen shapshots are now possible
• MaplyTexture was added for explicit texture management
• Animation is available through the startChanges/endChanges methods.

Maps / Maply

How about features that are useful in flat maps?  This version saw a lot of improvements to the Maply side of things.  It's quite the credible map toolkit now. 

• Wrapping across the date line, for data display, selection, gestures, and overlay.
• Vector tile support
• Mapnik vector tile support
• Seriously, go check this one out
• New gestures for 2D
• Screen importance logic is short circuited (e.g. faster) for 2D

Gestures & Feedback

This version saw a host of new gestures, particularly for 2D and improvements in the feedback methods for delegates.

• Heading control
• North up panning in the globe
• Double tap, two finger tap, tap hold and drag.
• Location animation control via delegate.
• Feedback with the didStartMoving/didStopMoving/willStoopMoving methods
• Screen to geo and geo to screen calculations
• Viewing bounds calculations for setting height to show a set of features
• MaplyAnnotation added.
• These are nice annotations based on the SMCalloutView library

Quadtree Paging Layers

The paging layers, both for general features and for imagery saw a ton of changes.  These are the things that handle vector tile and image based paging.

• Single and multi-level loading.
• This means you don't have to always load from level 0
• Animation in quad image layer and the multiplex loader.  Think weather.
• Changing color, enabling/disabling, reset/reload, changing tile sources.
• Switched to AFNetworking, handles lots of requests better.
• Broke the remote source from the tile info.
• WMS improvements
• Background overhead is much lower.

Wrapup

This was a big release.  Hard to believe I'm also working on 3.0 for Android.  At some point those will be merged, but there's no real hurry for the iOS users.

I'd like to do more documentation for this version.  I've got someone working on getting started examples for a globe and map.  We'll see if those help.

Anyway, 2.3 is pretty well tested.  I'll put it out there for a week or so to see what turns up and then make it official.

Next up, more blog posts describing what's in it.

Ericsson Texture Compression

I recently added support for the new OpenGL ES 3.0 compressed texture formats, ETC2 and EAC.

Now with Animated GIF technology.  2002 here we come!

Compressed Textures?

Compressed textures have been in OpenGL ES on iOS for quite a while.  What they let you do is represent a texture in less memory.  The hardware will do a little bit of math as it fetches a pixel, giving you a real RGBA value in the shader.

This is nice if you're short on memory when doing animation and we animate a lot when doing weather.

Sure, I wanted to animate this one too.  You're welcome.

Before 3.0, iOS hardware supported PVRTC through an extension.  That's a fine format, but it suffers from one big flaw:  You can't load just a piece of a PVRTC texture, you have to load the whole thing.  WhirlyGlobe-Maply makes heavy use of dynamic textures, so that was a non-starter.

The ETC2/EAC Compressed Formats

PVRTC is clever, but clever can be overrated.  The new formats, ETC2 and EAC are less clever, but their support is much deeper.  Most importantly, they support glTexSubImage2d().

I rely on that command for the dynamic texture atlases. Since textures can come in at any time, via map tiles or font glyphs, I can't pre-construct the atlases; I have to do it dynamically.  PVRTC didn't support modifications to textures.  ETC2 and EAC do.

ETC2 / EAC Display Support

You'll find support for 7 of the compressed formats in WG-Maply 2.3.  The first three are immediately useful, the rest are a bit obscure.

• MaplyImageETC2RGB8 - Nominally 8 bits per channel RGB.  This is compressed roughly 8:1 from the naive 32 bit RGB.
• MaplyImageETC2RGBA8 - RGBA, 8 bits per channel.  Compressed 4:1 from naive 32 bit RGBA.
• MaplyImageETC2RGBPA8 - RGB plus a single bit Alpha.
• MaplyImageEACR11 - Single channel, nominally 16 bits.
• MaplyImageEACR11S - Signed single channel.
• MaplyImageEACRG11 - Two channel, nominally 16 bits.
• MaplyImageEACRG11S - Two channel, signed.

The first one, RGB8, is useful for your full color basemap and the second one, RGBA8, is decent for full color transparent overlays.  The rest have their uses, but get weirder.

And I should Care Because....?

Let's do a little math.  WG-Maply likes to allocate 2048x2048 texel maps for its dynamic texture atlases.  For an RGBA texture, that takes up 16MB.  Now there are simple textures formats that just chop the lower bits out.  For some data sets that'll work in 8MB.  With ETC2 RGB8 you can represent the same data in 2MB.  That's either 8:1 or 4:1 depending on the data set and ETC2 looks much better.

The new texture formats are part of the OpenGL ES 3.0 standard.  Only a few devices support 3.0 at the moment, but that list is getting bigger every day.  For some apps, it's a huge win.

Great, so How Do I Serve These?

Mention anything other than JPG or PNG to server developers and they look at you with a mix of loathing and fear.  You can't really compress these efficiently on the client side, so the server's got to serve them.

There's a free program called etcpack that will do the conversion for you.  I've run some tests myself (at the behest of a client) and found conversion to ETC2 RGB8 and RGBA8 pretty easy.  You still have to store the images, of course, but disk is cheap.  Right?

Conclusion

For certain kinds of apps ETC2 and EAC are a huge win.  If you use a lot of image overlays and you find yourself low on memory, they're worth a look.  Full support is in WhirlyGlobe-Maply 2.3.