Thursday, August 20, 2015

Cesium.js Terrain Support

We've been adding support for one of Cesium's terrain formats recently.  Cesium, if you don't know it, is similar to WhirlyGlobe-Maply, but for the web.

Our toolkit has its own gridded terrain format, but I'd rather use someone else's.  Cesium has a better thought out and much more expansive set of terrain formats and tools available.

I was recently invited to discuss it at a Cesium meeting in Los Angeles.

SIGGRAPH Presentation

I used to attend to SIGGRAPH religiously, but I haven't been back in years.  It was a blast to walk the exhibition and see what's new.  Lots, of course, but a lot still looks the same.  Heck, it still smells the same.  No no, not that way.  It's the air conditioning and the projection displays.

Anyway, I gave a short presentation at Cesium's Birds of a Feather meeting last week.  Slides can be found here.

The Short Version

But what you're probably wondering (if you're still reading) is how well does the Cesium terrain work?

It's a start
Cesium has a funky way of representing terrain tiles that don't match to your standard TMS/Google/OpenStreetMap/Mapbox view of the world.  So we can't overlay those data sets on top.  This is limiting.

The Future!

Obviously, I'd like to fix that.  We'll need to do some resampling of image tiles as they come in to match the Cesium terrain.  It'll be a bit slower, but simple enough for the user.

So for now, you can show Cesium terrain, but without images on top.  If you'd like to fix that, hey it's open source, go for it.  Or you can wait until someone pays us to do that.  I suspect it'll happen in the next few months.

Friday, August 14, 2015

Revisiting Support for WhirlyGlobe-Maply

A few months ago I introduced a support contract for WhirlyGlobe-Maply.  $600 USD for all the questions and bug fixes on the iOS version.

Nice 'stache, Australia

I like how support has turned out, but I need to make some tweaks.  But first up, who uses the toolkit?

Who Are The Users Anyway?

I had the image of a lone hobbyist beavering away at their personal project.  Yeah... not so much.  There are a few hobbyists, and they're cool, but they're rare.

So who is using it?  It's the serious app developers.  People doing it for a living, often part of a big corporation or at least something with revenue.  Organizations, if you will, who can afford to pay for things.

So You're Raising Prices?

Oh yeah.  I'm raising prices.  I'm spending more hours on support than I'm getting back in revenue and I'm not getting the overhead time I was after.

Support will now be $950/year.  I came up with that number by looking at hours spent, the number of contracts, and the desire to keep it a three digit number.

The support thing is working out well in every other way.  Seeing what people need and how they use it is really great.  And I've learned something really vital: people can write apps without me.

Wait?  The Toolkit Actually Works?

I know, right?  You could always write simple apps without me.  Now you can write big, complex ones without me.  WhirlyGlobe-Maply <sniff> is all grown up </sniff>.

HTML jokes.  Right below puns on the humor scale.

But I don't mind.  Not needing me is excellent!

So You're Going Broke, Then? 

Happily, no.  mousebird consulting (inc!) has business in the queue through early 2016.  We've just passed the revenue number for 2014.  Like, just a few days ago.

To pull that off, I've found a happy medium between pure toolkit features and whole app development.  Sort of whole module development, if you will.

What About Android?

Support is just for iOS at the moment.  See, the iOS version works more or less like it claims and I can fix any bugs that crop up.  The Android version isn't there yet.

It'll get there.  I've got big clients pushing big projects out on Android.  So it'll get there.

More!  I want more!

Okay, okay.  Fine.  How about this.  If I get 10 new support contracts by the end of 2015 I will set up automatic daily builds, some simple unit tests, and try monthly releases.  We'll have to see how that last one goes.

Plus, hey, features.

2.4: Day/night Shader + Atmosphere + Stars = Daaaaaamn

WhirlyGlobe-Maply 2.4 is almost out the door.  Beta5 just went up the other day and it is a scorcher.  Wow.  The stuff that's in 2.4 is just... daunting.  To write up, anyway.  I'm way behind.

Bored Now.  Wrap it up!

If you've already started talking to me about a support contract, you're grandfathered in at the old price.  Renewals will be a bit higher, but won't go all the way up to the new price.

Everyone new pays the new price.  But look at the benefits!  People are writing actual complex apps with just the support contract.  So can you!

Wednesday, June 17, 2015

GeoSpatialKit - A suggestion for Apple

I spent last week at WWDC in San Francisco.  Not too much new, but I saw a lot of performance and optimization in iOS9.  I spend a lot of time on those myself, so I approve.

But Apple's geospatial support is still... mediocre, particular given what they must be paying for the map.  I got to thinking:  What would make geospatial display work better on iOS and OS/X?

First, what am I talking about?

Geospatial Is....

Geospatial data display is maps and globes.  A weather app is a geospatial app, as is any sort of map app and so is an Atlas.

Globe, map.  Map, globe.  Same thing.

Geospatial is what I do.   I make a popular toolkit for map and globe data display on iOS (and <cough> Android </cough>).  Presenting geospatial display to developers in a way they can handle is what I do.

And that leads to the important question:  What's wrong with the way Apple does it now?

The Current Situation on iOS (and OS/X)

Let's use weather as an example.  Everyone loves (or hates) their weather app.  In any case, they have an opinion.  So how do you make a serious weather app, with animated radar and such?

You build a serious weather app in one of two ways:  Core Graphics or OpenGL ES.  You can tell which was used immediately.  Is it a little weird and janky (Core Graphics) or smooth and pretty (OpenGL)?

Why that is has a little to do with rendering and a lot to do with data management.  The way you display 20 pins in UIViews is not the way you render 20 layers of animated streaming radar data.

And okay, that's a hard case, but it's also plain old map apps that are kind of 'meh' under Core Graphics.  There's another reason for that.

MapKit Under Glass

The Maps App is a big, capable and (I imagine) very expensive system.  It does a lot and it's pretty good at it.  MapKit looks to be a sort of wrapper on top of it.  That's a problem.

Hands up.  Who's reimplemented that compass?

You can't change the bulk of what MapKit is displaying.  Sure, you've got a few knobs to twiddle and you can toss a little of your data on top, but only grudgingly.   But there's an even bigger issue.  MapKit was obviously (obviously!) written with OpenGL ES, but none of that infrastructure is available to the rest of us.

So here's the question.  If you scooped out MapKit's brains and made neat little piles, what would that (unfortunate metaphor) look like?

CATiledLayer Revisited

I think you'd start with CATiledLayer, the go-to data structure for the pre-OpenGL map era.  As a reminder, CATiledLayer will break the world into little pieces and tell you when to load those pieces.

Something like this

A new CATiledLayer is a good place to start, but it's probably a bit low level and it's missing one giant thing: A Spatial Reference System.

Adding an SRS to a CATiledLayer(-like thing) would solve the first set of problems.  Your data could exist in one system while displaying in another.  I'd also make the levels and loading strategies explicit with budgets for tiles and such.  And threads everywhere.  EVERYWHERE.

One tile loading strategy

That helps with loading, but what do you do once you've got that little image or basket of vectors for a tile.  How do you render it fast?

With Core Graphics, you really don't render it fast.  And the layer based image solutions are kind of weak.  It's time to drill a hole in UIKit and plunge into the icy waters of OpenGL ES.

Images, Vectors, and Shaders - Oh My!

Most of what you want to display on a map is pretty simple:  Image tiles, symbols, text, lines, and polygons.  All pretty easy to display slow, but hard to render fast.

Sometimes a map is just a map

Here's what you do.  You develop a little system for abstracting those object types to hand to a renderer.  Then let the system manage them, batch them and move them around.  Essentially you've moved the developers job off into other threads:  Just load the data and let the system handle it.

This is suspiciously similar to what I do in WhirlyGlobe-Maply.

What About The Globe?

Well, yes, everyone loves the globe; even developers who should know better.  User interaction with a globe is just different from a map and switching between the two makes life harder than it needs to be (looking at you MapKit).

Globe + Weather = Awesome

But the globe is cool, so where do you put it?  SceneKit.  SceneKit is where you put it.

Pull the tile loading logic from CATiledLayer++ above, make a nice globe surface and voila!  You can reuse most of what you build for the 2D maps.  Sure, it's a little more work to set up SceneKit and live in OpenGL-land.  But globe.

Aaaaand back to MapKit

Okay, so you've built up a nice rendering infrastructure for map tiles, vectors, symbols and other such stuff.  It gets along with both UIViews and SceneKit.  But does it use anything from MapKit?  It also begs the question:  Should you rewrite MapKit in this stuff?  

No.  Good grief, no.  Only interns and the criminally insane rewrite working systems for the hell of it.  Instead, you steal data from the map.

Nothing personal, Santa Rosa

Apple Maps has a number of interesting data layers it would be fantastic to reuse in other apps:  Roads, water, symbols, land, parks, and labels to name a few.  Especially labels, which would make one of my dreams come true:  Putting the @%&#@ labels on top of the animated radar.

Map Data Reuse

Mere mortals can't reuse Apple Maps data... but Apple could.  If they started teasing it apart and tossing it into a system like I've described above, it would make maps apps that much more interesting.  And yes, the label thing, which bugs me a lot.  Seriously, that's annoying.

Missouri?  Maybe?

This would also let us mix and match data from varying sources.  There are some excellent map tiles out there and I'm sure Apple could come up with a few more of its own that don't quite belong in what they've got now.

Wrapping it Up

I've skipped over a few details, but here's the basic story.
  • Add a Spatial Reference System for selected layers
  • Soup up CATiledLayer for some serious tile management
  • Drill a hole down to OpenGL ES (and/or Metal) for some decent primitive display.
  • Globe meet SceneKit.  SceneKit, globe.
  • Start pulling data layers out of the map source.

Do that, Apple, and you've got yourself some seriously reusable geospatial display support.  It would be interesting to see what came of it.

Actually, feel free to not do that.  It would put a crimp in my WhirlyGlobe-Maply business.

Sunday, June 7, 2015

WhirlyGlobe-Maply Contract Job

I've got a short term contract available for the right person.  I need you to run through the tutorials, looking for errors.

The best thing about this:  You don't have to be an iOS expert!  In fact, it's better if you're not.

Tutorial Checker

The WhirlyGlobe-Maply tutorials have been a huge boon to beginning users.  But I suspect some of them have errors and spots that are difficult to follow.  I know of a few, but I'd like to find the rest.

That's where you come in.  As a Tutorial Checker you need to have the following:
  • Appropriate Apple hardware and Xcode already installed.
  • Some experience building at least a test app for iOS.
  • Excellent English skills.
  • Be in the US.  I'm not sure how to legally pay people outside the country.
If you've just wrapped up one of those six week code academy things for mobile, that would be absolutely perfect.

What's Next

If you're interested, drop me a line.  You can find my contact info on the WhirlyGlobe-Maply page.  Let me know what your development qualifications are (they can be thin) and we'll talk.

Friday, June 5, 2015

mousebird consulting Needs YOU!

I'm hiring.  If I can find the right person.

It's not relevant, I just like this image.

What We Do

You can read the web site yourself, but here's the tl;dr.  

We do high performance geospatial display for mobile devices.  Our customers include National GeographicForecast.IO and many others.  Our code burns millions of battery hours on millions of devices all over the world.

At any given time we're working on apps in weather, aerospace, education, and mapping.  And lately, all of them.  At once.

What It Is

I need another software developer, primarily.  These are the areas we work in.
  • OpenGL (ES or not)
  • iOS Development, Objective-C
  • Android Development, Java
  • User Interface development
  • C++
  • Geospatial data processing: GDAL, OGR, Proj.4
  • Data processing on Amazon ec2

If you have experience in any three of those, that would be interesting.  If you have experience in all of them... you're probably too expensive, but let's talk.

Where It's At

I'm in San Francisco.  If you are too, you can share the office.  Yes, an actual office with a door and everything.

If you're remote, that's fine.  Our clients are scattered around the world and don't visit much (sniff).  You have to be in the US, though.  I barely understand my own country's employment laws as it is.

What Is Next

Send me a resume, or your github profile, of Snapchat some code or something.  Just let me know if you're interested and we'll figure it out.

I'm open to part time, just returning to the work force, that sort of thing.  The mobile development market is red hot right now and I'm willing to be flexible.

Wednesday, June 3, 2015

Mapzen Vector Tile Service

Let's take a look at Mapzen's vector tile service.  They've recently announced a developer program and done a lot of work behind the scenes.

Sea levels aren't rising!  Shut up!

Vector Tiles

Mapzen is processing the OpenStreetMap planet file into nicely addressable vector tiles.  They provide the recipes for doing this yourself, but my attitude is better their Amazon account than mine.

Of the formats they offer, WhirlyGlobe-Maply supports two:  GeoJSON and PBF (aka Mapbox Vector Tiles).  I prefer the latter.

PBF (or MVT) contains compressed vector data with general attribution.  To make something visual, you have to match the attributes to a style.  In the past I've used Mapnik XML for this, but this time I've gone with something different.

Mapbox GL Style Sheet

Mapbox has been busily developing (yet another) styling format for their vector tiles.  The new format is nicely tuned to display rather than offline rendering.

The main disadvantage with the new format is there's no way to generate the damn things; you just have to edit the JSON directly.  I imagine we're all hoping they'll update Mapbox Studio one of these days.

OSM loves footpaths

But I like the format and it's clearly where this part of the open source community is going.  So I added a parser to WhirlyGlobe-Maply.

Mapzen Data + Mapbox Style + WhirlyGlobe-Maply

We've got the vector data from Mapzen, a style sheet in the Mapbox format and a big real time rendering toolkit.  How hard is it to put it all together?

Not hard at all.  There's a standard tiling loader (not shown) that will read it on the fly (in multiple threads).  We just have to set up the MapzenSource which involves..
  • Reading the MapzenGLStyle file.  It's in the local bundle.
  • Setting up the cache directory, so we're not always hitting Mapzen.
  • Specifying the base URL and API key for forming the Mapzen URL.
  • Picking the layers we want.  All of them in this case, but we could be selective.
  • Using PBF (MVT) for the source and Mapbox GL for the style.

Yup, it really is that simple.  I've been messing around with vector tiles for years now.  I've got it down.

This code is in the WhirlyGlobeComponentTester app in the develop branch on github.  I suggest using it in 2D mode.  Things get a little weird on the globe at lower resolutions.

Performance & Next Steps

It's pretty good on the device.  You could use this in a shipping product.  The style is a little bland, but it's fine for a demo.  Rendering performance is decent, but could be optimized.

Needs more labels
Mapzen's set things up so you can fetch individual layers.  I think it would be interesting to use, say the symbol and labels on top of one of NASA's live data sets.  Somebody go do that and let me know how it looks.

Saturday, May 30, 2015

State -o- The Business and Conferences Galore

I run a little consulting company called mousebird consulting inc. [yep, all lowercase]  We do development on and around the WhirlyGlobe-Maply toolkit.  This pushes new features in the toolkit and, ya know, pays the bills.

[Put gratuitous graphics here]

The toolkit brings in leads, I pursue them, clients sign contracts and invoices are paid.  It's a pleasant arrangement when it's chugging along.

So How's Business?

It's chugging along.  I logged my best month ever in January.  Then I did it again in February and April.  That was a ton of WhirlyGlobe-Maply work, some neat apps, and lots of great new features in 2.4.

The down side is my backlog.  I haven't had much time to track down problems for users who aren't clients, write new documentation, or make the Android version easier to install.

If you need some of my attention, the best way is a support contract.  I have a few of those now and I like how they're playing out.


Another downside of so much paying work is a dearth of presentations.  Good ones require time to write and I have no time.

Even so, I'll be on a panel for State Of The Map 2015 in New York next week.  I'll also be at WWDC in San Francisco the next week.  If you want to get together at either, drop me a line.