Vector Maps - Introduction

Here's the short version:  Vector Maps!  Yes we can!

Edit: Go check out Mapbox vector tiles and Thunderforest Lighting vector tiles too.

Somewhere in Brazil

WhirlyGlobe-Maply has had vector support for years.  As the versions went by that got much better, but vector maps were still a leap.  There were three missing pieces:

• Paging: Efficient loading of individual tiles.
• Styling: Specifying how a map looks.
• Workflow: Getting from raw data to pretty map (open source, 'natch)

I'd done a few experiments and I was pretty clear on how to solve this.  All I needed was clients with the money to get it done.  I found them, we did it, and it works!

There's a lot to cover, so let's look at the big picture.

TileMill

It all starts with TileMill, MapBox's excellent free map making tool.  With TileMill you set up your data and then style your map with CartoCSS.  It's fairly intuitive and, best of all, well documented.

Braaaaaziiiiiil!

Under the hood Mapnik is the open source workhorse for TileMill.  Mapnik takes the data for the map, the styling information and makes pretty map tiles.

Mapnik is tres cool, but we're not actually using it to do the work.

Mapnik XML

Okay, so we're not using Mapnik, but we are using its configuration file.  See, Mapnik has this great file format that specifies absolutely everything you could want in one place.

Yup it's XML.  XML is like reading a headache, but it's fantastic for data processing.  It's got references to data sources, like Shapefiles or PostGIS queries.  It's got styles laid out in glorious detail.  It's got everything we need.  And did you know you could get Mapnik XML out of TileMill? 

Bet you always pick MBTiles.

We style the pretty map in TileMill, we export the Mapnik XML and that goes directly into the next stage.

Carto Vector Dice

Here's where we go off the rails.  I've got a new command line program that does the rest of the work.

The vectors going into TileMill are bad for rendering.  We need things diced, sorted, and organized.  That's what this program does.  It sifts, chops, and organizes the data by style and level.

That's where it gets tricky.  CartoCSS and TileMill aren't really designed for vector maps and they cheat really well with image tiles.  Rather than carefully select the right data to go in, you can throw it all in and let priority sort it out.  Now, that's not entirely fair, it does slow down your tile construction, but it's a much bigger problem with vectors.

Well.... how about just the one 40MB tile?

carto_vector_dice can sort out much of that.  Referencing the same data multiple times is really common in CartoCSS.  We can catch this and use multiple styles for the same data.  Even so, picking the right detail for the right level is something of an art.

Once all the dicing, sorting, and grouping is done, carto_vector_dice writes all this glorious data out to a database.

Maply Vector Tiles

The vector tiles are stored in a sqlite database.  There's one tile per (x,y,level) and a table of style information.  The data for each tile is compressed and stored in a blob, encoding all the traditional vector types and attribution.

Water Table.  I'm hilarious.

Putting everything into one database makes it easy to move around, but we've also planned for the future.  The style table uses UUIDs so we can reference styles globally.  The tiles are self contained so fetching over the network will be a cinch.

Vector Map Rendering

All of this work has one goal:  Get the vectors to the renderer.  Maply can now render a vector map with the following code.

Seriously, that's it.  There's a lot going on under the hood, but it's all surprisingly rational.  And it works great.

Insira espirituoso legenda aqui

One thing I gotta point out here.  We're not rendering map tiles and drawing the images.  We're Drawing.  The Actual.  Vectors.

Look, smartphones can handle this.  They've been able to for years now.  There's no reason not to render the vectors in a lot of cases.

I Want It!

So can you use it?  Well, in theory you can.  If you're a developer and you're familiar with GIS and a bunch of other stuff, probably.  If you're not, I would wait a bit.

I use this stuff with clients and I'm very happy with it, but it's got some rough edges.  Every new CartoCSS style is an adventure.  If you try it yourself, be patient and maybe drop me an email.

Future Stuff

The big, risky part is done and works.  There's plenty more to do, though.  I've got a long, long list of things clients may want.  If you want one of them, you could be a client too!

So that's it.  Vector maps work in Maply.  The workflow is in place and it's open source.  Look for improvements in the future and, of course, I'll have more to say on the blog.