Advertisements
    Advertisements
  • SR_spatial tweets

    Error: Twitter did not respond. Please wait a few minutes and refresh this page.

  • Advertisements

Slippy maps, meet before-and-after jQuery slider (introductions by OpenLayers)

Our team at the Center for Urban Research (at the CUNY Graduate Center) has launched a set of maps showing race/ethnicity patterns from 2000 and 2010 in major cities across the US.  The maps combine several mapping/web technologies that offer a new way of visualizing population change.  This post explains how we did it.

(And by popular demand, we’ve also included a map of Congressman Anthony Weiner’s district in relation to demographic change — you may have heard of him and his Twitter travails recently?)

Race/Ethnicity Change

Briefly, the maps show race/ethnicity change from 2000 to 2010 at the local level throughout major urban regions across the U.S.  So far we include New York City, Los Angeles, Boston, Chicago, Houston, and San Francisco.  (Others are coming soon.)

For our methodology and data analysis (and static maps), we provide that here.  For the mapping and web techniques, see below.

Reactions

So far we’ve received a pretty good response to our maps.  Here are some tweets posted recently:

  • @dancow (web journalist for ProPublica): Cool before/after map from CUNY’s urban research center showing NYC ethnic changes at the block level, from 2000-10.
  • @mericson (deputy graphics editor at NY Times): Nice block-level maps by @SR_spatial & CUNY Urban Research Center showing racial/ethnic change in NYC from 2000 to 2010.
  • @kelsosCorner (former Washington Post cartographer): Digging new 2010 Census plurality maps of NYC.
  • @albertsun (graphics editor at Wall St Journal): Coolest census map I’ve seen yet.
  • @PJoice (HUD employee; tweets are his own): This is the coolest map I have ever seen. Nice work by @SR_spatial and CUNY!
  • @MapLarge: I like how you can use the slider or move the map! Great Visualization!

Technical overview

The map uses the “before and after” technique that media websites have used for images of natural disasters.  We enhanced this technique by integrating it with interactive maps using OpenLayers, the open source mapping framework.  Now the slider works with two sets of overlapping, but perfectly aligned, maps from 2000 and 2010.

As it turns out, we didn’t set out to create an interactive version of these maps. In fact, we originally created static maps, but everyone we showed them to for feedback wanted the ability to zoom in/out and click on the map for more info.  So we developed the OpenLayers version. (And when I say “we”, that mainly means David Burgoon, CUR’s application architect, who I can’t say enough good things about.  I made the maps, and CUR’s Joe Pereira of the CUNY Data Service created the data sets, but Dave brought it all to life.)

OpenLayers enables us to introduce interactivity into the before-and-after images. Maps like these (to our knowledge) have not been available before — where you can move a slider back and forth while also zooming in/out and clicking on individual Census blocks for detailed information. You can also change the transparency of the thematic map layer, and switch between a street view and aerial view basemap.

It involved a good amount of work to integrate the slider technique with OpenLayers and also have two overlapping map instances working in tandem. The two maps need to appear as one, and this involves painstaking effort to ensure that the pixels on your screen are translated accurately to latitude/longitude coordinates in each of the separate but related interactive map instances, and the maps pan together seamlessly as you drag the slider left or right or move the map and it crosses the slider.

Mashup

In order to create the application, we used a mix of software applications, technologies, and techniques, summarized below:

  • We used the statistical software package SPSS to extract the Census block-level data for both years (see our methodology), allocate the 2000 data to 2010 blocks using the Census Bureau’s block equivalency files, and calculate the race/ethnicity plurality for each block.
  • We exported these SPSS files in DBF format and used ESRI’s ArcGIS Desktop to join the DBFs with 2010 TIGER Census block shapefiles.
  • ArcGIS Desktop was also used to create the choropleth maps (based on color schemes from ColorBrewer.org);
  • The map layouts were published as temporary web map services using ESRI’s ArcGIS Server. We used these to create pre-cached tiles (.PNG files) for the 2000 and 2010 maps, corresponding to zoom levels 4 through 10 using the now-standard Google-Microsoft map scales for online web maps. (Our application accesses the choropleth tiles as PNGs directly from the cache created by ArcGIS Server, rather than accessing the ArcGIS web map service in order to assemble the tiles. The latter approach would be too slow and would undermine the transition as you dragged the slider across the map.)
  • The slider technique was adapted from the jQuery plugin by www.catchmyfame.com.
  • OpenLayers provides all the map navigation and serving the maps themselves, modified with customized JavaScript code.
  • The basemap shown beneath the color-shaded map tiles is provided by Microsoft’s Bing map service. The street map and aerial image tiles from Bing are accessed directly via OpenLayers, rather than using the Bing API. This is a key reason we used Bing for these maps; if we used Google Maps as a basemap, we were limited to accessing Google Maps via Google’s API, which would have slowed map drawing times and undermined the slider effect.
  • For geocoding we use the Yahoo! Placefinder API.
  • Some browsers are not able to handle the before/after slider effect smoothly. In particular, Firefox and Safari perform poorly; the slider transition between one map to the other is not smooth. Microsoft’s Internet Explorer is adequate, but Google’s Chrome browser is best.

Data sources/issues

We used block-level data from the Census Bureau’s 100% population counts from the 2000 and 2010 decennial censuses (from Table P2 in the “PL-94-171” files for 2000 and 2010).

The Census Bureau’s block geography changed between 2000 and 2010 — new blocks were created, blocks were merged, and block boundaries were modified in many places. In order to compare population data from 2000 and 2010 using a common set of blocks, we used the Census Bureau’s block relationship file to allocate the 2000 population counts to 2010 geography.

When you’re viewing the map, it is best to use the maps and block-level data to understand trends over a larger area, even over several blocks. Be careful when viewing a specific block on its own. It covers a small area, and the Census Bureau may have made errors.

Credits

Funding for much of the Center’s recent work on Census issues has been provided by the Building Resilient Regions Project of the John D. and Catherine T. MacArthur Foundation, the Hagedorn Foundation, as well as support from the CUNY Graduate Center and the City University of New York.

Several people provided feedback and helpful editorial suggestions on earlier versions of the maps and narrative. Though the materials at this site were prepared by the Center for Urban Research, those invdividuals improved our work. We greatly appreciate their contributions.

Advertisements

OASISnyc.net map updates

Today our team at the CUNY Graduate Center updated the www.OASISnyc.net mapping site with lots of new data.  There’s more to come by summer’s end, but here’s the latest:

  • The biggest change is that we’ve added the latest community garden inventory from GrowNYC.  Over a year in the making, it comes just in time to provide valuable context for the proposed new rules for gardens being considered by the city.  More info at the OASIS wiki.
  • We’ve integrated the latest subway and bus data that I’ve blogged about earlier (here and here), and also added bike routes via the NYC Dept of Transportation (one of the latest city agencies embracing #opendata).

(Lots of bike routes in Brooklyn.)

  • A side note to the subway routes is that we’ve responded to user requests and added the ability to turn on/off the subway routes separately from the streets and rail lines.  (May not seem like a lot, but hopefully this will make it easier to view an already busy map.)

  • We also have gotten around to updating New York City parks and playgrounds data based on the Datamine release from last year.  I’ll blog more about this separately, since it was a much more involved process than it should have been, given what I now realize are some major limitations to the city’s effort via Datamine and BigApps to provide open access to data.
  • The latest zoning districts, scenic districts, and historic houses are also added to OASIS.
  • The search options for community gardens, Stew-Map turfs, and the NY/NJ Harbor’s Comprehensive Restoration Plan data have been updated.
  • We’re starting to use the OASIS wiki again for tutorials and examples of how OASIS can benefit local groups. For examples, see http://oasisnyc.gc.cuny.edu/index.php/Featured_Maps
  • Finally, we’ve started to incorporate historic maps of Manhattan in partnership with the NY Public Library.  I blogged about this earlier, and we have a post on our wiki with more info.

We’ll have more updates soon.  Historic land use parcel by parcel citywide (with an accompanying statistical analysis), more data (such as parcels & open space) for northern New Jersey, more Google Earth/KML links, etc.

Stay tuned!

@NYPLMaps & OASIS provide context for 18th century ship find

The www.OASISnyc.net mapping team has been working with the great folks at New York Public Library’s Map Division to integrate digitized historic maps aligned to the city’s current street grid.  But as we were working with Map Division staff to incorporate their maps, an amazing find at the World Trade Center construction site prompted us to speed up our work — earlier this month, construction workers unearthed an 18th century ship, largely intact, that likely hadn’t been disturbed for over 200 years.

Now you can display some key maps of lower Manhattan from the from the 18th and 19th centuries, view them in relation to the current street grid, and compare them to each other using OASIS’s dynamic transparency tool.  We added a brief tutorial at the OASIS wiki.

Now you can fade between current property maps …

… the 1775 Montresor map …

… the 1817 Poppleton map …

… and more.

We’ve also added the Viele map from 1874, and more are on their way.  This is all due to the groundbreaking NYPL “Map Rectifier” project.

Thoughts about Google’s Styled Maps announcement

At the Google I/O developer event this week, an impressive item was included in the announcement about the latest Javascript Maps API (v3).  Now you can use Javascript to change the style of Google’s base map, in effect creating your own CSS for Google Maps.

Yesterday’s Google Geo Developers Blog highlighted the new feature and capabilities, including a link to a style wizard – see screen shot below – so you can see in real-time what each change looks like (the wizard is kind of like ColorBrewer, without the helpful suggestions for good vs. bad color schemes and symbology).  Note that the Style Wizard won’t work in IE.

Google Maps Style Wizard

This step for Google Maps is big, no doubt, but I’m ambivalent.  On the one hand, it opens up myriad possibilities for using Google’s map API.  Before, some geoweb developers (our team at CUNY included) tended not to use the Maps API because each and every map looked the same.  It got to be boring, and from a cartographic perspective it was basically an acknowledgment that you took the easy way out.  And often the Google map style got in the way of other layers you wanted to add to your map.

As Google’s own blog puts it,

No matter which [Google] Maps API site you are on, every map looks the same. If you want your map to stand out from the crowd, your options are limited to customizing the markers and controls, and if your brand has a particular colour scheme that is reflected on your site, Google Maps may not sit well with it.

EveryBlock’s Paul Smith wrote about these limitations two years ago at his post about “Google Maps Fatigue”.  For our part at CUNY, if we wanted a basemap with our own style that didn’t conflict with other thematic layers and choropleth overlays, either we rolled our own.  Or if we used Google Maps, we used the Flash API so we’d have the ability to desaturate its color (following the New York Times and others) — our Census 2010 Hard to Count mapping site used this technique. 

On the other hand, I’m sure more of us will now use Google Maps as a basemap, but will that further undermine the diversity of locally-crafted and customized online cartography?  With the advent of Google map styles, our team is now looking closely at GMaps for our map applications.  I think we’ll likely only use Google’s basemap for nationwide applications where it’d be too difficult to assemble all the base map layers at that scale.  But … maybe we’ll use it beyond that.

There are still major issues, of course.  Modifying Google’s map styles is one thing, but it’s still the Google basemap with the Google logo.  Using Google’s basemap whether we style it ourselves or not still means we’re stuck with the underlying data — with all its errors, inconsistencies, outdated place names, etc.  Google is trying to fix the errors, and errors are bound to exist, but until the basemap gets to a high level of quality and consistency, in effect you’re endorsing the errors when you use their basemap.  And though we can use the GMap style options to turn on/off place names, for example, this is nowhere near as robust as setting layer-specific filters that display or omit only *certain* place names, or using different road label types for different types of roads, etc.

As I understand it, CloudMade gives you much more control over map content as well as style.  You need to use OpenStreetMap, but perhaps the openness of OSM is more to your liking than Google Maps.  This post at Programmable Web highlights some of the differences between GMaps map styles and CloudMade’s editor features.

Some critics also point out that just making a map look different doesn’t necessarily mean it looks good, or better than the standard Google look-and-feel.  The examples provided via Google’s Geo Developer blog are intriguing …

… but this one via the Javascript Maps v3 documentation is, well, not so much:

Perhaps one cartographic benefit in all this is that it will help the mashup neogeographers of the world realize that there are good and bad map styles — cartography matters.  Web map developers will need to make choices about symbology, it won’t just be on a silver platter from Google.

What do you think?

“Neo”, “Paleo”, “Geo”, what?

Just a quick note that the online journal V1 Magazine today posted an interview with me about the CUNY Mapping Service’s online mapping work.  I’m honored that Matt Ball asked to talk with me about our work — he has interviewed some pretty impressive people over the years and it’s a thrill to have our humble contributions included in the list.

We mainly talked about our experiences developing some big online mapping applications — the Long Island Index interactive map, the new version of OASIS, and an upcoming nationwide application for the 2010 Census.

I’m glad one of Matt’s takeaways from the interview is to think of our work as cutting through the debate/discussion/tempest in a teapot that’s been taking place lately about whether you’re a “neo” or “paleo” or some other type of geographer.  I can’t say that we have any definitive answers or better answers than anyone else.  But if our work helps clarify things or maybe even point the discussion in a new direction, that’d be a contribution I’m glad to make.

Btw, my slide presentation from the 2009 GeoWeb conference provides some background on how we decided on different GIS technologies/techniques for the Long Island Index mapping site.  And an earlier blog post discusses our cartographic decision-making for the new OASIS website.

Homage to the people behind OASISnyc

Last week’s UrbanOmnibus features an article I wrote about a new and completely revamped version of the OASIS mapping website in New York City — see “A new OASIS for New York“.  (Also see an earlier blog post about how we designed the cartography in the new OASIS maps.)

OASIS is the Open Accessible Space Information System.  The UO piece focuses on the site’s new mapping tools, richer data sets, and a new approach using “web 2.0″ techniques to encourage more interaction and engagement via the OASIS maps with urban planning in New York.

But equally impressive, though not the focus of the piece, is the part of OASIS that doesn’t directly involve mapping and web technology.  It’s the people and groups “behind the scenes” that make it all worthwhile.  Their work and the collaborative effort that OASIS has helped facilitate are really amazing.

OASIS was the vision of several people in the Forest Service (Jim Lyons, Michael Rains, Phil Rodbell, Matt Arnn, Robin Morgan to name a few), ESRI (especially Dave LaShell), and local greening organizations in New York back in late 2000.  The idea was to bring together a bunch of groups interested in sharing resources and ideas about open space stewardship, create an online mapping site to integrate all this info (way before Google Maps was on the scene), and see where that would lead.

The OASIS mapping site is powerful, but the website without the people and partnerships would just be one more (though impressive) map mashup.  The collaborative nature of the effort from the start — inspired and sustained especially by Matt Arnn — always seemed special.

Some folks who deserve special mention (though this is certainly an incomplete list) are:

  • Council on the Environment of NYCLenny Librizzi at CENYC is absolutely wonderful, and has a great vision of involving students and community groups in the greening of the city.  He trains high school youth to inventory street trees in a way that teaches ecology, math, science, and urban planning in an engaging way.  He’s also led the effort to map community gardens across the city, maintaining a comprehensive database of gardens for analysis, advocacy, and teaching.  It’s been great working with Lenny as we’ve integrated all that data into OASIS.  The maps are powerful, but in some ways it’s more important that they’ve been a vehicle to get to know him and to give his students invaluable hands-on experiences.
  • Open Road of New York  — Paula Hewitt Amram is an inspiration to untold numbers of city youth who want to change the world or are just looking for a better spot to play in.  (Here’s an example: Amy Poehler interviewing 11-year old Valentine about her community gardening work at Open Road.)  She’s brought that energy and spirit to the groups involved in OASIS, with a real passion for wanting the collaborative effort to endure so it will continue to be an educational and participatory resource. 
  • The greening groups involved in GROW (Grassroots Reassuring OASIS Works) – they coordinated a series of focus groups early on to make sure that whatever online mapping tools were created, were developed with an eye toward the neighborhood organizations and activists who needed the information the most.  Their insights continue to resonate with the OASIS participants, even as the GROW coordinators Wendy Brawer and Hugh Hogan have gone on to do other amazing things.
  • The Forest Service team at its Northern Research StationErika Svendsen has consistently pointed out that maps of buildings and park boundaries are nice, but maps also need to convey a sense of who’s doing what where: the people and their activities in any given geographic area are obviously and critically important.  This interest enhanced the Forest Service’s Living Memorials Project, and simultaneously was a key theme in discussions among OASIS participants about the need to understand local environmental stewardship (who’s doing what environmental work where).  Eventually Erika, Lindsay Campbell, and the Forest Service’s NYC Urban Field Station turned this concept into reality with the unique Stewardship Mapping and Assessment Project (STEW-MAP).  Erika and Lindsay and their colleagues provide a refreshing perspective to us online cartographers, and hopefully this will also become obvious to anyone using the new OASIS maps when they view the STEW-MAP “turfs” that are displayed.
  • ESRIDave LaShell helped develop OASIS early on and has been a consistent booster.  Though Dave works for a software company, I think in another life he’d be an environmental and community organizing visionary. ESRI has provided substantial software and technical support resources to OASIS from the beginning.  But Dave always saw far beyond that, understanding intuitively that bringing people together was the important thing, not any particular technology or software package these people happened to be using.  He also does that in his ongoing efforts at ESRI’s NYC office, and in extracurricular work — for example, he volunteers with the Academy for Urban Planning, which has the only high school level GIS program in the city.  Dave epitomizes what I said about ESRI when Matt Arnn and I accepted the Municipal Art Society’s “Certificate of Merit” award for OASIS in 2001, that it’s a private company with a public conscience.
  • Individuals like Jane Sokolow, Bob Alpern, and Jack Eichenbaum – I’ve known Bob and Jack for years, and more recently Jane, and it’s been wonderful and enriching working with them together on OASIS.  Also, my colleague Christy Spielman was involved with OASIS at the start, designing the maps and managing the data and website.  But she’s been critical at coordinating and facilitating the various organizational partnerships that have developed through OASIS; those contributions have been as important as her GIS & graphic design skills.  And Dave Burgoon, who’s a dream to work with, has used his programming skills to make the new OASIS website worthy of this recent praise from a financial blogger at Reuters: “This is the most amazing map of NYC I’ve ever played with. Just, wow.”
  • Last but not least, the Forest Service itself.  When I first heard that the Forest Service was convening a meeting in 2000 to talk about mapping the city’s “urban canopy” (i.e., trees), I was skeptical.  I was most familiar with their work managing land in the western US or fighting forest fires, not working in cities, let alone New York.  But even then they had developed a strong argument for caring about trees and open spaces in urban areas – both for environmental reasons as well as for the ability to relatively easily engage large numbers of people in these densely populated areas.  Phil, Matt, Robin, et al. provided great and consistent support for OASIS in its early years and still value the benefits that this collaborative effort has provided.  It’s been terrific getting to know them and work with them.

Of course there are too many people and groups to mention in detail, so please visit the OASIS participants page to see the full list.  (And even on that page we’ve probably missed a few.) 

Although active participation in the steering committee has ebbed and flowed (it’s probably at a low point at the moment, with most of the effort going toward the website redesign), you can still get involved.  Here are some links with more info:

All in all, OASIS has made its impact felt – in me and my professional and personal relationships, in the work of many people and groups across the city, and hopefully beyond all that to the city and metropolitan region at large.  I’m thrilled to have had the opportunity to have been (and to continue to be) a part of it all.