Advertisements

Slippy maps, meet before-and-after jQuery slider (introductions by OpenLayers)

Our team at the Center for Urban Research (at the CUNY Graduate Center) has launched a set of maps showing race/ethnicity patterns from 2000 and 2010 in major cities across the US.  The maps combine several mapping/web technologies that offer a new way of visualizing population change.  This post explains how we did it.

(And by popular demand, we’ve also included a map of Congressman Anthony Weiner’s district in relation to demographic change — you may have heard of him and his Twitter travails recently?)

Race/Ethnicity Change

Briefly, the maps show race/ethnicity change from 2000 to 2010 at the local level throughout major urban regions across the U.S.  So far we include New York City, Los Angeles, Boston, Chicago, Houston, and San Francisco.  (Others are coming soon.)

For our methodology and data analysis (and static maps), we provide that here.  For the mapping and web techniques, see below.

Reactions

So far we’ve received a pretty good response to our maps.  Here are some tweets posted recently:

  • @dancow (web journalist for ProPublica): Cool before/after map from CUNY’s urban research center showing NYC ethnic changes at the block level, from 2000-10.
  • @mericson (deputy graphics editor at NY Times): Nice block-level maps by @SR_spatial & CUNY Urban Research Center showing racial/ethnic change in NYC from 2000 to 2010.
  • @kelsosCorner (former Washington Post cartographer): Digging new 2010 Census plurality maps of NYC.
  • @albertsun (graphics editor at Wall St Journal): Coolest census map I’ve seen yet.
  • @PJoice (HUD employee; tweets are his own): This is the coolest map I have ever seen. Nice work by @SR_spatial and CUNY!
  • @MapLarge: I like how you can use the slider or move the map! Great Visualization!

Technical overview

The map uses the “before and after” technique that media websites have used for images of natural disasters.  We enhanced this technique by integrating it with interactive maps using OpenLayers, the open source mapping framework.  Now the slider works with two sets of overlapping, but perfectly aligned, maps from 2000 and 2010.

As it turns out, we didn’t set out to create an interactive version of these maps. In fact, we originally created static maps, but everyone we showed them to for feedback wanted the ability to zoom in/out and click on the map for more info.  So we developed the OpenLayers version. (And when I say “we”, that mainly means David Burgoon, CUR’s application architect, who I can’t say enough good things about.  I made the maps, and CUR’s Joe Pereira of the CUNY Data Service created the data sets, but Dave brought it all to life.)

OpenLayers enables us to introduce interactivity into the before-and-after images. Maps like these (to our knowledge) have not been available before — where you can move a slider back and forth while also zooming in/out and clicking on individual Census blocks for detailed information. You can also change the transparency of the thematic map layer, and switch between a street view and aerial view basemap.

It involved a good amount of work to integrate the slider technique with OpenLayers and also have two overlapping map instances working in tandem. The two maps need to appear as one, and this involves painstaking effort to ensure that the pixels on your screen are translated accurately to latitude/longitude coordinates in each of the separate but related interactive map instances, and the maps pan together seamlessly as you drag the slider left or right or move the map and it crosses the slider.

Mashup

In order to create the application, we used a mix of software applications, technologies, and techniques, summarized below:

  • We used the statistical software package SPSS to extract the Census block-level data for both years (see our methodology), allocate the 2000 data to 2010 blocks using the Census Bureau’s block equivalency files, and calculate the race/ethnicity plurality for each block.
  • We exported these SPSS files in DBF format and used ESRI’s ArcGIS Desktop to join the DBFs with 2010 TIGER Census block shapefiles.
  • ArcGIS Desktop was also used to create the choropleth maps (based on color schemes from ColorBrewer.org);
  • The map layouts were published as temporary web map services using ESRI’s ArcGIS Server. We used these to create pre-cached tiles (.PNG files) for the 2000 and 2010 maps, corresponding to zoom levels 4 through 10 using the now-standard Google-Microsoft map scales for online web maps. (Our application accesses the choropleth tiles as PNGs directly from the cache created by ArcGIS Server, rather than accessing the ArcGIS web map service in order to assemble the tiles. The latter approach would be too slow and would undermine the transition as you dragged the slider across the map.)
  • The slider technique was adapted from the jQuery plugin by www.catchmyfame.com.
  • OpenLayers provides all the map navigation and serving the maps themselves, modified with customized JavaScript code.
  • The basemap shown beneath the color-shaded map tiles is provided by Microsoft’s Bing map service. The street map and aerial image tiles from Bing are accessed directly via OpenLayers, rather than using the Bing API. This is a key reason we used Bing for these maps; if we used Google Maps as a basemap, we were limited to accessing Google Maps via Google’s API, which would have slowed map drawing times and undermined the slider effect.
  • For geocoding we use the Yahoo! Placefinder API.
  • Some browsers are not able to handle the before/after slider effect smoothly. In particular, Firefox and Safari perform poorly; the slider transition between one map to the other is not smooth. Microsoft’s Internet Explorer is adequate, but Google’s Chrome browser is best.

Data sources/issues

We used block-level data from the Census Bureau’s 100% population counts from the 2000 and 2010 decennial censuses (from Table P2 in the “PL-94-171” files for 2000 and 2010).

The Census Bureau’s block geography changed between 2000 and 2010 — new blocks were created, blocks were merged, and block boundaries were modified in many places. In order to compare population data from 2000 and 2010 using a common set of blocks, we used the Census Bureau’s block relationship file to allocate the 2000 population counts to 2010 geography.

When you’re viewing the map, it is best to use the maps and block-level data to understand trends over a larger area, even over several blocks. Be careful when viewing a specific block on its own. It covers a small area, and the Census Bureau may have made errors.

Credits

Funding for much of the Center’s recent work on Census issues has been provided by the Building Resilient Regions Project of the John D. and Catherine T. MacArthur Foundation, the Hagedorn Foundation, as well as support from the CUNY Graduate Center and the City University of New York.

Several people provided feedback and helpful editorial suggestions on earlier versions of the maps and narrative. Though the materials at this site were prepared by the Center for Urban Research, those invdividuals improved our work. We greatly appreciate their contributions.

Advertisements

GIS and Census participation

It’s been too long since my last blog post. Have been quite busy with work, and even though Twitter is a microblogging service, sending a tweet now and then really isn’t an excuse to keep up my actual blog.

One of the projects keeping me (very) busy is our work to help boost participation in the 2010 Census. I thought I’d write about some of our interactive mapping and participation rate analysis along these lines.

In August I described how our team at CUNY’s Center for Urban Research was creating metro-scale maps showing where hard-to-count communities were located so local census advocates could target their outreach. Then in late January we launched our interactive version of those maps at www.CensusHardToCountMaps.org. Originally we designed the site to show hard-to-count areas, but these are only where it was expected there’d be low census participation. Then, on March 23, the Census Bureau started publishing the actual participation rates on a local and national basis. So a week later we updated our site to emphasize the latest participation rates (this link zooms in to Manhattan showing tract-level participation overlaid on a map of hard-to-count tracts).

Though the Census Bureau’s Take 10 map (and a related Google Earth application) display the daily participation rates nationwide, we decided to provide several features that the Census Bureau doesn’t. At our site you can:

  • type in a county and highlight the tracts below a certain participation rate (you can enter whatever rate you want);
  • sort the resulting list so you can see at a glance the highest and lowest performing tracts (this also will be highlighted on the map so you can see how concentrated they are); and
  • compare the 2010 rate map with the 2000 rate map (click the “More…” tab and check the box for “Participation Rate in 2000″).

(Of course, you can also click on any spot on the map to display the latest participation rate for that area — state, county, or tract — depending on how close in or out you’ve zoomed.)

These are the types of data analysis and spatial visualization tools that were requested by census advocates, so they can use the maps to focus on areas that need their help the most.

In order to provide some context for the interactive map, our Center also posted an analysis of the first week’s participation rate. It was a combination of basic statistical analysis and mapping. We examined the correlation between participation rate and hard-to-count scores at the tract level nationwide, and not surprisingly found that rates tended to be lower in hard to count areas. This should help bolster the work of groups who’ve been working in these communities, confirming that they’re focused on areas that need support the most if we want to achieve a 100% count.

We also examined county-level statistics on race/ethnicity using the Census Bureau’s latest population estimates from 2008. (The American Community Survey would provide a richer set of characteristics to examine, but any data from areas with less than 20,000 population are suppressed in the ACS — and this accounts for about 1,300 of the nation’s 3,200 counties.)

The county-level data indicate that race/ethnicity is strongly correlated with census participation (at least in the first week), with participation rates tending to be higher in counties with a greater percentage of whites while the rates tended to be lower in counties with a greater percentage of blacks and Latinos. Because we didn’t have other socio-economic data to evaluate, we weren’t able to disentangle the effects of other characteristics such as low educational attainment, poverty, housing conditions, etc. that may have a stronger correlation while cutting across racial and ethnic categories. An opportunity for further research. As a next step we may also examine county-level unemployment rates and participation rates, as well as evaluating how well the first week’s analysis holds up as time goes on.

Friday (April 2) we added another feature, information about the areas that will receive a second census questionnaire.  (The Funders Census Initiative sent out a news advisory highlighting this service on April 5.)  Now when you click on the www.CensusHardToCountMaps.org map or type in your street address, you’ll see a popup window that (among other things) tells you if households in your tract will be receiving replacement Census forms. We think this will help minimize confusion over people receiving another census form (even if they’ve already mailed their’s in!). This is a “just in case” thing from the Bureau — mailing another form to households in historically low response areas, and mailing another form to households in moderately low response areas who haven’t yet sent their’s in. But the geographic scope of the “blanket” and “target” replacement mailing areas are pretty extensive in most cities (see maps at CUR’s website), so lots of people may be confused. Our mapping site provides a simple way of clearing the air.

We’ve also mapped those second mailing areas. When you visit www.CensusHardToCountMaps.org, select the “More…” tab and zoom in to your area of interest. For example, here’s Boston, MA. Click either or both check boxes in the “April 2010 Replacement Questionnaires” section to map the tracts receiving replacement census forms.

Our hard-to-count mapping site still has its original functionality — such as visualizing the demographic characteristics that will make it difficult to achieve a complete Census count; overlaying ZIP Codes, mail return rates from 2000, recent foreclosure risks by tract; and seeing who’s tweeting about the Census in your area.

But we’re also planning for next steps, thinking of the mapping application as a platform for future Census-related efforts (tracking how successful census advocates were, displaying the 2010 results, enabling the general public to get involved in a meaningful way in the redistricting process). Any ideas? We’d love to hear them.

Mapping “hard to count” areas for 2010 Census

UPDATED 8/21/09: Newsday (Long Island’s daily newspaper) reproduced an island-wide version of one of our maps in their article today (though the map only appeared in the print version of the paper).

People are gearing up across the US for the 2010 Census — not just the Census Bureau, of course, but organizations large and small who are planning myriad outreach efforts to boost participation, especially in typically “undercounted” communities.

It’s important in so many ways for everyone to be counted, but historically not everyone is (and not just because of statistical anomalies or poor street address data).  For various reasons, key constituencies are not fully counted — people of color, renters, recent immigrants, people predominantly speaking languages other than English, etc.  There’s a special effort underway – supported by major foundations, local governments, and spearheaded by advocacy and civil rights groups – to make sure the Census Bureau doesn’t miss these “hard to count” groups.

Our teamat the Center for Urban Research, CUNY Graduate Center is doing its part by mapping the “hard to count” (HTC) population in more than two dozen metropolitan areas nationwide, with the support of the Hagedorn Foundation.  We’ll also be developing an online, interactive mapping application for funders, outreach groups, local officials, the media, and others to easily zoom in on their metro region and create custom maps to help focus their efforts.

Where are the hard-to-count communities?  The Census Bureau analyzed 2000 Census data to:

measure census coverage and to identify reasons people are missed in the census (de la Puente, 1993). The variables include housing indicators (percent renters, multi-units, crowded housing, lack of telephones, vacancy) and person indicators (poverty, not high school graduate, unemployed, complex households, mobility, language isolation).

From this, the Bureau calculated an HTC score for each tract in the nation (on a scale of 0 to 125).  They estimate that any tract with a score of 60 or higher will be at the greatest risk of undercount in 2010.  Our metro maps for the 2010 outreach campaign highlight the concentrations of HTC tracts, as well as the 2000 Census response rate by tract.  Here are two examples, for Chicago and New York City:

Chicago_HTC_2010Census

NYC_HTC_2010Census

Foundations are using these maps to help guide their grantmaking, and local groups are using the maps to target their outreach.  As an example, this month several Long Island-based foundations issued an RFPfor grants to local groups doing Census outreach.  The Hagedorn Foundation asked us to create a special set of Long Island maps to supplement the RFP – you can view the PDFs here and here(see below for a sample – these mapped the HTC scores relative to all tracts in Nassau, so some tracts with scores less than 60 — but still harder to count than others in the county — are shaded as hard to count).

Nassau_HTC_2010Census

Next step for us is to transform these printed maps into a nationwide online mapping website.  The site will enable people to: 

  1. Zoom to their neighborhood, county, city, or state to see the mapped patterns of hard-to-count Census tracts;
  2. Click on individual Census tracts to display detailed information about each one;
  3. Add other layers of Census demographics, 2000 Census response rates, related data from the 2005-07 American Community Survey, and recent foreclosure risks circa 2007-08 so people can see the interrelationships among multiple variables; and
  4. Display state- and local-level resources such as funding opportunities, regional Census contacts, contact info for groups participating in Census outreach, etc.  

We’ve been reviewing other projects that are similar so we don’t duplicate efforts.  These include:

A key question for us is which basemap do we use?  We’ll be serving the tract-level geography and attributes (probably as cached tiles) from our servers using ArcGIS Server on the backend, OpenLayers and ExtJS on the front end — a combination that has served us well (see here and here).  Likely we’ll use Google Maps (and/or their Hybrid or Terrain views) for the underlying street/reference geography.  But perhaps OpenStreetMap would be a better choice?  Or Bing Maps?  Advice from the GeoWeb community would be appreciated.

Timing? We’re shooting for a beta site in late September, seeking feedback from partner groups in the fall, and a production-level site by the end of the year.