• SR_spatial tweets

    • RT @dcvanriper: There it is. The official announcement that Census will cut short efforts to obtain a full and accurate count of the popula… 11 hours ago
    • RT @tribelaw: “The Trump administration is doing everything it can to sabotage the 2020 Census so it reflects an inaccurate and less divers… 11 hours ago
    • RT @vanitaguptaCR: NEW: Sharing my ⁦⁦@washingtonpost⁩ oped on how the Trump administration is once again trying to sabotage the census to… 11 hours ago

NYS Congressional districts & eligible voters mapped

Just three months after the US District Court approved the redistricted Congressional districts for New York, the state is holding primary elections for Republican and Democratic candidates for Congress and US Senate.

In an effort to help analysts understand the voting patterns for the primary and general elections for Congress in New York, our Center for Urban Research at the CUNY Graduate Center has updated a map of Congressional districts to highlight the differences between eligible voters by district and overall population counts.  The map is accessible at www.urbanresearchmaps.org/nycongress2012/map.html

Among other things, our new map displays the “citizen voting age population” (CVAP) estimates for each district as well as overall population counts.  When you visit the map, move your mouse over each district to display total population counts by race/ethnicity along with “citizen voting age population” (CVAP) estimates.   We also have a brief analysis of CVAP estimates compared with total population for each district.

In many Congressional districts, especially in New York City, news reports have noted the changing demographics, partly due to population shifts but also due to new boundaries that are the result of redistricting.  Districts that may once have had a predominantly Black population, for example, may now have a more mixed population.

But overall race and ethnicity population counts only tell part of the story.  Population data from the decennial 2010 Census include all residents — citizens as well as recent immigrants who may not yet be citizens, and people who are of voting age (18 or older) as well as children.  In some cases, the eligible voting population has a much different racial and ethnic profile than the overall population.

Welcome to 1940s New York

Now that data on an individual basis is available from the 1940 Census, our Center for Urban Research at the CUNY Graduate Center has launched Welcome to 1940s New York. The website is based on a 1943 “NYC Market Analysis” rich in local maps, photos, data, and narrative, providing a rare glimpse into life in New York City during that time.

We’re making this available both as context for the 1940 Census information, and for researchers and others generally interested in learning about New York in the ’40s. The New York Times has also published an article about the project, highlighting some then-and-now photos and demographic statistics of selected neighborhoods across the city.

My post below provides some background about how we came to develop the website. It also highlights some of the more intriguing things you’ll find there.

Piquing a graduate student’s interest

In 1997 the New York Bound bookstore was going out of business. I was a graduate student at Columbia University’s urban planning program, immersed in learning about all things New York. Of course, the bookstore’s sale was a must-visit event.

The bookstore was full of fascinating items, but most were either too expensive or too arcane for my interests. But one item fell right in the middle: not too pricey (the $100 was worth it, given the contents) and absolutely captivating, especially for someone like me who was also immersed in learning about computer mapping at the time.

The document was a New York City Market Analysis, published in 1943. Inside the cover the bookstore staff had written “Scarce Book”. I leafed through it and was amazed at the color-coded maps of every neighborhood in the city, visualizing down to the block what each area was paying in rent at the time. Each of 116 neighborhood profiles also included statistics from the 1940 Census, a narrative highlighting key socio-economic trends at the local level, and a handful of black & white photos.

My “Aha!” moment

I knew the document would come in handy one day. But once I bought it, it pretty much just sat idle on my shelf. That is, until earlier this year when news of the 1940 Census data coming online started to pick up steam. Lightbulb! If we could republish the 1943 Market Analysis, it would provide context for the individual 1940 data, and the 1940 Census would be a great hook to focus attention on this incredible historic resource documenting city life from that era.

The 1943 document was copyrighted. But copyright law as subsequently amended required copyright owners to explicitly renew the copyright within 28 years or forego rights to the material. In this case, the 28 year period ran to 1971. With the help of CUNY’s legal team and others, we determined that the copyright was not renewed. The 1943 document is in the public domain.

Welcome to 1940s New York

Our team at the CUNY Graduate Center decided that an easy but effective way to republish the material would be with a simple interactive map: click on a neighborhood to display its 1943 profile. The project became more involved than that — and our effort is still very much a work in progress — but that basic feature is what’s available at our Welcome to 1940s New York website.

We use DocumentCloud to provide easy access to the entire 1943 document, as well as neighborhood-specific profiles such as the example below:

Highlights from the neighborhood profiles

At CUR’s website we provide a detailed overview of how the Census statistics from 1940 compares with the city of today. I’ve highlighted some items below:

Population comparisons

Each neighborhood’s population size is compared with another U.S. city (e.g., with a population of almost 180,000 in 1940, Williamsburg, Brooklyn was “larger than Fort Worth, Tex.”) The comparisons reflect a time when the city’s population — overall, and even for specific neighborhoods — dwarfed most other urban areas across the country.

Color-coded Maps: rent too damn high even in 1940?!

The maps portray the geographic patterns of monthly rent levels across the city, ranging from under $30 to $150 per month or more. After adjusting for inflation, the high-end rent would be just under $2,500 in today’s dollars – in some contemporary neighborhoods, still a relatively modest rent.

With the maps, you can see for yourself how closely or not the patterns match life in our city today. As you do, take a moment to appreciate the cartographic craftsmanship involved in color coding each block based on Census data. No desktop computers or Google Maps back then!

Hundreds of Photos

Each profile includes black & white photos from the early 1940s, usually of typical residential or commercial blocks in the neighborhood. The photos are angled in the original, so don’t worry that the scanning process tilted the images.

Narratives

Each profile includes a brief description of the neighborhood. The emphasis is on local socio-economics, but the depictions offer a window into local demographic changes afoot at the time. Here’s the narrative for Maspeth, Queens as an example:

Maspeth is not a thickly settled district, but it enjoyed a 10 percent population growth in the 1930-1940 decade. The southwestern portion is an industrial area. Much of the southeastern portion is devoted to cemeteries. The residential area consists almost entirely of one and two-family dwellings. Most of the houses adjoining the industrial area are old and in the low rental group. There are some newer homes in the northern section of the district. The balance of the homes are of the less pretentious type. Grand Avenue is the main shopping street.

Borough maps and statistics

The 1943 document also provides six fold-out, color maps – one for each borough and one citywide – along with economic statistics at a boro-wide level such as:

  • manufacturers (number of establishments, wages, and value of products);
  • wholesale and retail trade;
  • number of families owning a radio set;
  • aggregate value of savings deposits; and
  • number of residential telephones.

A collaborative effort

The Welcome to 1940s New York website is the result of David Burgoon’s professionalism, creativity, and efficient, effective development. Kristen Grady georeferenced maps from the 1943 document in order to create a GIS layer of neighborhood areas which you see on the website, as well as the citywide map of rent levels. The website’s logo was designed by Jeannine Kerr.

The website relies on jQuery, the basemap is from MapBox, map navigation is provided through Leaflet.js, and the neighborhood map layer is hosted by cartoDB.

We are indebted to DocumentCloud for hosting the individual scanned pages from the 1943 document, and for providing online access to the material, including high-resolution versions of the Market Analysis profiles.

Several people reviewed early versions of Welcome to 1940s New York and provided helpful critiques and recommendations for improvement. Hopefully we did justice to their suggestions. They include: Jordan Anderson, Neil Freeman, Kristen Grady, Amanda Hickman, Michael Keller, Nathaniel V. Kelso, Jeannine Kerr, and Dan Nguyen.

The individual pages from the 1943 Market Analysis were scanned by the FedEx Office staff at the 34th St & Madison Ave location. Big thanks to them!

What’s next

We have reached out to potential partners to expand and enhance this project, hoping to leverage the 1940 Census data and other vintage statistics, maps, and photos to paint a richer picture of life in New York during the first half of the 20th century. This includes:

  • working with the NYC Department of City Planning’s Population Division — home to even more decades-old maps and data at the local level (down to city blocks) and citywide; and
  • discussing a potential exhibit (or exhibits) with local institutions such as the Museum of the City of New York, the New-York Historical Society, and/or the NY Public Library.

I’ve been lucky enough to pore over the original myself, and seeing it (and experiencing it in a tactile way) is inspiring. I worry that making it accessible interactively the way we’ve done it – neighborhood by neighborhood – disembodies it perhaps too much. (Online access makes it widely available, but maybe takes something away from the experience, sigh.) But nonetheless I hope everyone can check out the website, get a sense of what New York was like more than 70 years ago, and put the material to good use.

Enjoy!

Proposed Congressional districts for NYS available in GIS format

UPDATE June 25, 2012

We launched a companion map featuring Congressional districts with statistics on eligible voters by race/ethnicity compared with total population.

UPDATE March 6, 2012

We’ve added Congressional districts as proposed by District Court Judge Hon. Roanne Mann to our interactive redistricting site. Here’s a link that compares District 9 (Rep. Turner, in NYC) with one of the proposed districts that it would become under her proposed lines: http://t.co/01K4hMu8

We also submitted a letter today to the court [PDF] suggesting that they can use our maps to visually compare the different proposed lines.  Hopefully they’ll put our online maps to good use as they review the different Congressional district proposals.


UPDATE March 5, 2012

We’ve made two updates the information below.

  1. We’ve added the Congressional district data in shapefile and KMZ formats based on Common Cause’s submission to the court.  We think this will be especially helpful since the court has asked the intervenors to compare their maps with Common Cause’s proposal.
  2. Now you can visualize the proposed districts based on the mapped data below at the Center for Urban Research’s interactive redistricting site.
    1. compare with existing Congressional districts;
    2. easily switch among the Congressional proposals from Common Cause and the Senate & Assembly majorities; and
    3. view the proposed districts in relation to block-level demographic maps (do any of them appear to “pack,” “crack,” or dilute the potential voting power of minority populations?) or local voting patterns (click the “More Data” tab at the bottom right).

Here are some examples:

Today the New York World posted an analysis of how these different Congressional district proposals might impact Rep. Charles Rangel’s current district 15.


Original Post

If you’re hoping to use GIS or any of the online mapping tools to map the Congressional district lines in New York State that were proposed late yesterday, you’ll have some work to do.  The maps were released in PDF format as well as “block assignment lists” for the proposed districts.

But if you’d like to use shapefiles and/or KML files, you’ve come to right place!  Our team at the CUNY Graduate Center has created them and posted them for downloading here:

http://www.urbanresearch.org/news/proposed-congress-districts-in-gis-format/

We hope to add these soon to our interactive redistricting map. Stay tuned!

Redistricting’s partisan impacts: a GIS analysis

Our team at the Center for Urban Research is collaborating with The New York World to analyze the impacts of redistricting in New York State.  The latest effort was featured today on the front page of the Times Union; it focuses on how the majority parties in the State Senate and Assembly would likely retain — and strengthen — their control of both houses through the redrawn district lines.

Briefly, we found that the new boundaries for state Senate and Assembly districts proposed by LATFOR would increase the number of seats held by the majority parties in both chambers.  We based the analysis on 2010 election data available from LATFOR’s website.  The goal was to determine the results of state legislative elections held within the new districts if voters cast their ballots in the exact same way as they did in 2010, the most recent election year for State Senate and Assembly.

  • In the State Senate, the Republican Party’s 32-to-30 majority would expand to 34-to-29 if each voter cast his or her ballot in support of the same party as in the 2010 elections.
  • In the State Assembly, the 98-to-50 advantage the Democrats enjoyed following 2010’s elections would also increase, to 102-to-48.

The project was a good example of the power of GIS.  The analysis didn’t necessarily need a map to display the results (though Michael Keller at the NY World put together a nice one). But the analysis effectively wouldn’t have been possible without GIS.

Converting Polygons to Points

We analyzed election results at the level of voter tabulation districts, or VTDs, which are several blocks in size and typically cast no more than a few hundred votes in state legislative elections.  We mapped the VTDs onto the new lines proposed by LATFOR, then added up the votes of all VTDs that fell within a proposed district to determine its outcome.

In order to allocate the VTD-level vote counts to LATFOR’s proposed districts, CUR matched VTDs spatially with the current and proposed legislative district using ESRI’s ArcGIS Desktop software. The current and proposed Senate and Assembly districts are coterminous with Census blocks (in fact, the districts are “built” using Census blocks).  Unfortunately, neither LATFOR nor the state’s Board of Elections provides election results at the block level.  The Board of Elections records data by election district, which sometimes are smaller than VTDs, but for this project we did not have access to the election district data.

The challenge was that where the VTDs were larger than Census blocks in some places, the VTD boundaries crisscrossed the district lines (see example at right from Queens; click to enlarge).  In order to assign Senate and Assembly district IDs to each VTD, CUR converted the VTD boundaries to centroids (the geographic center-point of each VTD).  We used the lat/lon centroid values provided by the Census Bureau’s TIGER data.  Then we used a spatial join using ArcGIS to add legislative district identifiers to each VTD based on the legislative district its centroid was inside.  See the image below for the locations of the VTD centroids in this area of Queens.

In the instances where VTDs crisscross legislative districts, this technique will allocate all of a VTD’s votes to a single legislative district rather than splitting them across multiple districts.  This will over- and underestimate vote totals in some districts. But the process avoids the cumbersome effort involved in the alternative: splitting VTD vote counts.  The splitting process uses one of two methods:

  • using block-level population to “spread” the VTD votes across the VTD (multiplying the VTD vote count by the percentage of the VTD population occupied by each block and assigning the result to each block), or
  • weighting the VTD vote count based on the area of the portion of the VTD in each district.

Either of these approaches will result in fractions of people being assigned to one legislative district or another.  In fact, LATFOR appears to have used some sort of weighting method to assign election district vote counts to VTDs, since some of LATFOR’s VTD vote totals included fractions.

The centroid-approach and the weighted population / area approach both make assumptions about how to allocate vote counts.  But we tested the centroid process with current legislative districts and found that our VTD-allocated vote totals either exactly matched the results from the Board of Elections or were within a few hundred votes (which did not change the 2010 outcome).

Whether we used the centroid-approach and the weighted population / area technique, it otherwise would’ve been difficult if not impossible to determine how to allocate the VTD-level vote counts to legislative districts without GIS.  There are almost 15,000 VTDs across New York State, and there are (currently) 62 Senate districts and 150 Assembly districts.  With GIS, the process was relatively straightforward and efficient.

Aggregating by District

At the VTD-level, LATFOR provides the total number of votes cast by party in each election, not by candidate.  One challenge that we confronted was assigning the votes cast for fusion candidates who were backed by a major party but also received support on smaller parties’ ballot lines.  For example, many Democratic candidates received significant numbers of votes on the Working Families Party ballot line, and many Republicans got substantial support on the Conservative Party line.  Cross-party endorsements were even more variable for the Independence Party: in some districts, the Democrat received support on the Independence Party line; in others, its endorsement went to the Republican.

We decided that the most accurate way to re-map the election results was to assign the votes for each VTD based on the actual vote patterns for the Senate or Assembly district that contained that VTD in 2010.  In other words, if the Democratic candidate in an Assembly district ran on the Democratic, Conservative, and Independence lines, we assigned the Democratic, Conservative, and Independence votes in all the VTDs in that district to the Democratic candidate.  When we allocated the VTDs to the proposed Senate and Assembly districts, we added up the votes based on these patterns.  This ensured that the local voting patterns from 2010 were allocated accurately to the proposed districts.

The Results: Maps vs. Plain Old Numbers

The result is that we were able to calculate the number of proposed districts that, all other things being equal, would have had a Democratic winner in the Assembly and a Republican winner in the State Senate.  The important finding is that both parties would have increased their majority — which is especially interesting in the Senate, where the Republicans currently only have a 1-seat majority.  In Albany, the majority in each house is extremely powerful, so holding on to (or improving) those margins is all-important.

Of course, as the New York World/Times Union article points out,

To be sure, no district votes the exactly the same way in consecutive elections: the quality of candidates, changes in the population and the national political climate (which in 2010 favored Republicans) all play important roles. But voting behavior in previous elections offers the best available indication as to how a district is likely to perform.

The map that the New York World published along with the article uses red/blue color-shading to visualize the impact of the voting patterns on the proposed districts.  In the state Senate, the analysis shows the majority party increasing the number of seats by two.  On the map, that result is almost lost in the sea of red districts (most of the Republican seats are in upstate New York and Long Island, where the districts cover much larger areas than the more densely populated and largely Democratic districts in New York City).  The real power of our finding is the change in number: from 32 to 34.  In some ways, that says it all.

Nonetheless, the map (along with CUR’s interactive map comparing current and proposed district boundaries) provides a strong graphic and interactive element to the story, and provides context as you move your mouse over the districts to see the vote totals change from one to the next.

Watch for more analysis as LATFOR publishes its proposed Congressional district lines, and when the final Senate and Assembly districts are drawn.

Interactive NY redistricting map with cartoDB and more

UPDATE Nov. 5, 2012

In preparation for the Nov. 2012 election, many news organizations and others are linking to our interactive State Legislature and Congressional redistricting maps. We’ve posted examples at the Center for Urban Research website.


UPDATE Sept. 7, 2012

We’ve updated our map of redistricted State Senate and Assembly districts, highlighting the differences in race/ethnicity characteristics between total population and voter-eligible population – in other words, comparing the characteristics of all those who live in the new districts versus the smaller group who will be eligible to vote for each district’s representatives.  In some cases the differences are striking.

Our examination of the district-by-district data is available here.


New York State, like all other states, is in the midst of redrawing its legislative district lines. To help you follow along, our team at the Center for Urban Research has launched an interactive redistricting map for New York.  We collaborated with The New York World to develop the maps (though we encourage anyone and everyone to use them!).

The World’s reporters and editors are using our maps to go between the lines and explain how redistricting really works in the Empire State. (Here’s their first piece: The art of redistricting war.)  And we hope you’ll be able to use the maps too, to help answer questions such as:

  • Will you still be represented by the same State Senate or Assembly district you live in now?
  • Will you live in the newly proposed (and controversial) 63rd Senate district?
  • Is your neighborhood, town, or county going to be “carved up” by a new legislative seat?
  • Will your community’s historical voting power be diluted by the new districts?

We have some examples of gerrymandering at our Center’s website. In the meantime, here’s how you can use the maps.

Map features

The maps compare the current and proposed district lines (which our team mapped based on Census block lists published by the state’s redistricting task force, known as LATFOR). Here’s how they work:

  • Enter your address to find out what district currently represents you, and which proposed district you’d live in.
  • The current districts are on the left, and the proposed districts on the right.
  • You can also click on either map to highlight the current and proposed districts. As you move one map, the other moves in sync.
  • When you enter an address or click on the map, an info window pops up listing the current and proposed districts. You can click the link for the current district to go to that Senator or Assemblymember’s website.
  • Switch between State Senate and Assembly districts. Congressional districts will be posted once the data is available from LATFOR.
  • You can zoom in to street level, or zoom out to a statewide view. Switch between a street basemap or an aerial view from Microsoft’s Bing maps to see geographic details.

If you’re using the “Overlay” view, you can move the transparency slider to the right to display proposed districts, and to the left to fade back to current districts. The video below shows how:

If you want to share the map you’ve made, click the “Link” in the upper right of the map page to get a direct link to the area of the map you’re viewing. It will look like this:

http://www.urbanresearchmaps.org/nyredistricting/map.html?
lat=40.72852&lon=-73.99655&zoom=13&maptype=SIDEBYSIDE
&districttype=SENATE
  • You can share this on Twitter, Facebook, etc and email it to friends and colleagues.
  • You can also embed the map at your site. Use this link …
http://www.urbanresearchmaps.org/nyredistricting/map.html?output=embed
  • … or add < &output=embed > to any of the direct links you create, like this:
http://www.urbanresearchmaps.org/nyredistricting/map.html?
lat=40.72852&lon=-73.99655&zoom=13&maptype=SIDEBYSIDE
&districttype=SENATE&output=embed
  • … or wrap the snippet below in an iframe tag (I’d wrap it myself, but wordpress.com strips out iframe tags):
src="http://www.urbanresearchmaps.org/nyredistricting/map.html?output=embed" 
frameborder="0" marginwidth="0" marginheight="0" 
scrolling="no" width="600" height="700"

Side-by-side maps with OpenLayers

We borrowed from our “Census Comparinator” mapping site that Dave Burgoon artfully developed, in order to provide three ways to compare the current and proposed legislative districts:

  • a side-by-side view — two maps that are synced and move as one;
  • an overlay — a single map where you can fade between current and proposed districts; and
  • the vertical “before-and-after” slider approach.

I blogged about the Comparinator approach here and here. John Reiser also gave the technique a shoutout at his “Learning Web Mapping” blog for Rowan University.

With Census data, our “Comparinator” approach helped visualize changing spatial patterns of race/ethnicity trends – in cartographic terms, between two choropleth maps. With legislative districts, the comparison is between two sets of boundary files with no inner fill. So here we’ve set the side-by-side view as the default — we think the side by side maps give the easiest way of visualizing how the districts may change. But we also give you the option of viewing the districts with our vertical slider bar if you’d like, or the overlay.

Behind the scenes

For the proposed districts, we used ArcGIS to create the legislative district shapefiles based on LATFOR’s Census block assignment lists.  The current district boundaries are from the Census Bureau’s TIGER files (here’s the FTP page if you’d like to download the “lower” house districts — in New York, that’s the Assembly — or the “upper” house shapefiles — the State Senate).

We use OpenLayers for the map display and navigation with this application, as we’ve done with most of our other interactive maps. OpenLayers is easy to use, enables us to access Bing map tiles directly (so the basemap performance is smooth), and provides a robust JavaScript library for online maps.

That said, newer approaches such as Leaflet.js enable more interaction such as mouseovers, so we’ve started experimenting with some impressive new tools. More to follow!

One of those new tools is the powerful backend geospatial database engine from the team at Vizzuality: cartoDB. Hosting the legislative district shapefiles on cartoDB provided lots of advantages over hosting the data ourselves or setting up an Amazon cloud instance on our own. cartoDB provides:

  • great performance — not only for the district boundaries, but soon we’ll be adding election district maps to show voting patterns within each Senate and Assembly district. We don’t want to bother with creating pre-rendered tiles for this data. cartoDB will render it speedily on the fly.
  • cartographic flexibility: cartoDB uses cartoCSS for map symbology and labeling. Though there are still some quirks with cartoCSS, it was easy to grasp and it’s basically just CSS, so it makes styling easy if you’re familiar with modern web design. And cartoCSS incorporates scale-dependent rendering as well as attribute-based symbology, which makes it powerful and flexible. CartoCSS can be implemented using the cartoDB management interface, or programmatically.
  • easy data management: if you know SQL — and even better, if you’re familiar with SQL commands with PostGIS — you can quickly and easily modify tables, filter data, and perform spatial operations. (The screenshots at the cartoDB github page offer some examples.) Very cool.
  • scaling: cartoDB uses PostGIS and makes use of Amazon’s platform. So if our maps go viral, we’re ready for the usage spike!
  • open source: if you want to manage your own instance of cartoDB, just download the code and go! Big props to Vizzuality for an amazing geospatial toolkit.

Other thanks go to:

  • LATFOR, the state’s redistricting task force. Whatever you think about their redistricting process, they’ve done a great job with open data. They’ve not only posted the list of Census blocks that make up each proposed legislative district. But they also posted a wealth of data at the Census block level and also at the election district level (with a crosswalk between EDs and Census “voter tabulation districts”). This data is indispensable for visualizing, analyzing, and (hopefully) making sense of the new districts.
  • Dave Burgoon and the CUR team. Dave put together the redistricting mapping site in record time. Although it’s based on work he had already done with the Census Comparinator maps, it still involved substantial modifications and enhancements. But he made it happen as professionally and elegantly as always.
  • The New York World. We had been planning to create an interactive mapping application to build on our Census Comparinator site and to help people visualize the impacts of the redistricting process and demographic changes more broadly.  But the World team – Alyssa Katz, Michael Keller, and Sasha Chavkin – met with us a few weeks ago to discuss how we could collaborate on analyzing and mapping the upcoming district proposals from LATFOR.  The discussion inspired us to roll out a mapping site specific to New York State and focused on comparing the current and proposed districts. We’re thrilled to be able to work closely with them on this project (watch for more maps and data in the near future!).
  • The Hagedorn Foundation. The Foundation has provided funding support for our efforts to map and analyze Census data for a variety of civic engagement purposes, especially for Hagedorn’s Long Island-based grantees but also nationwide. Their support has been essential for us to develop innovative mapping applications like the NYS redistricting maps – not to advocate specific district plans one way or another, but to give local residents and others the tools they need to understand the impact of redistricting and hopefully get involved in the process.

Proposed NYS Senate & Assembly districts available in GIS format

UPDATE Nov. 5, 2012

In preparation for the Nov. 2012 election, many news organizations and others are linking to our interactive State Legislature and Congressional redistricting maps. We’ve posted examples at the Center for Urban Research website.


UPDATE Sept. 7, 2012

We’ve updated our map of redistricted State Senate and Assembly districts, highlighting the differences in race/ethnicity characteristics between total population and voter-eligible population – in other words, comparing the characteristics of all those who live in the new districts versus the smaller group who will be eligible to vote for each district’s representatives.  In some cases the differences are striking.

Our examination of the district-by-district data is available here.  The New York Times gave our analysis a shout-out in their CityRoom primary election day column.

You can also visit our original NYS redistricting “comparinator” map described below, at www.urbanresearchmaps.org/nyredistricting/map.html


UPDATE February 5, 2012

You can visualize these proposed districts in relation to the current New York State Senate and Assembly districts with our new interactive redistricting map.  We developed the interactive map in collaboration with The New York World, and here’s an article using the maps to describe the redistricting process in the Empire State.  For more background on the interactive map, visit this blog post.


Original Post

If you’re hoping to use GIS or any of the online mapping tools to map the legislative district lines in New York State that were proposed today by the state’s redistricting task force, you’ll have some work to do.  The Task Force released PDF maps as well as “block assignment lists” for the proposed districts.

Unless you’d like to use the shapefiles and/or KML files that our team at the CUNY Graduate Center created!  Here’s our web page with the info: http://www.urbanresearch.org/news/proposed-nys-districts-in-gis-format

Happy redistricting mapping!

Innovative map comparisons – Census change in 15 cities

Our team at the Center for Urban Research (at the CUNY Graduate Center) has updated our interactive maps showing race/ethnicity patterns from 2000 and 2010 in major cities across the US. We’ve enhanced the maps in several ways:

  1. Added more cities. We now have 15 major urban regions mapped across the US (Atlanta, Baltimore, Boston, Charlotte, Chicago, Detroit, Houston, Los Angeles, Miami, New York, Orlando, Philadelphia, Phoenix, San Francisco, and Washington D.C.).
  2. The maps now have three ways of comparing 2000 and 2010 racial patterns:
  3. We color-coded the population change data in the popup window. Population increase is shown in green; decrease is shown in red. See image below.

Here’s our news release with more info.

Btw, we’ve also updated our static maps to show New York City Council districts, to begin to get a sense of how demographic changes will shape upcoming redistricting efforts at the local level.  Here’s the link:www.urbanresearchmaps.org/plurality/nyccouncil.htm (For the static maps, you can view 2000-2010 demographic change with the vertical slider bar, but you can’t zoom in/out, etc.)

An initial version of the maps launched in June with the vertical bar technique, integrating it with interactive, online maps for the first time. Our Center crafted the maps so you could not only drag the bar left and right but also zoom in and out, click on the map to obtain detailed block-level population counts, and change the underlying basemap from a street view to an aerial image (via OpenLayers use of Microsoft’s Bing maps tiles), while also changing the transparency of the thematic Census patterns.

The latest iteration of CUNY’s Census maps continues to use the vertical slider but now incorporates this technique with two more comparison options. Each approach serves different purposes:

  1. The vertical slider bar provides a “before (2000) and after (2010)” visualization of change, either regionally or at the scale of a city neighborhood.
  2. The side-by-side comparison is ideal for lingering over a given area, especially at the local level, taking the time to absorb the differences in demographic patterns mapped with 2000 Census data on the left and 2010 on the right. We incorporated this approach specifically at the suggestion of the great interactive team at the Chicago Tribune, who have created some similar Census maps.
  3. The single-map 2010/2000 overlay is especially helpful for revealing the increase in diversity over a given area.

For example, you can zoom to Atlanta, GA on the single-map overlay and see the city’s predominantly Black population in 2000 surrounded by suburban Census blocks shaded dark blue, denoting a White population of 90% or more (see images below). As you transition the map from 2000 to 2010, the dark blue in the suburbs fades to a lighter shade (indicating a more mixed population demographically) coupled with more Census blocks shaded green, purple, and orange – each corresponding to communities that are now predominantly (even if only by a few percentage points) Hispanic, Asian, or Black respectively. This pattern is replicated in many of the urban regions featured at the website.

Atlanta & suburbs in 2000

Race/ethnicity change in Atlanta by 2010

Eventually we’ll be moving all this from pre-rendered tiles to vector tiles. CUR’s application architect Dave Burgoon contributed code he developed to TileStache to enable TileStache to produce AMF-based output for use in Flash-based interactive mapping applications. This will give us flexibility in mapping as many Census variables as needed, and also providing complete geographic coverage (hopefully down to the block level) nationwide. That’s the plan, anyway! Stay tuned.

Credits

Funding for much of the Center’s recent work on Census issues has been provided by the Building Resilient Regions Project of the John D. and Catherine T. MacArthur Foundation, the Hagedorn Foundation, as well as support from the CUNY Graduate Center and the City University of New York.

Several people provided feedback and helpful editorial suggestions on earlier versions of the maps and narrative. Though the materials at this site were prepared by the Center for Urban Research, those invdividuals improved our work. We greatly appreciate their contributions.

Slippy maps, meet before-and-after jQuery slider (introductions by OpenLayers)

Our team at the Center for Urban Research (at the CUNY Graduate Center) has launched a set of maps showing race/ethnicity patterns from 2000 and 2010 in major cities across the US.  The maps combine several mapping/web technologies that offer a new way of visualizing population change.  This post explains how we did it.

(And by popular demand, we’ve also included a map of Congressman Anthony Weiner’s district in relation to demographic change — you may have heard of him and his Twitter travails recently?)

Race/Ethnicity Change

Briefly, the maps show race/ethnicity change from 2000 to 2010 at the local level throughout major urban regions across the U.S.  So far we include New York City, Los Angeles, Boston, Chicago, Houston, and San Francisco.  (Others are coming soon.)

For our methodology and data analysis (and static maps), we provide that here.  For the mapping and web techniques, see below.

Reactions

So far we’ve received a pretty good response to our maps.  Here are some tweets posted recently:

  • @dancow (web journalist for ProPublica): Cool before/after map from CUNY’s urban research center showing NYC ethnic changes at the block level, from 2000-10.
  • @mericson (deputy graphics editor at NY Times): Nice block-level maps by @SR_spatial & CUNY Urban Research Center showing racial/ethnic change in NYC from 2000 to 2010.
  • @kelsosCorner (former Washington Post cartographer): Digging new 2010 Census plurality maps of NYC.
  • @albertsun (graphics editor at Wall St Journal): Coolest census map I’ve seen yet.
  • @PJoice (HUD employee; tweets are his own): This is the coolest map I have ever seen. Nice work by @SR_spatial and CUNY!
  • @MapLarge: I like how you can use the slider or move the map! Great Visualization!

Technical overview

The map uses the “before and after” technique that media websites have used for images of natural disasters.  We enhanced this technique by integrating it with interactive maps using OpenLayers, the open source mapping framework.  Now the slider works with two sets of overlapping, but perfectly aligned, maps from 2000 and 2010.

As it turns out, we didn’t set out to create an interactive version of these maps. In fact, we originally created static maps, but everyone we showed them to for feedback wanted the ability to zoom in/out and click on the map for more info.  So we developed the OpenLayers version. (And when I say “we”, that mainly means David Burgoon, CUR’s application architect, who I can’t say enough good things about.  I made the maps, and CUR’s Joe Pereira of the CUNY Data Service created the data sets, but Dave brought it all to life.)

OpenLayers enables us to introduce interactivity into the before-and-after images. Maps like these (to our knowledge) have not been available before — where you can move a slider back and forth while also zooming in/out and clicking on individual Census blocks for detailed information. You can also change the transparency of the thematic map layer, and switch between a street view and aerial view basemap.

It involved a good amount of work to integrate the slider technique with OpenLayers and also have two overlapping map instances working in tandem. The two maps need to appear as one, and this involves painstaking effort to ensure that the pixels on your screen are translated accurately to latitude/longitude coordinates in each of the separate but related interactive map instances, and the maps pan together seamlessly as you drag the slider left or right or move the map and it crosses the slider.

Mashup

In order to create the application, we used a mix of software applications, technologies, and techniques, summarized below:

  • We used the statistical software package SPSS to extract the Census block-level data for both years (see our methodology), allocate the 2000 data to 2010 blocks using the Census Bureau’s block equivalency files, and calculate the race/ethnicity plurality for each block.
  • We exported these SPSS files in DBF format and used ESRI’s ArcGIS Desktop to join the DBFs with 2010 TIGER Census block shapefiles.
  • ArcGIS Desktop was also used to create the choropleth maps (based on color schemes from ColorBrewer.org);
  • The map layouts were published as temporary web map services using ESRI’s ArcGIS Server. We used these to create pre-cached tiles (.PNG files) for the 2000 and 2010 maps, corresponding to zoom levels 4 through 10 using the now-standard Google-Microsoft map scales for online web maps. (Our application accesses the choropleth tiles as PNGs directly from the cache created by ArcGIS Server, rather than accessing the ArcGIS web map service in order to assemble the tiles. The latter approach would be too slow and would undermine the transition as you dragged the slider across the map.)
  • The slider technique was adapted from the jQuery plugin by www.catchmyfame.com.
  • OpenLayers provides all the map navigation and serving the maps themselves, modified with customized JavaScript code.
  • The basemap shown beneath the color-shaded map tiles is provided by Microsoft’s Bing map service. The street map and aerial image tiles from Bing are accessed directly via OpenLayers, rather than using the Bing API. This is a key reason we used Bing for these maps; if we used Google Maps as a basemap, we were limited to accessing Google Maps via Google’s API, which would have slowed map drawing times and undermined the slider effect.
  • For geocoding we use the Yahoo! Placefinder API.
  • Some browsers are not able to handle the before/after slider effect smoothly. In particular, Firefox and Safari perform poorly; the slider transition between one map to the other is not smooth. Microsoft’s Internet Explorer is adequate, but Google’s Chrome browser is best.

Data sources/issues

We used block-level data from the Census Bureau’s 100% population counts from the 2000 and 2010 decennial censuses (from Table P2 in the “PL-94-171” files for 2000 and 2010).

The Census Bureau’s block geography changed between 2000 and 2010 — new blocks were created, blocks were merged, and block boundaries were modified in many places. In order to compare population data from 2000 and 2010 using a common set of blocks, we used the Census Bureau’s block relationship file to allocate the 2000 population counts to 2010 geography.

When you’re viewing the map, it is best to use the maps and block-level data to understand trends over a larger area, even over several blocks. Be careful when viewing a specific block on its own. It covers a small area, and the Census Bureau may have made errors.

Credits

Funding for much of the Center’s recent work on Census issues has been provided by the Building Resilient Regions Project of the John D. and Catherine T. MacArthur Foundation, the Hagedorn Foundation, as well as support from the CUNY Graduate Center and the City University of New York.

Several people provided feedback and helpful editorial suggestions on earlier versions of the maps and narrative. Though the materials at this site were prepared by the Center for Urban Research, those invdividuals improved our work. We greatly appreciate their contributions.

GIS and Census participation

It’s been too long since my last blog post. Have been quite busy with work, and even though Twitter is a microblogging service, sending a tweet now and then really isn’t an excuse to keep up my actual blog.

One of the projects keeping me (very) busy is our work to help boost participation in the 2010 Census. I thought I’d write about some of our interactive mapping and participation rate analysis along these lines.

In August I described how our team at CUNY’s Center for Urban Research was creating metro-scale maps showing where hard-to-count communities were located so local census advocates could target their outreach. Then in late January we launched our interactive version of those maps at www.CensusHardToCountMaps.org. Originally we designed the site to show hard-to-count areas, but these are only where it was expected there’d be low census participation. Then, on March 23, the Census Bureau started publishing the actual participation rates on a local and national basis. So a week later we updated our site to emphasize the latest participation rates (this link zooms in to Manhattan showing tract-level participation overlaid on a map of hard-to-count tracts).

Though the Census Bureau’s Take 10 map (and a related Google Earth application) display the daily participation rates nationwide, we decided to provide several features that the Census Bureau doesn’t. At our site you can:

  • type in a county and highlight the tracts below a certain participation rate (you can enter whatever rate you want);
  • sort the resulting list so you can see at a glance the highest and lowest performing tracts (this also will be highlighted on the map so you can see how concentrated they are); and
  • compare the 2010 rate map with the 2000 rate map (click the “More…” tab and check the box for “Participation Rate in 2000″).

(Of course, you can also click on any spot on the map to display the latest participation rate for that area — state, county, or tract — depending on how close in or out you’ve zoomed.)

These are the types of data analysis and spatial visualization tools that were requested by census advocates, so they can use the maps to focus on areas that need their help the most.

In order to provide some context for the interactive map, our Center also posted an analysis of the first week’s participation rate. It was a combination of basic statistical analysis and mapping. We examined the correlation between participation rate and hard-to-count scores at the tract level nationwide, and not surprisingly found that rates tended to be lower in hard to count areas. This should help bolster the work of groups who’ve been working in these communities, confirming that they’re focused on areas that need support the most if we want to achieve a 100% count.

We also examined county-level statistics on race/ethnicity using the Census Bureau’s latest population estimates from 2008. (The American Community Survey would provide a richer set of characteristics to examine, but any data from areas with less than 20,000 population are suppressed in the ACS — and this accounts for about 1,300 of the nation’s 3,200 counties.)

The county-level data indicate that race/ethnicity is strongly correlated with census participation (at least in the first week), with participation rates tending to be higher in counties with a greater percentage of whites while the rates tended to be lower in counties with a greater percentage of blacks and Latinos. Because we didn’t have other socio-economic data to evaluate, we weren’t able to disentangle the effects of other characteristics such as low educational attainment, poverty, housing conditions, etc. that may have a stronger correlation while cutting across racial and ethnic categories. An opportunity for further research. As a next step we may also examine county-level unemployment rates and participation rates, as well as evaluating how well the first week’s analysis holds up as time goes on.

Friday (April 2) we added another feature, information about the areas that will receive a second census questionnaire.  (The Funders Census Initiative sent out a news advisory highlighting this service on April 5.)  Now when you click on the www.CensusHardToCountMaps.org map or type in your street address, you’ll see a popup window that (among other things) tells you if households in your tract will be receiving replacement Census forms. We think this will help minimize confusion over people receiving another census form (even if they’ve already mailed their’s in!). This is a “just in case” thing from the Bureau — mailing another form to households in historically low response areas, and mailing another form to households in moderately low response areas who haven’t yet sent their’s in. But the geographic scope of the “blanket” and “target” replacement mailing areas are pretty extensive in most cities (see maps at CUR’s website), so lots of people may be confused. Our mapping site provides a simple way of clearing the air.

We’ve also mapped those second mailing areas. When you visit www.CensusHardToCountMaps.org, select the “More…” tab and zoom in to your area of interest. For example, here’s Boston, MA. Click either or both check boxes in the “April 2010 Replacement Questionnaires” section to map the tracts receiving replacement census forms.

Our hard-to-count mapping site still has its original functionality — such as visualizing the demographic characteristics that will make it difficult to achieve a complete Census count; overlaying ZIP Codes, mail return rates from 2000, recent foreclosure risks by tract; and seeing who’s tweeting about the Census in your area.

But we’re also planning for next steps, thinking of the mapping application as a platform for future Census-related efforts (tracking how successful census advocates were, displaying the 2010 results, enabling the general public to get involved in a meaningful way in the redistricting process). Any ideas? We’d love to hear them.

Mapping “hard to count” areas for 2010 Census

UPDATED 8/21/09: Newsday (Long Island’s daily newspaper) reproduced an island-wide version of one of our maps in their article today (though the map only appeared in the print version of the paper).

People are gearing up across the US for the 2010 Census — not just the Census Bureau, of course, but organizations large and small who are planning myriad outreach efforts to boost participation, especially in typically “undercounted” communities.

It’s important in so many ways for everyone to be counted, but historically not everyone is (and not just because of statistical anomalies or poor street address data).  For various reasons, key constituencies are not fully counted — people of color, renters, recent immigrants, people predominantly speaking languages other than English, etc.  There’s a special effort underway – supported by major foundations, local governments, and spearheaded by advocacy and civil rights groups – to make sure the Census Bureau doesn’t miss these “hard to count” groups.

Our teamat the Center for Urban Research, CUNY Graduate Center is doing its part by mapping the “hard to count” (HTC) population in more than two dozen metropolitan areas nationwide, with the support of the Hagedorn Foundation.  We’ll also be developing an online, interactive mapping application for funders, outreach groups, local officials, the media, and others to easily zoom in on their metro region and create custom maps to help focus their efforts.

Where are the hard-to-count communities?  The Census Bureau analyzed 2000 Census data to:

measure census coverage and to identify reasons people are missed in the census (de la Puente, 1993). The variables include housing indicators (percent renters, multi-units, crowded housing, lack of telephones, vacancy) and person indicators (poverty, not high school graduate, unemployed, complex households, mobility, language isolation).

From this, the Bureau calculated an HTC score for each tract in the nation (on a scale of 0 to 125).  They estimate that any tract with a score of 60 or higher will be at the greatest risk of undercount in 2010.  Our metro maps for the 2010 outreach campaign highlight the concentrations of HTC tracts, as well as the 2000 Census response rate by tract.  Here are two examples, for Chicago and New York City:

Chicago_HTC_2010Census

NYC_HTC_2010Census

Foundations are using these maps to help guide their grantmaking, and local groups are using the maps to target their outreach.  As an example, this month several Long Island-based foundations issued an RFPfor grants to local groups doing Census outreach.  The Hagedorn Foundation asked us to create a special set of Long Island maps to supplement the RFP – you can view the PDFs here and here(see below for a sample – these mapped the HTC scores relative to all tracts in Nassau, so some tracts with scores less than 60 — but still harder to count than others in the county — are shaded as hard to count).

Nassau_HTC_2010Census

Next step for us is to transform these printed maps into a nationwide online mapping website.  The site will enable people to: 

  1. Zoom to their neighborhood, county, city, or state to see the mapped patterns of hard-to-count Census tracts;
  2. Click on individual Census tracts to display detailed information about each one;
  3. Add other layers of Census demographics, 2000 Census response rates, related data from the 2005-07 American Community Survey, and recent foreclosure risks circa 2007-08 so people can see the interrelationships among multiple variables; and
  4. Display state- and local-level resources such as funding opportunities, regional Census contacts, contact info for groups participating in Census outreach, etc.  

We’ve been reviewing other projects that are similar so we don’t duplicate efforts.  These include:

A key question for us is which basemap do we use?  We’ll be serving the tract-level geography and attributes (probably as cached tiles) from our servers using ArcGIS Server on the backend, OpenLayers and ExtJS on the front end — a combination that has served us well (see here and here).  Likely we’ll use Google Maps (and/or their Hybrid or Terrain views) for the underlying street/reference geography.  But perhaps OpenStreetMap would be a better choice?  Or Bing Maps?  Advice from the GeoWeb community would be appreciated.

Timing? We’re shooting for a beta site in late September, seeking feedback from partner groups in the fall, and a production-level site by the end of the year.