• SR_spatial tweets

Interactive “Comparinator” maps launched for NYC Council districting

UPDATE Sept 7, 2012

The City Council Comparinator site is now embeddable for your website, blog, etc.

Here’s how to use it:

  • go to the site,
  • zoom to a district and click to highlight it (or enter a street address),
  • choose one of the tabs above the map (Side-by-Side vs Overlay, for example), and
  • pick which proposal you’re comparing with (Districting Commission or Unity Plan).
  • Then click “Link” in the upper right and you’ll see the embed code as well as the basic linking code.

Sample embed code:

We also added a feature: if you turn off the popup window before clicking “Link”, it’ll add a “popup=false” property to the URL, so the person viewing the link (or the starting image for your embedded map) won’t have the popup in the way but the district will still be highlighted.

Here’s an example:


ORIGINAL POST Sept 5, 2012:

NYC Council Districting and You

How existing City Council districts compare with proposed lines

councilcomparinatorscreenshot.PNG

Our Center for Urban Research (CUR) at the CUNY Graduate Center has launched an interactive map today to visualize proposed New York City Council districts compared with existing ones along with the demographic characteristics and patterns within the districts.

The Center hopes the map will help involve people in the NYC districting process simply by showing them how proposed or newly drawn lines looked in relation to their homes or workplaces. Our map is not for drawing districts; others such as the NYC Districting Commission are providing that service.   But CUR’s comparison maps are designed to be be engaging enough to visualize the impact of redistricting for everyone from local citizens to redistricting professionals, hopefully inspiring people to participate more actively in the process.

CUR’s map was designed and is being maintained independently from the NYC Districting Commission’s website. However, we hope that people who use CUR’s maps will then access the Districting Commission’s website for drawing maps online.

Using the map

The main features of the map are as follows:

  • Enter your address to find out what district currently represents you, and which proposed district you’d live in.
  • If you’re using the “Side-by-Side” view, the current districts are displayed on the left, and the proposed districts on the right.
  • If you’re using the “Overlay” view, you can move the transparency slider to the right to display proposed districts, or to the left to fade back to current districts.
  • Click anywhere on the map to highlight the current and proposed districts.
  • When you enter an address or click on the map, an info window pops up listing the current and proposed districts. You can click the link for the current district to go to that Councilmember’s website.
  • Click the “Link” in the upper right of the page to get a direct link to the area of the map you’re viewing. (This one zooms in on City Council District 8 in Manhattan.) You can share this on Twitter or Facebook, email it to friends and colleagues, or blog about it and include the link.

Credits

The mapping application was developed by the Center for Urban Research. David Burgoon, CUR’s application architect, constructed and designed the site, with data analysis support and overall conception from CUR’s Mapping Service director Steven Romalewski.

The application relies on geographic data hosting by cartoDB, open source mapping frameworks and services including OpenLayers and Bing maps, and ESRI’s ArcGIS software for cartography and data analysis.

Data sources

Current City Council district boundaries and proposed maps from the NYC Districting Commission are based on block assignment lists provided at the Districting Commission’s website.

Other proposed maps such as the Unity Map are provided by the advocacy organizations who developed those proposals.

Mapping Hurricane Irene in NYC (plus some thoughts on the city’s digital response to the storm)

A disaster, natural or otherwise, always creates an opportunity to demonstrate the power of maps. Hurricane Irene did not disappoint. In New York City, which hadn’t seen a hurricane of this magnitude in decades, there were at least a half dozen websites with interactive maps related to the storm (plus at least one PDF map – more on that below) that were used extensively and were tweeted about extensively. My team at the CUNY Graduate Center was in the mix with our OASISnyc.net site, and I was watching with keen interest as more maps kept coming online as Irene kept coming closer. I thought I’d share some observations below about how Irene was mapped in New York.

I think I kept good track of the various maps that were deployed, but I’m sure my list and descriptions are incomplete so please chime in if I’ve missed anyone or mischaracterized any of the efforts.

The Context

Hurricane maps are nothing new, but usually the maps show the path of a hurricane while it’s happening or analyze its impact after the storm has past. This time, for New York City, the more interesting and useful maps were focused primarily on the possibility of evacuation, and the potential impact of the storm on New York’s shores.

(That said, the damage from Irene continues north of NYC, and several important mapping efforts are helping with the recovery effort there. For example, follow tweets from @DonMeltz and @watershedpost in upstate New York, or @jarlathond in Vermont.)

The interest in these maps was also perhaps more intense than in earlier situations. First, New Yorkers almost never evacuate for anything (at least on a scale of hundreds of thousands of residents), so the idea that so many people from only certain areas of the city needed to move to higher ground meant that everyone wanted/needed to know: am I in the evacuation zone? And that meant maps.

Second, online interest in this storm in particular was high. Other storms have hit since Twitter and Facebook have been around, but not in the New York area and not at this scale. One writer for GigaOm who had lived through hurricanes on the Gulf Coast wrote that she was “overwhelmed” by the “overall hoopla surrounding Irene online.” For her, it replaced TV as a key source of news (I agree, I barely checked TV news throughout the storm. Twitter and weather-related websites provided all the information I needed, and the news from these sources was more up-to-date.) And because so many New Yorkers were online and hungry for information about evacuations and storm impacts, online maps were critically important.

Will I Need to Evacuate?

Mayor Bloomberg and other officials started talking about the possibility of evacuation on Wednesday (8/24). That night, my wife reminded me that our flagship mapping site OASISnyc.net included a layer of “coastal storm impact zones”.

Actually, we’ve had that data online since 2007, when it was a Map of the Day on Gothamist. It shows areas at greatest risk of storm surges from a hurricane (and, as it turns out, those areas closely match the boundaries of the city’s evacuation zones – see screenshot below). I had also received a couple of emails that night from other groups wanting to map the evacuation zones and were worried that the city’s mapping resources weren’t up to the task.

So I wrote a blog post about how using OASIS could help people see if they were in harm’s way if the storm hit the city. I published the post the next morning (Thursday, 8/25).

That same morning, Gothamist posted an item about the potential for evacuation, and they embedded the city’s evacuation zone map. I was the first to add a comment on the Gothamist piece (via our @oasisnycmaps Twitter account), and I included a link to my blog post and to the maps.

PDF maps: blessing, curse, or both?

Let’s look at the city’s evacuation zone map [PDF – see image at right]. It’s a PDF file. It shows all the city’s streets in black ink, in an 8.5″ x 11″ layout, overlain on color shaded areas (muted green, yellow, and brown) corresponding to the 3 evacuation zones A, B, and C. And it has the evacuation center locations labeled on the map.

So it puts a lot of information into one map, which is challenging on its own. But trying to view that as a PDF online can be especially problematic. People who expected something better complained — it was described (perhaps too harshly) as “terrible” and “useless” in the Gothamist comments. People said it was hard to read, took too long to download, it didn’t work well on mobile phones, etc. And quickly after I posted my comment at Gothamist, several people were thankful that they could access OASIS as an alternative to the city’s map.

Distributing a PDF map in a situation like this has pros and cons. On the one hand, it has flexibility. The PDF format can be viewed in any web browser, or can be downloaded to your computer and viewed there, and can be printed out to share with someone who doesn’t have Internet access. And lots of people on Twitter were appreciative. On the other hand, it’s not something that can be easily updated, and it’s not what the growing population of digitally savvy New Yorkers would expect or desire. NYC has been touting itself as the most digital city on the planet, and all they could do was put out a PDF? People were underwhelmed.

To be fair, the city also had an online “Hurricane Evacuation Zone Finder.” You’d type in an address and it would display a zoomed in zone map of your location. But that provided little context, and it wasn’t as user-friendly as the public was expecting. For a long time this type of web service would’ve been considered state of the art. But these days, I think a lot of people were wondering if New York couldn’t do any better.

Luckily the city had posted a dataset in GIS format representing the city’s evacuation zone boundaries. It was available on Datamine, and anyone could download it for free and use it without restriction. So when people asked me if I had the evacuation zones in a format that could be mapped, I just pointed them to Datamine.

(In a mix of optimism and revisionist history, New York City’s Panglossian chief digital officer was quoted saying that As always, we support and encourage developers to develop civic applications using public data” (emphasis added), in reference to other groups that were using the evacuation zone map in their websites. I chuckled when I read this. If you’ve been around this business for more than just a year or two, you’d know that it hasn’t “always” been this way. It’s terrific that at least some of the city’s data is openly available now. But let’s keep it in perspective, and also remember that there are still important public datasets the city is not making easily available to developers or others.)

NYC.gov goes down

By Thursday afternoon, online interest in NYC’s impending evacuation announcement was so intense that not only did the city’s zone finder application go down, but even the city’s website — in particular, its homepage — was inaccessible.

Although the city will certainly congratulate itself for using social media to get the word out (and I agree they did a good job in this area), it’s not good that the city that strives to be the nation’s premier digital city could not even serve up its homepage at the exact moment when everyone was relying on that web page for information on what was happening next. And with a situation as complex as an approaching hurricane, 140 character tweets are just not enough. I can’t imagine it’s easy to withstand several million hits in a day, but I and a lot of others expected better.

(After complaints from the community, at least one civic activist posted links to hurricane resources on his own and shared that via Twitter.)

The city was left apologizing for no web access and pointing people to its PDF map (at this point hosted on Tumblr and elsewhere). Mayor Bloomberg posted the PDF on his website, but that’s the least he could do. Simply taking a PDF and putting it on another website? Doesn’t take much to pull that off.

More Maps Come Online

In the meantime, more maps appeared. WNYC was next. John Keefe, the public radio station’s Senior Executive News Producer, mashed up the city’s evacuation zone data with Google Maps, and put a simple, easy to use interface together. The map didn’t include evacuation centers at first, but it was clean, effective, and … in the absence of the city’s online resources … it worked.

In fact, several people noted the irony. When @nycgov tweeted that the city’s hurricane zone finder was down “due to high traffic”, a Google representative quickly tweeted back that “WNYC’s map is based on NYC OEM data and is running fine.”

John has developed a successful system creating news-oriented maps in short order, and his hurricane map was the latest example. And it was embeddable, so sites such as Gothamist that originally embedded the city’s PDF map, quickly replaced the PDF with WNYC’s interactive map. People were happy.

By Friday morning, the city was still having difficulty providing online access to its web page and its hurricane evacuation zone finder app, so more mapping sites stepped up. ESRI published an interactive map of the evacuation zones and evacuation centers using their relatively new ArcGIS.com online platform. The map looked great, and included the evacuation centers that WNYC’s map was missing.

But the ESRI map didn’t have the slimmed down, focused look and feel of WNYC’s site. It included ArcGIS.com options such as geographic feature editing that maybe weren’t needed for this situation. (That’s just a quibble. Though at one point I clicked “Edit” and it seemed like I was about to delete all of Zone A!)

One nice thing about the WNYC site is that is uses Google’s Fusion Tables service on the backend, which makes it easy to set up geographic data and then overlay that data on a Google map or any other modern, online mapping site. At the CUNY Graduate Center we’ve started to use Fusion Tables to integrate community-oriented mapped information into the OASISnyc site. By Friday morning we were able to use Fusion Tables to display the city’s evacuation centers on OASIS’s maps. The OASIS site provides a wealth of information such as subway and bus routes, schools, public housing sites, etc. so it provided a way (hopefully an easy way) to locate evacuation sites in relation to these other locations.

By Friday, Google had also stepped in with a mapping service of its own, a customized version of its crisis mapping application.

Originally Google’s map omitted the city’s evacuation zones or centers, but it did include several other layers of data related to potential storm impacts (like the storm surge map at OASIS). The federal weather and environmental agencies such as NOAA and FEMA have consistently done a great job of providing free, online access to observation and modeling data about storms, and Google put this information to use.

Regional Maps

On Friday our team at the CUNY Graduate Center also made two enhancements to our mapping applications to make it easy for a wide range of people to find out if they might be hardest hit by Irene. First, we reconfigured the OASIS maps so the storm surge layer could load quick. We created a pre-cached tiled layer instead of a dynamic layer and also set up the map page so that most of the dynamic layers were turned off by default. This made the map page load quicker, and made the storm surge layer load instantaneously (our site had bogged down a bit on Thursday due to increased traffic — site usage almost tripled to 9,000 pageviews almost solely from my comment at Gothamist with a link to OASISnyc.net — so quick loading was key).

We also incorporated the storm surge layer to an interactive mapping site we maintain with the Long Island Index focused on Nassau and Suffolk counties. It seemed that the storm might have a greater impact on Long Island. The storm surge data we used for OASIS was statewide in scope (it was created by NY SEMO), so we coordinated with our partners at the Index and updated the site Friday afternoon.

Newsday included a link to the LI Index mapping site, and usage soared over the weekend.

Understandably, an organization such as WNYC would limit its map to the city’s 5 boroughs. But there weren’t similar maps for any other part of the tri-state region.

Even though mandatory evacuations had been called for much of Long Island’s south shore, the best data available on those areas were lists (some in PDF format) of affected addresses and affected streets. Given the surge in usage of the LI Index mapping site, I like to think that we helped meet a key need.

Mandatory Evacuation and More Maps

During the day on Friday, Mayor Bloomberg announced the city’s mandatory evacuation plans. The scramble was on to see if you were in Zone A!

Not to be outdone by WNYC, Google, or anyone else, the New York Times launched its version of an interactive evacuation zone map late in the day Friday.

Like WNYC’s version, the NY Times map was focused and easy to use. But it was also limited to NYC, despite the Times’s readership outside the 5 boroughs who had also been required to evacuate.

By then, WNYC and Google had also added the locations of evacuation centers to its maps.

Lessons Learned?

So what to make of all these maps?

I think the first thing is that they were all generally helpful. When the nation’s premier digital city was incapable of providing digital information in a timely, useful way, others stepped in and succeeded.

These efforts, however, suffered to some extent from inconsistencies and lack of coordination.

For example, different mapping sites displayed different kinds of information in ways that may have been confusing to the person on the street.

Google and OASIS posted storm surge zones and the city (and WNYC, ESRI, and the Times – and eventually Google too) posted evacuation zones. Ultimately what most people wanted to know was if they lived in evacuation Zone A. The storm surge areas were important in terms of anticipating where the storm would do the most damage, but perhaps a more pressing issue was the evacuation.

But this difference in approaches underscores the lack of coordination among the various mapping entities. It was as if everyone just wanted to get *their* map online.

We’re as guilty of that as anyone. I know top staff at OEM and I easily could’ve contacted them to coordinate the OASIS layer with their’s. But it was somewhat frantic at the time, and the communication didn’t happen. I’d say WNYC was the most earnest in this regard, since they probably just saw a hole that needed to be filled – the city was talking about evacuation, but the city’s evacuation map was sorely lacking or not online.

But once WNYC went online, as far as I know there was little coordination among them, us, ESRI, the NY Times, Google, etc. I think you could reasonably ask — since WNYC’s map worked perfectly well, and provided the information about evacuation zones — why have essentially the same map from ESRI, Google, and the NY Times. Were these groups talking with each other? For the media outlets (WNYC and the Times), was it just a competition thing?

I do know that when the city’s GIS community was more cohesive, this probably would’ve been coordinated a bit more, perhaps through GISMO. Not that the lack of cohesion is a bad thing necessarily. And not to fault GISMO or other coordinating groups. But I wonder if better information could’ve been provided to the public in a better way if all of us making the maps were in communication.

For example, for at least a day WNYC’s map lacked the evacuation center locations. I added the locations to OASIS using Fusion Tables. Then WNYC added the locations to its map, also using Fusion Tables. We easily could’ve shared the backend data, but WNYC never contacted us to discuss it. I sent a tweet to @jkeefe about it, but didn’t hear back. It was important to keep the evacuation center data up-to-date and consistent because the city changed the locations of 4 centers before Irene hit. Keeping the maps in sync would’ve minimized any confusion for the public.

Overall, I think the biggest takeaway is that the Mayor’s office and NYC agencies – especially DoITT (since they’re responsible for coordinating the city’s technology resources) – need to engage better with mapping/data/online communities in a much more open, collaborative way.

Despite the city’s talk of apps and open data, there’s still very much a closed approach on the city’s part when it comes to public/private partnerships. True, the city has developed partnerships with local startup tech companies. But the city’s nonprofit and academic communities, along with established private entities, have much to share and have proven they have the technological resources to do as good if not a better job than the city providing essential information online.

In terms of mapping Hurricane Irene in NYC, NGOs filled a big void. The city should not only recognize that effort, but cultivate it and help sustain it so that it works more smoothly and effectively next time.

Coastal storm impact risk mapped on Long Island

UPDATED (Sunday, Oct. 28, 2012 11am)

In anticipation of Hurricane Sandy, you can use the Long Island Index mapping site to view the areas that would be at greatest risk from the storm.

You can also use Nassau County‘s or Suffolk County‘s websites for more information.


ORIGINAL POST (August 2011)

The latest tracking information for Hurricane Irene (as of Friday morning, 8/26) shows that the storm is likely going to pass east of New York City and make a head on collision with Long Island.  Newsday is reporting that it will hit western Suffolk County’s south shore on Saturday with “tropical-storm-force winds” and then ramp up to 110 mph winds by Sunday.  Yikes!

To help prepare for the storm, our team at the CUNY Graduate Center in collaboration with the Long Island Index has updated the Index’s mapping website with areas at greatest risk of hurricane-level storms.

Here’s the map:

This news release [PDF] provides more information.  The yellow-to-red shaded areas are “coastal storm impact zones.”  In 2005, the New York State Office of Emergency Management developed a map of areas that would be at greatest risk of hurricane impacts based on wind speed and other factors.

The shorthand for the map is a “SLOSH” map, because the zones of impact are based on NOAA’s “Sea, Lake and Overland Surge from Hurricanes” (SLOSH) model projections of vertical surge heights associated with category 1 – 4 storms.  In order to map this information at the Index site, we downloaded the SLOSH data from the NYS GIS Clearinghouse website (metadata here) and used the following color shading to represent the zones:

  • A Category 1 storm (impact areas shown on the map in yellow) means winds of 74-95 mph.
  • A Category 2 storm (impact areas shown on the map in light orange) means winds of 96-110 mph.
  • A Category 3 storm (impact areas shown on the map in dark orange) means winds of 111-130 mph.
  • A Category 4 storm (impact areas shown on the map in red) means winds of 131 mph or more.
If you’re in an area highlighted on the map, be sure to contact your local officials and follow media reports about the hurricane’s progress. Nassau County’s Office of Emergency Management has posted a map of evacuation routes if it comes to that:
Hopefully Irene passes us by, but if not: be prepared.  Better safe than sorry.

Coastal storm impact risk mapped in NYC

UPDATED (Monday, Oct. 29, 2012 1pm)

If you need to locate any of NYC’s 76 hurricane evacuation shelters, you can use the OASIS mapping site.

You can also use NYC.gov to find out the latest with Hurricane Sandy.


UPDATED (8/25/11 9am): We’ve added a temporary map layer on OASIS showing the locations of NYC’s hurricane evacuation centers. Here’s the link: http://bit.ly/oBsUY8 . It’s easy to use:

  • Hover your mouse over each one to highlight it (the site details will also be highlighted in the panel on the right).
  • Click on a map marker to bring the site details up to the top of the list.
  • Double-click on a site in the list and the map will zoom in right to that location.

You can type in your address above the map to see if you’re in an area that’s at risk of storm impacts, and how close you are to an evacuation center. The OASIS map automatically also shows any nearby subway stations. And you can add any other layers from the Legend list to the right of the map.

For more information about Hurricane Irene and what you should do to prepare, visit NYC’s website.

UPDATED (8/24/11 12 noon): We’ve expanded the map layer showing hurricane impact zones throughout the downstate region. It now shows potential impact zones on Long Island’s south shore, as well as some areas along the coast of the Long Island Sound.

ORIGINAL POST

For the past several years the www.OASISnyc.net mapping site has displayed a map of “coastal storm impact zones” in New York City (in addition to the wealth of other mapped data included with OASIS). This coming weekend, it seems like the coastal storm map may be especially useful with city officials bracing for the risk of Hurricane Irene bringing 72 mph winds or more to the NYC region by Sunday.

In 2005, the New York State Office of Emergency Management developed a map of areas that would be at greatest risk of hurricane impacts based on wind speed and other factors. The shorthand for the map is a “SLOSH” map, because the zones of impact are based on NOAA’s “Sea, Lake and Overland Surge from Hurricanes” (SLOSH) model projections of vertical surge heights associated with category 1 – 4 storms.

In 2006, we downloaded the SLOSH data from the NYS GIS Clearinghouse website (metadata here), and used the following color shading to represent the zones:

  • A Category 1 storm (impact areas shown on the map in yellow) means winds of 74-95 mph.
  • A Category 2 storm (impact areas shown on the map in light orange) means winds of 96-110 mph.
  • A Category 3 storm (impact areas shown on the map in dark orange) means winds of 111-130 mph.
  • A Category 4 storm (impact areas shown on the map in red) means winds of 131 mph or more.

Here’s the map:

Through the OASIS website, you can easily enter your address and find out if you’re in an area that might be at greatest risk of the hurricane, depending on how severe the winds are by the time Irene moves up the coast. But the real power of OASIS’s maps is that you can do much more. For example, you can:

  • customize the OASIS map to show transit routes, schools, public housing, and libraries in or near the zones;
  • use OASIS’s mapping tools to see areas near the zones that have been recently developed (using the 1996-2010 aerial image timeline tool);
  • zoom in to see individual property boundaries and click on each one to determine ownership, zoning, and land use characteristics; and
  • find out which elected officials represent the area, as well as the local Community Board (good sources for planning and safety resources).

In the images below, I’ve used the aerial timeline slider and the dynamic transparency tool to show recent housing development in an area at risk of impact from a Category 1 storm, along South Beach on Staten Island (at the OASIS website, you can click and drag the name of any map layer in the Legend to the “Transparency Control” box at the bottom of the legend and make it more or less transparent, so the layers underneath can shine through):

Sparse housing near the beach (1996)

Dense housing development (2010)

The zones mapped on OASIS closely mirror the city’s hurricane evacuation zones (mapped here [PDF] and described here). The city also provides a Hurricane Evacuation Zone Finder where you can enter an address and get useful safety tips depending on your zone.

The message: hopefully Irene passes us by, but if not: be prepared. Better safe than sorry.

Innovative map comparisons – Census change in 15 cities

Our team at the Center for Urban Research (at the CUNY Graduate Center) has updated our interactive maps showing race/ethnicity patterns from 2000 and 2010 in major cities across the US. We’ve enhanced the maps in several ways:

  1. Added more cities. We now have 15 major urban regions mapped across the US (Atlanta, Baltimore, Boston, Charlotte, Chicago, Detroit, Houston, Los Angeles, Miami, New York, Orlando, Philadelphia, Phoenix, San Francisco, and Washington D.C.).
  2. The maps now have three ways of comparing 2000 and 2010 racial patterns:
  3. We color-coded the population change data in the popup window. Population increase is shown in green; decrease is shown in red. See image below.

Here’s our news release with more info.

Btw, we’ve also updated our static maps to show New York City Council districts, to begin to get a sense of how demographic changes will shape upcoming redistricting efforts at the local level.  Here’s the link:www.urbanresearchmaps.org/plurality/nyccouncil.htm (For the static maps, you can view 2000-2010 demographic change with the vertical slider bar, but you can’t zoom in/out, etc.)

An initial version of the maps launched in June with the vertical bar technique, integrating it with interactive, online maps for the first time. Our Center crafted the maps so you could not only drag the bar left and right but also zoom in and out, click on the map to obtain detailed block-level population counts, and change the underlying basemap from a street view to an aerial image (via OpenLayers use of Microsoft’s Bing maps tiles), while also changing the transparency of the thematic Census patterns.

The latest iteration of CUNY’s Census maps continues to use the vertical slider but now incorporates this technique with two more comparison options. Each approach serves different purposes:

  1. The vertical slider bar provides a “before (2000) and after (2010)” visualization of change, either regionally or at the scale of a city neighborhood.
  2. The side-by-side comparison is ideal for lingering over a given area, especially at the local level, taking the time to absorb the differences in demographic patterns mapped with 2000 Census data on the left and 2010 on the right. We incorporated this approach specifically at the suggestion of the great interactive team at the Chicago Tribune, who have created some similar Census maps.
  3. The single-map 2010/2000 overlay is especially helpful for revealing the increase in diversity over a given area.

For example, you can zoom to Atlanta, GA on the single-map overlay and see the city’s predominantly Black population in 2000 surrounded by suburban Census blocks shaded dark blue, denoting a White population of 90% or more (see images below). As you transition the map from 2000 to 2010, the dark blue in the suburbs fades to a lighter shade (indicating a more mixed population demographically) coupled with more Census blocks shaded green, purple, and orange – each corresponding to communities that are now predominantly (even if only by a few percentage points) Hispanic, Asian, or Black respectively. This pattern is replicated in many of the urban regions featured at the website.

Atlanta & suburbs in 2000

Race/ethnicity change in Atlanta by 2010

Eventually we’ll be moving all this from pre-rendered tiles to vector tiles. CUR’s application architect Dave Burgoon contributed code he developed to TileStache to enable TileStache to produce AMF-based output for use in Flash-based interactive mapping applications. This will give us flexibility in mapping as many Census variables as needed, and also providing complete geographic coverage (hopefully down to the block level) nationwide. That’s the plan, anyway! Stay tuned.

Credits

Funding for much of the Center’s recent work on Census issues has been provided by the Building Resilient Regions Project of the John D. and Catherine T. MacArthur Foundation, the Hagedorn Foundation, as well as support from the CUNY Graduate Center and the City University of New York.

Several people provided feedback and helpful editorial suggestions on earlier versions of the maps and narrative. Though the materials at this site were prepared by the Center for Urban Research, those invdividuals improved our work. We greatly appreciate their contributions.

Slippy maps, meet before-and-after jQuery slider (introductions by OpenLayers)

Our team at the Center for Urban Research (at the CUNY Graduate Center) has launched a set of maps showing race/ethnicity patterns from 2000 and 2010 in major cities across the US.  The maps combine several mapping/web technologies that offer a new way of visualizing population change.  This post explains how we did it.

(And by popular demand, we’ve also included a map of Congressman Anthony Weiner’s district in relation to demographic change — you may have heard of him and his Twitter travails recently?)

Race/Ethnicity Change

Briefly, the maps show race/ethnicity change from 2000 to 2010 at the local level throughout major urban regions across the U.S.  So far we include New York City, Los Angeles, Boston, Chicago, Houston, and San Francisco.  (Others are coming soon.)

For our methodology and data analysis (and static maps), we provide that here.  For the mapping and web techniques, see below.

Reactions

So far we’ve received a pretty good response to our maps.  Here are some tweets posted recently:

  • @dancow (web journalist for ProPublica): Cool before/after map from CUNY’s urban research center showing NYC ethnic changes at the block level, from 2000-10.
  • @mericson (deputy graphics editor at NY Times): Nice block-level maps by @SR_spatial & CUNY Urban Research Center showing racial/ethnic change in NYC from 2000 to 2010.
  • @kelsosCorner (former Washington Post cartographer): Digging new 2010 Census plurality maps of NYC.
  • @albertsun (graphics editor at Wall St Journal): Coolest census map I’ve seen yet.
  • @PJoice (HUD employee; tweets are his own): This is the coolest map I have ever seen. Nice work by @SR_spatial and CUNY!
  • @MapLarge: I like how you can use the slider or move the map! Great Visualization!

Technical overview

The map uses the “before and after” technique that media websites have used for images of natural disasters.  We enhanced this technique by integrating it with interactive maps using OpenLayers, the open source mapping framework.  Now the slider works with two sets of overlapping, but perfectly aligned, maps from 2000 and 2010.

As it turns out, we didn’t set out to create an interactive version of these maps. In fact, we originally created static maps, but everyone we showed them to for feedback wanted the ability to zoom in/out and click on the map for more info.  So we developed the OpenLayers version. (And when I say “we”, that mainly means David Burgoon, CUR’s application architect, who I can’t say enough good things about.  I made the maps, and CUR’s Joe Pereira of the CUNY Data Service created the data sets, but Dave brought it all to life.)

OpenLayers enables us to introduce interactivity into the before-and-after images. Maps like these (to our knowledge) have not been available before — where you can move a slider back and forth while also zooming in/out and clicking on individual Census blocks for detailed information. You can also change the transparency of the thematic map layer, and switch between a street view and aerial view basemap.

It involved a good amount of work to integrate the slider technique with OpenLayers and also have two overlapping map instances working in tandem. The two maps need to appear as one, and this involves painstaking effort to ensure that the pixels on your screen are translated accurately to latitude/longitude coordinates in each of the separate but related interactive map instances, and the maps pan together seamlessly as you drag the slider left or right or move the map and it crosses the slider.

Mashup

In order to create the application, we used a mix of software applications, technologies, and techniques, summarized below:

  • We used the statistical software package SPSS to extract the Census block-level data for both years (see our methodology), allocate the 2000 data to 2010 blocks using the Census Bureau’s block equivalency files, and calculate the race/ethnicity plurality for each block.
  • We exported these SPSS files in DBF format and used ESRI’s ArcGIS Desktop to join the DBFs with 2010 TIGER Census block shapefiles.
  • ArcGIS Desktop was also used to create the choropleth maps (based on color schemes from ColorBrewer.org);
  • The map layouts were published as temporary web map services using ESRI’s ArcGIS Server. We used these to create pre-cached tiles (.PNG files) for the 2000 and 2010 maps, corresponding to zoom levels 4 through 10 using the now-standard Google-Microsoft map scales for online web maps. (Our application accesses the choropleth tiles as PNGs directly from the cache created by ArcGIS Server, rather than accessing the ArcGIS web map service in order to assemble the tiles. The latter approach would be too slow and would undermine the transition as you dragged the slider across the map.)
  • The slider technique was adapted from the jQuery plugin by www.catchmyfame.com.
  • OpenLayers provides all the map navigation and serving the maps themselves, modified with customized JavaScript code.
  • The basemap shown beneath the color-shaded map tiles is provided by Microsoft’s Bing map service. The street map and aerial image tiles from Bing are accessed directly via OpenLayers, rather than using the Bing API. This is a key reason we used Bing for these maps; if we used Google Maps as a basemap, we were limited to accessing Google Maps via Google’s API, which would have slowed map drawing times and undermined the slider effect.
  • For geocoding we use the Yahoo! Placefinder API.
  • Some browsers are not able to handle the before/after slider effect smoothly. In particular, Firefox and Safari perform poorly; the slider transition between one map to the other is not smooth. Microsoft’s Internet Explorer is adequate, but Google’s Chrome browser is best.

Data sources/issues

We used block-level data from the Census Bureau’s 100% population counts from the 2000 and 2010 decennial censuses (from Table P2 in the “PL-94-171” files for 2000 and 2010).

The Census Bureau’s block geography changed between 2000 and 2010 — new blocks were created, blocks were merged, and block boundaries were modified in many places. In order to compare population data from 2000 and 2010 using a common set of blocks, we used the Census Bureau’s block relationship file to allocate the 2000 population counts to 2010 geography.

When you’re viewing the map, it is best to use the maps and block-level data to understand trends over a larger area, even over several blocks. Be careful when viewing a specific block on its own. It covers a small area, and the Census Bureau may have made errors.

Credits

Funding for much of the Center’s recent work on Census issues has been provided by the Building Resilient Regions Project of the John D. and Catherine T. MacArthur Foundation, the Hagedorn Foundation, as well as support from the CUNY Graduate Center and the City University of New York.

Several people provided feedback and helpful editorial suggestions on earlier versions of the maps and narrative. Though the materials at this site were prepared by the Center for Urban Research, those invdividuals improved our work. We greatly appreciate their contributions.

OASISnyc.net map updates

Today our team at the CUNY Graduate Center updated the www.OASISnyc.net mapping site with lots of new data.  There’s more to come by summer’s end, but here’s the latest:

  • The biggest change is that we’ve added the latest community garden inventory from GrowNYC.  Over a year in the making, it comes just in time to provide valuable context for the proposed new rules for gardens being considered by the city.  More info at the OASIS wiki.
  • We’ve integrated the latest subway and bus data that I’ve blogged about earlier (here and here), and also added bike routes via the NYC Dept of Transportation (one of the latest city agencies embracing #opendata).

(Lots of bike routes in Brooklyn.)

  • A side note to the subway routes is that we’ve responded to user requests and added the ability to turn on/off the subway routes separately from the streets and rail lines.  (May not seem like a lot, but hopefully this will make it easier to view an already busy map.)

  • We also have gotten around to updating New York City parks and playgrounds data based on the Datamine release from last year.  I’ll blog more about this separately, since it was a much more involved process than it should have been, given what I now realize are some major limitations to the city’s effort via Datamine and BigApps to provide open access to data.
  • The latest zoning districts, scenic districts, and historic houses are also added to OASIS.
  • The search options for community gardens, Stew-Map turfs, and the NY/NJ Harbor’s Comprehensive Restoration Plan data have been updated.
  • We’re starting to use the OASIS wiki again for tutorials and examples of how OASIS can benefit local groups. For examples, see http://oasisnyc.gc.cuny.edu/index.php/Featured_Maps
  • Finally, we’ve started to incorporate historic maps of Manhattan in partnership with the NY Public Library.  I blogged about this earlier, and we have a post on our wiki with more info.

We’ll have more updates soon.  Historic land use parcel by parcel citywide (with an accompanying statistical analysis), more data (such as parcels & open space) for northern New Jersey, more Google Earth/KML links, etc.

Stay tuned!

@NYPLMaps & OASIS provide context for 18th century ship find

The www.OASISnyc.net mapping team has been working with the great folks at New York Public Library’s Map Division to integrate digitized historic maps aligned to the city’s current street grid.  But as we were working with Map Division staff to incorporate their maps, an amazing find at the World Trade Center construction site prompted us to speed up our work — earlier this month, construction workers unearthed an 18th century ship, largely intact, that likely hadn’t been disturbed for over 200 years.

Now you can display some key maps of lower Manhattan from the from the 18th and 19th centuries, view them in relation to the current street grid, and compare them to each other using OASIS’s dynamic transparency tool.  We added a brief tutorial at the OASIS wiki.

Now you can fade between current property maps …

… the 1775 Montresor map …

… the 1817 Poppleton map …

… and more.

We’ve also added the Viele map from 1874, and more are on their way.  This is all due to the groundbreaking NYPL “Map Rectifier” project.

Homage to the people behind OASISnyc

Last week’s UrbanOmnibus features an article I wrote about a new and completely revamped version of the OASIS mapping website in New York City — see “A new OASIS for New York“.  (Also see an earlier blog post about how we designed the cartography in the new OASIS maps.)

OASIS is the Open Accessible Space Information System.  The UO piece focuses on the site’s new mapping tools, richer data sets, and a new approach using “web 2.0″ techniques to encourage more interaction and engagement via the OASIS maps with urban planning in New York.

But equally impressive, though not the focus of the piece, is the part of OASIS that doesn’t directly involve mapping and web technology.  It’s the people and groups “behind the scenes” that make it all worthwhile.  Their work and the collaborative effort that OASIS has helped facilitate are really amazing.

OASIS was the vision of several people in the Forest Service (Jim Lyons, Michael Rains, Phil Rodbell, Matt Arnn, Robin Morgan to name a few), ESRI (especially Dave LaShell), and local greening organizations in New York back in late 2000.  The idea was to bring together a bunch of groups interested in sharing resources and ideas about open space stewardship, create an online mapping site to integrate all this info (way before Google Maps was on the scene), and see where that would lead.

The OASIS mapping site is powerful, but the website without the people and partnerships would just be one more (though impressive) map mashup.  The collaborative nature of the effort from the start — inspired and sustained especially by Matt Arnn — always seemed special.

Some folks who deserve special mention (though this is certainly an incomplete list) are:

  • Council on the Environment of NYCLenny Librizzi at CENYC is absolutely wonderful, and has a great vision of involving students and community groups in the greening of the city.  He trains high school youth to inventory street trees in a way that teaches ecology, math, science, and urban planning in an engaging way.  He’s also led the effort to map community gardens across the city, maintaining a comprehensive database of gardens for analysis, advocacy, and teaching.  It’s been great working with Lenny as we’ve integrated all that data into OASIS.  The maps are powerful, but in some ways it’s more important that they’ve been a vehicle to get to know him and to give his students invaluable hands-on experiences.
  • Open Road of New York  — Paula Hewitt Amram is an inspiration to untold numbers of city youth who want to change the world or are just looking for a better spot to play in.  (Here’s an example: Amy Poehler interviewing 11-year old Valentine about her community gardening work at Open Road.)  She’s brought that energy and spirit to the groups involved in OASIS, with a real passion for wanting the collaborative effort to endure so it will continue to be an educational and participatory resource. 
  • The greening groups involved in GROW (Grassroots Reassuring OASIS Works) – they coordinated a series of focus groups early on to make sure that whatever online mapping tools were created, were developed with an eye toward the neighborhood organizations and activists who needed the information the most.  Their insights continue to resonate with the OASIS participants, even as the GROW coordinators Wendy Brawer and Hugh Hogan have gone on to do other amazing things.
  • The Forest Service team at its Northern Research StationErika Svendsen has consistently pointed out that maps of buildings and park boundaries are nice, but maps also need to convey a sense of who’s doing what where: the people and their activities in any given geographic area are obviously and critically important.  This interest enhanced the Forest Service’s Living Memorials Project, and simultaneously was a key theme in discussions among OASIS participants about the need to understand local environmental stewardship (who’s doing what environmental work where).  Eventually Erika, Lindsay Campbell, and the Forest Service’s NYC Urban Field Station turned this concept into reality with the unique Stewardship Mapping and Assessment Project (STEW-MAP).  Erika and Lindsay and their colleagues provide a refreshing perspective to us online cartographers, and hopefully this will also become obvious to anyone using the new OASIS maps when they view the STEW-MAP “turfs” that are displayed.
  • ESRIDave LaShell helped develop OASIS early on and has been a consistent booster.  Though Dave works for a software company, I think in another life he’d be an environmental and community organizing visionary. ESRI has provided substantial software and technical support resources to OASIS from the beginning.  But Dave always saw far beyond that, understanding intuitively that bringing people together was the important thing, not any particular technology or software package these people happened to be using.  He also does that in his ongoing efforts at ESRI’s NYC office, and in extracurricular work — for example, he volunteers with the Academy for Urban Planning, which has the only high school level GIS program in the city.  Dave epitomizes what I said about ESRI when Matt Arnn and I accepted the Municipal Art Society’s “Certificate of Merit” award for OASIS in 2001, that it’s a private company with a public conscience.
  • Individuals like Jane Sokolow, Bob Alpern, and Jack Eichenbaum – I’ve known Bob and Jack for years, and more recently Jane, and it’s been wonderful and enriching working with them together on OASIS.  Also, my colleague Christy Spielman was involved with OASIS at the start, designing the maps and managing the data and website.  But she’s been critical at coordinating and facilitating the various organizational partnerships that have developed through OASIS; those contributions have been as important as her GIS & graphic design skills.  And Dave Burgoon, who’s a dream to work with, has used his programming skills to make the new OASIS website worthy of this recent praise from a financial blogger at Reuters: “This is the most amazing map of NYC I’ve ever played with. Just, wow.”
  • Last but not least, the Forest Service itself.  When I first heard that the Forest Service was convening a meeting in 2000 to talk about mapping the city’s “urban canopy” (i.e., trees), I was skeptical.  I was most familiar with their work managing land in the western US or fighting forest fires, not working in cities, let alone New York.  But even then they had developed a strong argument for caring about trees and open spaces in urban areas – both for environmental reasons as well as for the ability to relatively easily engage large numbers of people in these densely populated areas.  Phil, Matt, Robin, et al. provided great and consistent support for OASIS in its early years and still value the benefits that this collaborative effort has provided.  It’s been terrific getting to know them and work with them.

Of course there are too many people and groups to mention in detail, so please visit the OASIS participants page to see the full list.  (And even on that page we’ve probably missed a few.) 

Although active participation in the steering committee has ebbed and flowed (it’s probably at a low point at the moment, with most of the effort going toward the website redesign), you can still get involved.  Here are some links with more info:

All in all, OASIS has made its impact felt – in me and my professional and personal relationships, in the work of many people and groups across the city, and hopefully beyond all that to the city and metropolitan region at large.  I’m thrilled to have had the opportunity to have been (and to continue to be) a part of it all.

Twittering observations

I attended the GeoWeb 2009 conference and one of my “takeaways” didn’t have anything to do with geography or the geoweb at all (directly, anyway).

I noticed that at each session, some dozen or so attendees were typing away at their laptops or iPhones.  I assumed they were just taking notes, but when I peered closer over their shoulder(s) I realized they were tweeting.  I had my new Palm Pre with me, and had signed up a few weeks before for a Twitter account, so I investigated further.  (At the time I was using the @oasisnycmaps handle, though I’ve added another one since then — and maybe there’ll be more, read on!)

Each of them were adding the #geoweb or #geoweb2009 hashtags to their tweets, so I could easily search and see all of the commentary.  Mainly they were posting comments about whatever session they happened to be attending.  But then they were also commenting on each other’s tweets, retweeting, and posting detailed feedback (like Anthony Beck’s neat MindMaps).  It was a multi-way, simultaneous, interactive, Twitter-ized conversation among attendees across different sessions and even in the same session.  Cool.

Btw, my sense of Twitter vs. FB/FF, given the recent acquisition, is that it’s somewhat apples and oranges.  To me (relative newbie), Facebook is “personal social” (updates on all manner of personal idiosyncracies) while Twitter is more apt for “professional social”, based on my experience at GeoWeb and since then following people in my professional space.

Anyway, I’m hooked.  I’ve added a Twitter handle (SR_spatial — though maybe I’ll drop the underscore, that seems to be the conventional wisdom) and started a blog (this one!).  Looking forward to diving into this brave new virtual world (but with quite tangible utility in the real one).