The “Socratic Method” of publishing city data?
I was encouraged at the OpenGov Camp this past Sunday by an announcement from NYC DoITT that the city will be using Socrata to provide online access to its data. It’s a great platform. It doesn’t ensure that the city will actually provide good data, or update it in a timely way, or expand its available data sets — but it’s a good step forward and hopefully a harbinger of better things to come.
OpenBaltimore opened my eyes
Earlier this year I had tweeted that a new municipal data portal — OpenBaltimore — blew away sites like NYC’s Datamine.
I was asked by Alex Howard (@digiphile) for my thoughts on OpenBaltimore and other, similar portals. At the time I didn’t realize OpenBaltimore was using Socrata, but after I looked into it further, I came away impressed. The platform is visually appealing, easy to search, and offers multiple ways of accessing/extracting data.
(I don’t want to endorse the Socrata product/service, but it seems to me to be a good choice for NYC.)
Useful features, and lots of them
One nice aspect of the platform is the ability to immediately preview the data, in your browser (no downloading needed just to see what it contains). You can also view more details about each row in the file. And you can visualize the data in multiple ways — using an interactive map option built into the platform (if the dataset has a location component) or using one of 9 different chart options.
And if you want to download/export a data set, they give you at least 8 formats for extracting/exporting, as well as an API for programmatic access. NYC says that “all datasets will now be available as APIs” once they replace Datamine with NYC/Socrata.
Short links and “perma” links are available to each data set. And there’s a “Discuss” option where anyone can attach notes and commentary for each data set. It’s user-generated metadata — you can immediately see, for example, if anyone else has commented about the data’s quality, or completeness, or how up-to-date it is. I didn’t notice too many comments at the OpenBaltimore site, but there were some, and they were helpful (including responses from that city’s data team).
The option includes a map, but didn’t seem to have real time geocoding. So even if a list has street addresses, it can’t be mapped through Socrata on the fly. Each list needs a “location column” which presumably means lat/lon. (It’s easy to submit feature requests to the Socrata team, though, so hopefully we’ll be seeing this addition soon.)
However, not a silver bullet
Even though OpenBaltimore’s portal has been online for just a few months, already there are criticisms (for example, data hasn’t been updated since February, some data sets have quality problems, etc). Many people (including me) have leveled these same criticisms at NYC’s Datamine effort. So simply having a better portal won’t solve these issues.
But at least a platform like Socrata will make it easier to deploy data sets, it’ll certainly make it easier for the public to access those data sets, and it’ll make it easier to suggest improvements to the substance and the process.
NYC’s Datamine was an improvement in some ways over earlier opendata efforts in New York. Now that it’s been around for two years, I think it’s fair to say that Datamine is clunky at best. For me, I can’t wait for it to be replaced by something better. I’m looking forward to the NYC/Socrata roll out.
What do you think?