An Open Data Story
by Paul Baker
Dec 1, 2011 – In almost all open data projects, providing raw data is not enough. And it’s not enough to be a good programmer or designer. To be useful, we need to be able to figure out what questions the data might reveal and which of these might be useful for the public to know.
In early July 2011, the City of Chicago released 14 sets of lobbying-related data, including lobbyists, lobbying agencies, gifts to city employees, expenditures, clients of lobbyists, etc. However, an ordinary human being could not make heads or tails of this without some sort of integration.
That’s why we built ChicagoLobbyists.org.
Chicago Lobbyists v1.0
First, we went through each data set, attempting to draw relationships among them by looking for common fields such as clients, lobbying firms, and individual lobbyists. Our goal was to reveal answers to questions such as, “WalMart paid which lobbyist, with which lobbying firm, how much money, to argue on their behalf in front of which department or committee, and for what purpose?”
We soon found out that we couldn’t answer that question. In the above example, we could show which lobbyists Walmart paid, which firm the lobbyist worked for, and which city agencies the lobbyist appeared before, but not which tasks lobbyists were hired to do for WalMart.
Meeting with the City
After the July Google hackathon, at which we built and launched the site, we met with the city’s Chief Data Officer, Brett Goldstein, and some IT staff, and asked:
- Is there a way to determine compensation that comes to a firm versus lobbyist? (We can’t be sure adding up compensation of all lobbyists within a firm is accurate.)
- Is there a way to determine how much a client paid to a lobbying firm or lobbyist?
- Related to the question above, can we retrieve the purpose for a client payment?
- Can we receive data before 2010? If so, what years are available?
- Is there a way to determine which client requested an action in front of an agency? We only know which lobbyist requested an action—see example: Theodore Brunsvold
- Are street addresses available for lobbying firms (some give political donations from many different addresses)?
- Do you have a good state contact from which we could request political donation data?
- Is there a list of all city datasets available (besides those on Socrata)?
- Is there a schedule of when particular datasets are updated (frequency, time period)?
The City responds with updated data
By late August, Brett’s team had collected and posted the data that we had requested. Now, we are able to incorporate the new data and change the site’s design to show more detailed and new views (these changes are coming in the next day or so – stay tuned). Now we can answer most of the important questions.
Next steps …
Our next steps are to incorporate political donations to city officials by lobbyists and organizations. It would also be good to show how elected officials voted on issues of interest to lobbyists who have donated to them and their clients.
The entire 2012 city budget is about $3 billion—all of it paid by residents. Fees paid to lobbyists to influence government decisions will probably be in the neighborhood of $13 million, or about .4% of $3 billion.
We’re the 99.6%. Lobbyists paid by organizations are .4%. Are lobbyists the tail that wags the dog?
blog comments powered by Disqus
An open process
In the spirit of openness, the goal of this blog is to share our milestones, setbacks and thoughts as we continue to develop and expand this project.
Who are we?
We are Open City, a group of developers and designers based in Chicago that build civic-minded apps using open data.
- Paul Baker
- President and co-founder of Webitects
- Derek Eder
- Developer at Webitects and organizer for OpenGov Chicago
- Chad W Pry
- Engineer at Groupon, Code Academy mentor, and all around charming fellow
- Nick Rougeux
- Designer and CSS wiz at Webitects