CityBeat: Visualizing the Social Media Pulse of the City


CityBeat is a an academic research project set to develop an application that sources, monitors and analyzes hyper-local information from multiple social media platforms such as Instagram and Twitter in real time.

This project was led by researchers at the Jacobs Institute at Cornell Tech,  in collaboration with the The New York World (Columbia Journalism School), Rutgers University, NYU, and Columbia University….

If you are interested in the technical details, we have published several papers detailing the process of building CityBeat. Enjoy your read!

Xia C., Schwartz, R., Xie K., Krebs A., Langdon A., Ting J. and Naaman M., CityBeat: Real-time Social Media Visualization of Hyper-local City Data. In Proceedings, WWW 2014, Seoul, Korea, April 2014. [PDF]

Xie K., Xia C., Grinberg N., Schwartz R., and Naaman M., Robust detection of hyper-local events from geotagged social media data. In Proceedings of the 13th Workshop on Multimedia Data Mining in KDD, 2013. [PDF]

Schwartz, R., Naaman M., Matni, Z. (2013) Making Sense of Cities Using Social Media: Requirements for Hyper-Local Data Aggregation Tools. In Proceedings, WCMCW at ICWSM 2013, Boston, USA, July 2013. [PDF]

Smart Inclusive Cities: How New Apps, Big Data, and Collaborative Technologies Are Transforming Immigrant Integration


New report by Meghan Benton for the Migration Policy Institute: “The spread of smartphones—cellphones with high-speed Internet access and geolocation technology—is transforming urban life. While many smartphone apps are largely about convenience, policymakers are beginning to explore their potential to address social challenges from disaster response to public health. And cities, in North America and Europe alike, are in the vanguard in exploring creative uses for these apps, including how to improve engagement.
For disadvantaged and diverse populations, accessing city services through a smartphone can help overcome language or literacy barriers and thus increase interactions with city officials. For those with language needs, smartphones allow language training to be accessed anywhere and at any time. More broadly, cities have begun mining the rich datasets that smartphones collect, to help attune services to the needs of their whole population. A new crop of social and civic apps offer new tools to penetrate hard-to-reach populations, including newly arrived and transient groups.
While these digital developments offer promising opportunities for immigrant integration efforts, smartphone apps’ potential to address social problems should not be overstated. In spite of potential shortcomings, since immigrant integration requires a multipronged policy response, any additional tools—especially inexpensive ones—should be examined.
This report explores the kinds of opportunities smartphones and apps are creating for the immigrant integration field. It provides a first look at the opportunities and tradeoffs that smartphones and emerging technologies offer for immigrant integration, and how they might deepen—or weaken—city residents’ sense of belonging…” (Download Report)

Mapping the Next Frontier of Open Data: Corporate Data Sharing


Stefaan Verhulst at the GovLab (cross-posted at the UN Global Pulse Blog): “When it comes to data, we are living in the Cambrian Age. About ninety percent of the data that exists today has been generated within the last two years. We create 2.5 quintillion bytes of data on a daily basis—equivalent to a “new Google every four days.”
All of this means that we are certain to witness a rapid intensification in the process of “datafication”– already well underway. Use of data will grow increasingly critical. Data will confer strategic advantages; it will become essential to addressing many of our most important social, economic and political challenges.
This explains–at least in large part–why the Open Data movement has grown so rapidly in recent years. More and more, it has become evident that questions surrounding data access and use are emerging as one of the transformational opportunities of our time.
Today, it is estimated that over one million datasets have been made open or public. The vast majority of this open data is government data—information collected by agencies and departments in countries as varied as India, Uganda and the United States. But what of the terabyte after terabyte of data that is collected and stored by corporations? This data is also quite valuable, but it has been harder to access.
The topic of private sector data sharing was the focus of a recent conference organized by the Responsible Data Forum, Data and Society Research Institute and Global Pulse (see event summary). Participants at the conference, which was hosted by The Rockefeller Foundation in New York City, included representatives from a variety of sectors who converged to discuss ways to improve access to private data; the data held by private entities and corporations. The purpose for that access was rooted in a broad recognition that private data has the potential to foster much public good. At the same time, a variety of constraints—notably privacy and security, but also proprietary interests and data protectionism on the part of some companies—hold back this potential.
The framing for issues surrounding sharing private data has been broadly referred to under the rubric of “corporate data philanthropy.” The term refers to an emerging trend whereby companies have started sharing anonymized and aggregated data with third-party users who can then look for patterns or otherwise analyze the data in ways that lead to policy insights and other public good. The term was coined at the World Economic Forum meeting in Davos, in 2011, and has gained wider currency through Global Pulse, a United Nations data project that has popularized the notion of a global “data commons.”
Although still far from prevalent, some examples of corporate data sharing exist….

Help us map the field

A more comprehensive mapping of the field of corporate data sharing would draw on a wide range of case studies and examples to identify opportunities and gaps, and to inspire more corporations to allow access to their data (consider, for instance, the GovLab Open Data 500 mapping for open government data) . From a research point of view, the following questions would be important to ask:

  • What types of data sharing have proven most successful, and which ones least?
  • Who are the users of corporate shared data, and for what purposes?
  • What conditions encourage companies to share, and what are the concerns that prevent sharing?
  • What incentives can be created (economic, regulatory, etc.) to encourage corporate data philanthropy?
  • What differences (if any) exist between shared government data and shared private sector data?
  • What steps need to be taken to minimize potential harms (e.g., to privacy and security) when sharing data?
  • What’s the value created from using shared private data?

We (the GovLab; Global Pulse; and Data & Society) welcome your input to add to this list of questions, or to help us answer them by providing case studies and examples of corporate data philanthropy. Please add your examples below, use our Google Form or email them to us at corporatedata@thegovlab.org”

Ants Are Cool but Teach Us Nothing


in Bloomberg View: “…For nearly seven decades, starting in boyhood, I’ve studied hundreds of kinds of ants around the world, and this qualifies me, I believe, to offer some advice on ways their lives can be applied to ours. I’ll start with the question I’m most often asked: “What can I do about the ants in my kitchen?” My response comes from the heart: Watch your step, be careful of little lives. Ants especially like honey, tuna and cookie crumbs. So put down bits of those on the floor, and watch as the first scout finds the bait and reports back to her colony by laying an odor trail. Then, as a little column follows her out to the food, you will see social behavior so strange it might be on another planet. Think of kitchen ants not as pests or bugs, but as your personal guest superorganism.
Another question I hear a lot is, “What can we learn of moral value from the ants?” Here again I will answer definitively: nothing. Nothing at all can be learned from ants that our species should even consider imitating. For one thing, all working ants are female. Males are bred and appear in the nest only once a year, and then only briefly. They are pitiful creatures with wings, huge eyes, small brains and genitalia that make up a large portion of their rear body segment. They have only one function in life: to inseminate the virgin queens during the nuptial season. They are built to be robot flying sexual missiles. Upon mating or doing their best to mate, they are programmed to die within hours, usually as victims of predators.
Many kinds of ants eat their dead — and their injured, too. You may have seen ant workers retrieve nestmates that you have mangled or killed underfoot (accidentally, I hope), thinking it battlefield heroism. The purpose, alas, is more sinister.
As ants grow older, they spend more time in the outermost chambers and tunnels of the nest, and are more prone to undertake dangerous foraging trips. They also are the first to attack enemy ants and other intruders. Here indeed is a major difference between people and ants: While we send our young men to war, ants send their old ladies.
The most complex societies of all ant species, and arguably of all animals everywhere, are the leafcutters of the American tropics. In lowland forests and grasslands from Mexico to South America, you find conspicuous long files of reddish ants. Many carry freshly cut pieces of leaves, flowers and twigs. The ants don’t eat this vegetation. They carry it deep into their nests, where they convert it into complex, spongelike structures. On this substrate they grow a fungus, which they do eat. The entire process employs a sequence of specialists: The leafcutters in the field are medium in size. As they head home with their burdens, tiny sister ant workers ride on their backs to protect them from parasitic phorid flies. Inside the nest, workers somewhat smaller than the gatherers scissor the leaf fragments into pieces. Still smaller ants chew the fragments into lumps and add their own fecal material as fertilizer. Even smaller workers use the gooey lumps thus created to construct the gardens. And workers as small as the fly guards plant and tend the fungus.
The largest caste of leafcutter ants have razor-sharp mandibles and the adductor muscles to close them with enough force to slice mammalian skin. These soldiers defend the nest against the most dangerous predators, including anteaters.
Species that have been able to evolve superorganismic colonies — almost purely on the basis of instinct — have as a whole been enormously successful. The 20,000 or so known species of social insects make up only 2 percent of the million known species of insects but three-fourths of the insect biomass.
With complexity, however, comes vulnerability, and that brings me to one of the other superorganism superstars, the domestic honeybee. When disease strikes solitary animals that we have embraced in symbiosis, such as chickens, pigs and dogs, veterinarians can usually diagnose and fix the problem. Honeybees, on the other hand, have by far the most complex lives of all our domestic partners. There are a great many more twists and turns in their adaptation to their environment that, upon failing, can damage some part of the colony life cycle. The intractability thus far of the honeybee colony collapse disorder of Europe and North America, which threatens so much of crop pollination, may represent an intrinsic weakness of superorganisms.
You may occasionally hear human societies described as superorganisms. This is a bit of a stretch. It is true that we form societies dependent on cooperation, labor specialization and frequent acts of altruism. But where social insects are ruled almost entirely by instinct, we base labor division on transmission of culture. Also, unlike social insects, we are too selfish to behave like cells in an organism. Human beings seek their own destiny. They will always revolt against slavery, and refuse to be treated like worker ants.”

Fighting Inequality in the New Gilded Age


Book Review by K. Sabeel Rahman in the Boston Review:

White Collar Government: The Hidden Role of Class in Economic Policy Making 
Nicholas Carnes
The Promise of Participation: Experiments in Participatory Governance in Honduras and Guatemala
Daniel Altschuler and Javier Corrales
Making Democracy Fun: How Game Design Can Empower Citizens and Transform Politics
Josh Lerner

“In the years since the financial crisis, the realities of rapid economic recovery for some and stagnant wages for most has made increasingly clear that we live in a new Gilded Age: one marked by growing income inequality, decreasing social mobility, and concentrated corporate power. At the same time, we face an increasingly dysfunctional political system, apparently incapable of addressing these fundamental economic challenges.
This is not the first time the country has been caught in this confluence of economic inequality and political dysfunction. The first Gilded Age, in the late nineteenth century, experienced a similar moment of economic upheaval, instability, inequality, rising corporate power, and unresponsive government. These challenges triggered some of the most powerful reform movements in American history: the labor and antitrust movements, the Populist movement of agrarian reformers, and the Progressive movement of urban social and economic reformers. These reformers were not perfect—their record on racial and ethnic inequality is especially glaring—but they were enormously successful in creating new institutions and ideas that reshaped our economy and our politics. In particular, many of them were convinced that to address economic inequality, they had to first democratize politics, creating more robust forms of accountability and popular sovereignty against the influence of economic and political elites….
With his new book, White Collar Government: The Hidden Role of Class in Economic Policy-Making (2013), Nicholas Carnes argues that there is a third, even more important source of elite political influence: the dominance of upper class individuals in the composition of legislatures themselves. Despite the considerable external pressures of donors, constituent preferences, parties, and interest groups, legislators still possess significant discretion, and as a result their personal views about economic policy matter. Legislators of different class backgrounds, Carnes demonstrates, have distinct views on everything from labor to welfare programs and anti-poverty policies, to the very idea of government itself. On unemployment, labor rights, tax policy, and corporate protections, many of the central economic policy issues of our time involve a cleavage between wealthy and working class interests. The underrepresentation of the working class results in an underrepresentation of working class interests, exacerbating income inequality. “Whether our political system listens to one voice or another depends not just on who’s doing the talking or how loud they are,” writes Carnes; “it also depends on who’s doing the listening.”….
In The Promise of Participation: Experiments in Participatory Governance in Honduras and Guatemala (2013), Daniel Altschuler and Javier Corrales focus similar questions to those animating Carnes’ account: What institutional contexts enable ordinary citizens—especially poorer ones—to expand their representation in decision-making? What expands their knowledge of issues, their political networks, and their willingness to participate more broadly to advocate for their interests? To gain traction on this question, they undertook the first large-scale study of participatory governance, examining the nation-wide community-managed schools program in Honduras and Guatemala. These programs operated in areas that conventionally might be considered inhospitable to participatory governance: poor, rural districts. These programs engaged parents by giving them management and administrative duties in the daily activities of the school. In both countries, the programs were established to both address pervasive disparities in educational attainment, and to improve the accountability of government officials in delivering basic services to the poor….
In Making Democracy Fun: How Game Design Can Empower Citizens and Transform Politics, Lerner takes a practitioners’ look at participatory governance. Lerner is the Executive Director of the Participatory Budgeting Project, a non-profit dedicated to adapting participatory budgeting systems and implementing them in cities such as New York, Chicago, and Boston. Where Altschuler and Corrales are primarily concerned with the macro-institutional contexts that make participatory governance systems work well, Lerner’s insights revolve around the micro-practices of how to make participation effective at the face-to-face level….
Our recent experience of economic inequality has fueled the rise of a new social science of economic inequality and oligarchy, most recently and famously captured in the debates over Thomas Piketty’s Capital in the Twenty-First Century. But we also need a constructive account of what a more responsive and representative democratic politics looks like, and how to achieve it. Reformers coming out of the Gilded Age of the late nineteenth century similarly located the roots of economic inequality in political inequality. The era of Standard Oil and J.P. Morgan (the man, before the firm), and of widening income inequality was also the era of dysfunctional machine politics and a conservative Supreme Court that stymied social reform. These challenges fueled reform movements that struggled to restore popular sovereignty and genuine democracy—proposing everything from antitrust restraints on corporate power, to the first campaign finance systems, to new procedures for popular elections of Senators, party primaries, and direct democratic referenda. It was during this period that state and federal governments experimented with antitrust laws, rate regulation, and labor regulation. Many of the economic ideas first developed out of this ferment came to fruition in the New Deal.
Today we see the echoes of this zeal in the debates around campaign finance reform and the problem of “too-big-to-fail” banks. But reviving genuine democratic equality to address economic inequality requires a broader view of potential democratizing reforms. Carnes reminds us that the identity of who governs matters as much for class and economic policy as for any other dimension of representation. But Altschuler, Corrales, and Lerner suggest as well the importance of looking outside legislatures. Governing involves more than writing statutes; it is solving disputes, administering social services, implementing directives at the local level. And these are spaces where the prospects for greater political power—especially on the part of economically marginalized groups—may even be greater than at national scale legislatures. The proliferation of open government efforts in the United States—from governmental transparencyto engaging citizens to report potholes—suggests a growing reform interest in creating alternative channels for participation and representation. But too often these efforts are more limited than their rhetoric, focusing more narrowly on making existing policies well known or efficient, rather than empowering participants to challenge and reshape them. These books underscore that genuine democratic reform requires actually empowering ordinary citizens to drive the business of governing.”
 
 
 

From #Ferguson to #OfficerFriendly


at Bloomberg View: “In the tiny town of Jun, Spain, (population: 3,000) meeting rooms in city hall have their own Twitter accounts. When residents want to reserve them, they send a direct message via Twitter; when it’s time, the door to the room unlocks automatically in response to a tweet. Jun’s mayor, Jose Antonio Rodriguez, says he coordinates with other public servants via Twitter. Residents routinely tweet about public services, and city hall answers. Every police officer in Jun has a Twitter handle displayed on his uniform.
Now the New York Police Department, the largest in the U.S., is starting a broad social media initiative to get every precinct talking and listening online via Twitter, to both serve citizens and manage police personnel. The question is whether the kind of positive, highly local responsiveness the residents of Jun expect is possible across all parts of local government — not just from the police — in a big city. If it works, the benefits to the public from this kind of engagement could be enormous.
In the age of Michael Brown’s death in Ferguson, Missouri, and Eric Garner‘s in New York, when police abuses can be easily documented by citizens wielding smartphones, relationships between police departments and the communities they serve can quickly become strained. And social media use by the police runs the risk of being initially dismissed as a publicity stunt. But after decades of losing the trust of important New York City communities, this step may help the department gain civic support.
There will be bumps along the way. Last spring, the NYPD kicked off a social media campaign, asking people to share photos accompanied by the Twitter hashtag #myNYPD. Within 24 hours the hashtag was famous worldwide, as activists posted pictures of clashes between residents and the police. But Commissioner Bill Bratton brushed off the criticism, calling the pictures old news and saying the media event was not going to cause the NYPD to change its plans to be active on social media. “I welcome the attention,” he said.
Bratton will roll out a long list of social media efforts this week. The NYPD is training its dozens of commanding officers to understand and use Twitter on their own, both to ask questions and to respond timely to comments and concerns. For example, police in New York City spend a lot of time looking for missing people; now they will be able to get assistance from eyes on the street…”

Online Petitions Proposed to Offer New Yorkers a New Way to Speak Out


in The New York Times:  “Since introducing a petition site in 2011 and promising to respond to any request that received enough signatures, the White House has been compelled to release its beer recipe, inform Texas that it would not be permitted to secede and weigh the merits of a “Death Star” for national defense.

“The administration,” the response to that petition read, “does not support blowing up planets.”

So it is perhaps with some trepidation that New York City lawmakers consider a local model: an online petition system that would allow residents to ask anything they want of their public officials and, with sufficient support, receive a response.

“Not everyone can go to a public hearing,” said the bill’s sponsor, Councilman James Vacca, Democrat of the Bronx. “This would be a way for people to register their views collectively.”

The proposal to create something resembling a Reddit for the body politic was introduced on Wednesday by Mr. Vacca and referred to the City Council’s Committee on Technology, of which he is chairman. Spokesmen for Mayor Bill de Blasio and Melissa Mark-Viverito, the Council speaker, said their offices were reviewing the bill.

Mr. Vacca’s office said the petition system would be the first of its kind on the municipal level anywhere, a claim that could not be immediately confirmed. Under his bill, the city’s Department of Information Technology and Telecommunications would determine the threshold number of electronic signatures that would prompt a response. The department would also be asked to establish the website, creating a system that “allows city agencies or public authorities to post public responses” to the petitions….

Dick Dadey, the executive director of Citizens Union, a civic group, called the petition proposal “a novel idea” worthy of debate. But he sounded several notes of caution, wondering whether the setup might be subject to manipulation, favoring “a preordained outcome directed by public officials” on a given issue….”

How Open Data Is Transforming City Life


Joel Gurin, The GovLab, at Techonomy: “Start a business. Manage your power use. Find cheap rents, or avoid crime-ridden neighborhoods. Cities and their citizens worldwide are discovering the power of “open data”—public data and information available from government and other sources that can help solve civic problems and create new business opportunities. By opening up data about transportation, education, health care, and more, municipal governments are helping app developers, civil society organizations, and others to find innovative ways to tackle urban problems. For any city that wants to promote entrepreneurship and economic development, open data can be a valuable new resource.
The urban open data movement has been growing for several years, with American cities including New York, San Francisco, Chicago, and Washington in the forefront. Now an increasing number of government officials, entrepreneurs, and civic hackers are recognizing the potential of open data. The results have included applications that can be used across many cities as well as those tailored to an individual city’s needs.
At first, the open data movement was driven by a commitment to transparency and accountability. City, state, and local governments have all released data about their finances and operations in the interest of good government and citizen participation. Now some tech companies are providing platforms to make this kind of city data more accessible, useful, and comparable. Companies like OpenGov and Govini make it possible for city managers and residents to examine finances, assess police department overtime, and monitor other factors that let them compare their city’s performance to neighboring municipalities.
Other new businesses are tapping city data to provide residents with useful, practical information. One of the best examples is NextBus, which uses metropolitan transportation data to tell commuters when to expect a bus along their route. Commuter apps like this have become common in cities in the U.S. and around the world. Another website, SpotCrime, collects, analyzes, and maps crime statistics to tell city dwellers which areas are safest or most dangerous and to offer crime alerts. And the Chicago-based Purple Binder helps people in need find city healthcare services. Many companies in the Open Data 500, the study of open data companies that I direct at the GovLab at NYU, use data from cities as well as other sources….
Some of the most ambitious uses of city data—with some of the greatest potential—focus on improving education. In Washington, the nonprofit Learn DC has made data about public schools available through a portal that state agencies, community organizations, and civic hackers can all use. They’re using it for collaborative research and action that, they say, has “empowered every DC parent to participate in shaping the future of the public education system.”…”

EU: GLOW (Global Legislative Openness Week)


GLOW is a celebration of open, participatory legislative processes around the world as well as an opportunity for diverse stakeholders to collaborate with one another and make progress toward adopting and implementing open-government commitments. The week is being led by the Legislative Openness Working Group of the Open Government Partnership, which is co-anchored by the National Democratic Institute and the Congress of Chile. 
The campaign kicks off with the International Day of Democracy on September 15, and throughout the 10 days you are invited to share your ideas and experiences, kickstart new transparency tools and engage members of your community in dialogue. Learn more about the global open government movement at OGP, and stay tuned into GLOW events by following this site and #OpenParl2014.
Where will GLOW be happening?
GLOW will connect a range of legislative openness activities, organized independently by civil society organizations and parliaments around the world. You can follow the action on Twitter by using the hashtag #OpenParl2014. We hope the GLOW campaign will inspire you to design and organize your own event or activity during this week. If you’d like to share your event and collaborate with others during GLOW, please send us a note.
The week’s festivities will be anchored by two Working Group meetings of civil society and parliamentary members. Beginning on the International Day of Democracy, September 15, the Working Group will host a regional meeting on expanding civic engagement through parliamentary openness in Podgorica, Montenegro, hosted in partnership with the Parliament of Montenegro. The week will conclude with the Working Group’s annual meeting in Chile, on September 25 and 26, 2014, where members will discuss progress made in the year since the Working Group’s launch. This meeting coincides with the 11th Plenary Assembly of ParlAmericas, an independent network composed of the national legislatures of the 35 independent states of the Americas, which will also consider issues of legislative openness as part of its meeting….” (More)

Participatory Budgeting: Ten Actions to Engage Citizens via Social Media


New report by Victoria Gordon for the IBM Center for the Business of Government: “Participatory budgeting is an innovation in direct citizen participation in government decision-making that began 25 years ago in a town in Brazil. It has since spread to 1,000 other cities worldwide and is gaining interest in U.S. cities as well.
Dr. Gordon’s report offers an overview of the state of participatory budgeting, and the potential value of integrating the use of social media into the participatory process design. Her report details three case studies of U.S. communities that have undertaken participatory budgeting initiatives.  While these cases are relatively small in scope, they provide insights into what potential users need to consider if they wanted to develop their own initiatives.
Based on her research and observations, Dr. Gordon recommends ten actions community leaders can take to create the right participatory budgeting infrastructure to increase citizen participation and assess its impact.  A key element in her recommendations is to proactively incorporate social media strategies”