What Should We Do About Big Data Leaks?


Paul Ford at the New Republic: “I have a great fondness for government data, and the government has a great fondness for making more of it. Federal elections financial data, for example, with every contribution identified, connected to a name and address. Or the results of the census. I don’t know if you’ve ever had the experience of downloading census data but it’s pretty exciting. You can hold America on your hard drive! Meditate on the miracles of zip codes, the way the country is held together and addressable by arbitrary sets of digits.

You can download whole books, in PDF format, about the foreign policy of the Reagan Administration as it related to Russia. Negotiations over which door the Soviet ambassador would use to enter a building. Gigabytes and gigabytes of pure joy for the ephemeralist. The government is the greatest creator of ephemera ever.

Consider the Financial Crisis Inquiry Commission, or FCIC, created in 2009 to figure out exactly how the global economic pooch was screwed. The FCIC has made so much data, and has done an admirable job (caveats noted below) of arranging it. So much stuff. There are reams of treasure on a single FCIC web site, hosted at Stanford Law School: Hundreds of MP3 files, for example, with interviews with Jamie Dimonof JPMorgan Chase and Lloyd Blankfein of Goldman Sachs. I am desperate to find  time to write some code that automatically extracts random audio snippets from each and puts them on top of a slow ambient drone with plenty of reverb, so that I can relax to the dulcet tones of the financial industry explaining away its failings. (There’s a Paul Krugman interview that I assume is more critical.)

The recordings are just the beginning. They’ve released so many documents, and with the documents, a finding aid that you can download in handy PDF format, which will tell you where to, well, find things, pointing to thousands of documents. That aid alone is 1,439 pages.

Look, it is excellent that this exists, in public, on the web. But it also presents a very contemporary problem: What is transparency in the age of massive database drops? The data is available, but locked in MP3s and PDFs and other documents; it’s not searchable in the way a web page is searchable, not easy to comment on or share.

Consider the WikiLeaks release of State Department cables. They were exhausting, there were so many of them, they were in all caps. Or the trove of data Edward Snowden gathered on aUSB drive, or Chelsea Manning on CD. And the Ashley Madison leak, spread across database files and logs of credit card receipts. The massive and sprawling Sony leak, complete with whole email inboxes. And with the just-released Panama Papers, we see two exciting new developments: First, the consortium of media organizations that managed the leak actually came together and collectively, well, branded the papers, down to a hashtag (#panamapapers), informational website, etc. Second, the size of the leak itself—2.5 terabytes!—become a talking point, even though that exact description of what was contained within those terabytes was harder to understand. This, said the consortia of journalists that notably did not include The New York Times, The Washington Post, etc., is the big one. Stay tuned. And we are. But the fact remains: These artifacts are not accessible to any but the most assiduous amateur conspiracist; they’re the domain of professionals with the time and money to deal with them. Who else could be bothered?

If you watched the movie Spotlight, you saw journalists at work, pawing through reams of documents, going through, essentially, phone books. I am an inveterate downloader of such things. I love what they represent. And I’m also comfortable with many-gigabyte corpora spread across web sites. I know how to fetch data, how to consolidate it, and how to search it. I share this skill set with many data journalists, and these capacities have, in some ways, become the sole province of the media. Organs of journalism are among the only remaining cultural institutions that can fund investigations of this size and tease the data apart, identifying linkages and thus constructing informational webs that can, with great effort, be turned into narratives, yielding something like what we call “a story” or “the truth.” 

Spotlight was set around 2001, and it features a lot of people looking at things on paper. The problem has changed greatly since then: The data is everywhere. The media has been forced into a new cultural role, that of the arbiter of the giant and semi-legal database. ProPublica, a nonprofit that does a great deal of data gathering and data journalism and then shares its findings with other media outlets, is one example; it funded a project called DocumentCloud with other media organizations that simplifies the process of searching through giant piles of PDFs (e.g., court records, or the results of Freedom of Information Act requests).

At some level the sheer boredom and drudgery of managing these large data leaks make them immune to casual interest; even the Ashley Madison leak, which I downloaded, was basically an opaque pile of data and really quite boring unless you had some motive to poke around.

If this is the age of the citizen journalist, or at least the citizen opinion columnist, it’s also the age of the data journalist, with the news media acting as product managers of data leaks, making the information usable, browsable, attractive. There is an uneasy partnership between leakers and the media, just as there is an uneasy partnership between the press and the government, which would like some credit for its efforts, thank you very much, and wouldn’t mind if you gave it some points for transparency while you’re at it.

Pause for a second. There’s a glut of data, but most of it comes to us in ugly formats. What would happen if the things released in the interest of transparency were released in actual transparent formats?…(More)”

Social app for refugees and locals translates in real-time


Springwise: “Europe is in the middle of a major refugee crisis, with more than one million migrants arriving in 2015 alone. Now, developers in Stockholm are coming up with new ways for arrivals to integrate into their new homes.

Welcome! is an app based in Sweden, a country that has operated a broadly open policy to immigration in recent years. The developers say the app aims to break down social and language barriers between Swedes and refugees. Welcome! is translated into Arabic, Persian, Swedish and English, and it enables users to create, host and join activities, as well as ask questions of locals, chat with new contacts, and browse events that are nearby.

The idea is to solve one of the major difficulties for immigrants arriving in Europe by encouraging the new arrivals and locals to interact and connect, helping the refugees to settle in. The app offers real-time auto-translation through its four languages, and can be downloaded for iOS and Android….We have already seen an initiative in Finland helping to set up startups with refugees…(More)

Evaluating e-Participation: Frameworks, Practice, Evidence


Book edited by Georg Aichholzer, Herbert Kubicek and Lourdes Torres: “There is a widely acknowledged evaluation gap in the field of e-participation practice and research, a lack of systematic evaluation with regard to process organization, outcome and impacts. This book addresses the state of the art of e-participation research and the existing evaluation gap by reviewing various evaluation approaches and providing a multidisciplinary concept for evaluating the output, outcome and impact of citizen participation via the Internet as well as via traditional media. It offers new knowledge based on empirical results of its application (tailored to different forms and levels of e-participation) in an international comparative perspective. The book will advance the academic study and practical application of e-participation through fresh insights, largely drawing on theoretical arguments and empirical research results gained in the European collaborative project “e2democracy”. It applies the same research instruments to a set of similar citizen participation processes in seven local communities in three countries (Austria, Germany and Spain). The generic evaluation framework has been tailored to a tested toolset, and the presentation and discussion of related evaluation results aims at clarifying to what extent these tools can be applied to other consultation and collaboration processes, making the book of interest to policymakers and scholars alike….(More)”

The creative citizen unbound


The creative citizen unbound

Book by Ian Hargreaves and John Hartley on “How social media and DIY culture contribute to democracy, communities and the creative economy”: “The creative citizen unbound introduces the concept of ‘creative citizenship’ to explore the potential of civic-minded creative individuals in the era of social media and in the context of an expanding creative economy. Drawing on the findings of a 30-month study of communities supported by the UK research funding councils, multidisciplinary contributors examine the value and nature of creative citizenship, not only in terms of its contribution to civic life and social capital but also to more contested notions of value, both economic and cultural. This original book will be beneficial to researchers and students across a range of disciplines including media and communication, political science, economics, planning and economic geography, and the creative and performing arts….(More)”

Crowdsourcing Human Rights


Faisal Al Mutar at The World Post: “The Internet has also allowed activists to access information as never before. I recently joined the Movements.org team, a part of the New York-based organization, Advancing Human Rights. This new platform allows activists from closed societies to connect directly with people around the world with skills to help them. In the first month of its launch, thousands of activists from 92 countries have come to Movements.org to defend human rights.

Movements.org is a promising example of how technology can be utilized by activists to change the world. Dissidents from some of the most repressive dictatorships — Russia, Iran, Syria and China — are connecting with individuals from around the globe who have unique skills to aid them.

Here are just a few of the recent success stories:

  • A leading Saudi expert on combatting state-sponsored incitement in textbooks posted a request to speak with members of the German government due to their strict anti-hate-speech laws. A former foundation executive connected him with senior German officials.
  • A secular Syrian group posted a request for PR aid to explain to Americans that the opposition is not comprised solely of radical elements. The founder of a strategic communication firm based in Los Angeles responded and offered help.
  • A Yemeni dissident asked for help creating a radio station focused on youth empowerment. He was contacted by a Syrian dissident who set up Syrian radio programs to offer advice.
  • Journalists from leading newspapers offered to tell human rights stories and connected with activists from dictatorships.
  • A request was created for a song to commemorate the life of Sergei Magnitsky, a Russia tax lawyer who died in prisoner. A NYC-based song-writer created a beautiful song and activists from Russia (including a member of Pussy Riot) filmed a music video of it.
  • North Korean defectors posted requests to get information in and out of their country and technologists posted offers to help with radio and satellite communication systems.
  • A former Iranian political prisoner posted a request to help sustain his radio station which broadcasts into Iran and helps keep information flowing to Iranians.

There are more and more cases everyday….(More)

How to Crowdsource the Syrian Cease-Fire


Colum Lynch at Foreign Policy: “Can the wizards of Silicon Valley develop a set of killer apps to monitor the fragile Syria cease-fire without putting foreign boots on the ground in one of the world’s most dangerous countries?

They’re certainly going to try. The “cessation of hostilities” in Syria brokered by the United States and Russia last month has sharply reduced the levels of violence in the war-torn country and sparked a rare burst of optimism that it could lead to a broader cease-fire. But if the two sides lay down their weapons, the international community will face the challenge of monitoring the battlefield to ensure compliance without deploying peacekeepers or foreign troops. The emerging solution: using crowdsourcing, drones, satellite imaging, and other high-tech tools.

The high-level interest in finding a technological solution to the monitoring challenge was on full display last month at a closed-door meeting convened by the White House that brought together U.N. officials, diplomats, digital cartographers, and representatives of Google, DigitalGlobe, and other technology companies. Their assignment was to brainstorm ways of using high-tech tools to keep track of any future cease-fires from Syria to Libya and Yemen.

The off-the-record event came as the United States, the U.N., and other key powers struggle to find ways of enforcing cease-fires from Syria at a time when there is little political will to run the risk of sending foreign forces or monitors to such dangerous places. The United States has turned to high-tech weapons like armed drones as weapons of war; it now wants to use similar systems to help enforce peace.

Take the Syria Conflict Mapping Project, a geomapping program developed by the Atlanta-based Carter Center, a nonprofit founded by former U.S. President Jimmy Carter and his wife, Rosalynn, to resolve conflict and promote human rights. The project has developed an interactive digital map that tracks military formations by government forces, Islamist extremists, and more moderate armed rebels in virtually every disputed Syrian town. It is now updating its technology to monitor cease-fires.

The project began in January 2012 because of a single 25-year-old intern, Christopher McNaboe. McNaboe realized it was possible to track the state of the conflict by compiling disparate strands of publicly available information — including the shelling and aerial bombardment of towns and rebel positions — from YouTube, Twitter, and other social media sites. It has since developed a mapping program using software provided by Palantir Technologies, a Palo Alto-based big data company that does contract work for U.S. intelligence and defense agencies, from the CIA to the FBI….

Walter Dorn, an expert on technology in U.N. peace operations who attended the White House event, said he had promoted what he calls a “coalition of the connected.”

The U.N. or other outside powers could start by tracking social media sites, including Twitter and YouTube, for reports of possible cease-fire violations. That information could then be verified by “seeded crowdsourcing” — that is, reaching out to networks of known advocates on the ground — and technological monitoring through satellite imagery or drones.

Matthew McNabb, the founder of First Mile Geo, a start-up which develops geolocation technology that can be used to gather data in conflict zones, has another idea. McNabb, who also attended the White House event, believes “on-demand” technologies like SurveyMonkey, which provides users a form to create their own surveys, can be applied in conflict zones to collect data on cease-fire violations….(More)

Pigeon patrol takes flight to tackle London’s air pollution crisis


 at The Guardian: They’ve been driven from Trafalgar square for being a nuisance, derided as rats with wings and maligned as a risk to public health.

But now pigeons could play a small part in helping Londoners overcome one of the capital’s biggest health problems – its illegal levels of air pollution blamed for thousands of deaths a year.

On Monday, a flock of half a dozen racing pigeons were set loose from a rooftop in Brick Lane by pigeon fancier, Brian Woodhouse, with one strapped with a pollution sensor to its back and one with a GPS tracker.

But while the 25g sensor records the nitrogen dioxide produced by the city’s diesel cars, buses, and trucks and tweets it at anyone who asks for a reading, its real purpose – and the use of the pigeons – is to raise awareness.

“It is a scandal. It is a health and environmental scandal for humans – and pigeons. We’re making the invisible visible,” said Pierre Duquesnoy, who won a London Design Festival award for the idea last year.

“Most of the time when we talk about pollution people think about Beijing or other places, but there are some days in the year when pollution was higher and more toxic in London than Beijing, that’s the reality.”

He said he was inspired by the use of pigeons in the first and second world wars to deliver information and save lives, but they were also a practical way of taking mobile air quality readings and beating London’s congested roads. They fly relatively low, at 100-150ft, and fast, at speeds up to 80mph.

“There’s something about taking what is seen as a flying rat and reversing that into something quite positive,” said Duquesnoy, who is creative director at marketing agency DigitasLBI.

Gary Fuller, an air quality expert at King’s College London, said it was the first time he had heard of urban animals being put to such use.

“It’s great that unemployed pigeons from Trafalgar Square are being put to work. Around 15 years ago tests were done on around 150 stray dogs in Mexico City, showing the ways in which air pollution was affecting lungs and heart health. But this is the first time that I’ve heard of urban wild animals being used to carry sensors to give us a picture of the air pollution over our heads.”

The release of the pigeons for three days this week, dubbed the Pigeon Air Patrol, came as moderate to high pollution affected much of the city, with Battersea recording ‘very high’, the top of the scale.

Elsewhere in the UK, Stockton-on-tees and Middlesbrough recorded high pollution readings and the forecast is for moderate and possibly high pollution in urban areas in northern England and Scotland on Tuesday. Other areas will have low pollution levels….(More).

Participatory Budgeting


Cities, Data, and Digital Innovation


Paper by Mark Kleinman: “Developments in digital innovation and the availability of large-scale data sets create opportunities for new economic activities and new ways of delivering city services while raising concerns about privacy. This paper defines the terms Big Data, Open Data, Open Government, and Smart Cities and uses two case studies – London (U.K.) and Toronto – to examine questions about using data to drive economic growth, improve the accountability of government to citizens, and offer more digitally enabled services. The paper notes that London has been one of a handful of cities at the forefront of the Open Data movement and has been successful in developing its high-tech sector, although it has so far been less innovative in the use of “smart city” technology to improve services and lower costs. Toronto has also made efforts to harness data, although it is behind London in promoting Open Data. Moreover, although Toronto has many assets that could contribute to innovation and economic growth, including a growing high-technology sector, world-class universities and research base, and its role as a leading financial centre, it lacks a clear narrative about how these assets could be used to promote the city. The paper draws some general conclusions about the links between data innovation and economic growth, and between open data and open government, as well as ways to use big data and technological innovation to ensure greater efficiency in the provision of city services…(More)

How tech is forcing firms to be better global citizens


Catherine Lawson at the BBC: “…technology is forcing companies to up their game and interact with communities more directly and effectively….

Platforms such as Kritical Mass have certainly given a fillip to the idea of crowd-supported philanthropy, attracting individuals and corporate sponsors to its projects, whether that’s saving vultures in Kenya or bringing solar power to rural communities in west Africa.

Sponsors can offer funding, volunteers, expertise or marketing. So rather than imposing corporate ideas of “do-gooding” on communities in a patronising manner, firms can simply respond to demand.

HelpfulPeeps has pushed its volunteering platform into more than 40 countries worldwide, connecting people who want to share their time, knowledge and skills with each other for free.

In the UK, online platform Neighbourly connects community projects and charities with companies and people willing to volunteer their resources. For example, Starbucks has pledged 2,500 days of volunteering and has so far backed 70 community projects….

Judging by the strong public appetite for supporting good causes and campaigning against injustice on sites such as Change.org, Avaaz.org, JustGiving andGoFundMe, his assessment appears to be correct.

And LinkedIn says millions of members have signalled on their profiles that they want to serve on a non-profit board or use their skills to volunteer….

Tech companies in particular are offering expertise and skills to good causes as way of making a tangible difference.

For example, in January, Microsoft announced that through its new organisation,Microsoft Philanthropies, it will donate $1bn-worth (£700m) of cloud computing resources to serve non-profits and university researchers over the next three years…

And data analytics specialist Applied Predictive Technologies (APT) has offered its data-crunching skills to help the Capital Area Food Bank charity distribute food more efficiently to hungry people around the Washington DC area.

APT used data to develop a “hunger heat map” to help CAFB target resources and plan for future demand better.

In another project, APT helped The Cara Program – a Chicago-based charity providing training and job placements to people affected by homelessness or poverty – evaluate what made its students more employable….

And Launch, an open platform jointly founded by Nasa, Nike, the US Agency for International Development, and the US Department of State aims to provide support for start-ups and “inspire innovation”.

In the age of internet transparency, it seems corporates no longer have anywhere to hide – a spot of CSR whitewashing is not going to cut it anymore….(More)”.