6 lessons from sharing humanitarian data


Francis Irving at LLRX: “The Humanitarian Data Exchange (HDX) is an unusual data hub. It’s made by the UN, and is successfully used by agencies, NGOs, companies, Governments and academics to share data.

They’re doing this during crises such as the Ebola epidemic and the Nepal earthquakes, and every day to build up information in between crises.

There are lots of data hubs which are used by one organisation to publish data, far fewer which are used by lots of organisations to share data. The HDX project did a bunch of things right. What were they?

Here are six lessons…

1) Do good design

HDX started with user needs research. This was expensive, and was immediately worth it because it stopped a large part of the project which wasn’t needed.

The user needs led to design work which has made the website seem simple and beautiful – particularly unusual for something from a large bureaucracy like the UN.

HDX front page

2) Build on existing software

When making a hub for sharing data, there’s no need to make something from scratch. Open Knowledge’s CKANsoftware is open source, this stuff is a commodity. HDX has developers who modify and improve it for the specific needs of humanitarian data.

ckan

3) Use experts

HDX is a great international team – the leader is in New York, most of the developers are in Romania, there’s a data lab in Nairobi. Crucially, they bring in specific outside expertise: frog design do the user research and design work;ScraperWiki, experts in data collaboration, provide operational management.

ScraperWiki logo

4) Measure the right things

HDX’s metrics are about both sides of its two sided network. Are users who visit the site actually finding and downloading data they want? Are new organisations joining to share data? They’re avoiding “vanity metrics”, taking inspiration from tech startup concepts like “pirate metrics“.

HDX metrics

5) Add features specific to your community

There are endless features you can add to data hubs – most add no value, and end up a cost to maintain. HDX add specific things valuable to its community.

For example, much humanitarian data is in “shape files”, a standard for geographical information. HDX automatically renders a beautiful map of these – essential for users who don’t have ArcGIS, and a good check for those that do.

Syrian border crossing

6) Trust in the data

The early user research showed that trust in the data was vital. For this reason, anyone can’t just come along and add data to it. New organisations have to apply – proving either that they’re known in humanitarian circles, or have quality data to share. Applications are checked by hand. It’s important to get this kind of balance right – being too ideologically open or closed doesn’t work.

Apply HDX

Conclusion

The detail of how a data sharing project is run really matters….(More)”

Global fact-checking up 50% in past year


Mark Stencel at Duke Reporters’ Lab: “The high volume of political truth-twisting is driving demand for political fact-checkers around the world, with the number of fact-checking sites up 50 percent since last year.

The Duke Reporters’ Lab annual census of international fact-checking currently counts 96 active projects in 37 countries. That’s up from 64 active fact-checkers in the 2015 count. (Map and List)

Active Fact-checkers 2016A bumper crop of new fact-checkers across the Western Hemisphere helped increase the ranks of journalists and government watchdogs who verify the accuracy of public statements and track political promises. The new sites include 14 in the United States, two in Canada as well as seven additional fact-checkers in Latin America.There also were new projects in 10 other countries, from North Africa to Central Europe to East Asia…..

The growing numbers have even spawned a new global association, the International Fact-Checking Network hosted by the Poynter Institute, a media training center in St. Petersburg, Florida.

Promises, Promises

Some of the growth has come in the form of promise-tracking. Since January 2015, fact-checkers launched six sites in five countries devoted to tracking the status of pledges candidates and party leaders made in political campaigns. In Tunisia, there are two new sites dedicated to promise-tracking — one devoted to the country’s president and the other to its prime minister.

There are another 20 active fact-checkers elsewhere that track promises,…

Nearly two-thirds of the active fact-checkers (61 of 96, or 64 percent) are directly affiliated with a new organization. However this breakdown reflects the dominant business structure in the United States, where 90 percent of fact-checkers are part of a news organization. That includes nine of 11 national projects and 28 of 30 state/local fact-checkers…The story is different outside the United States, where less than half of the active fact-checking projects (24 of 55, or 44 percent) are affiliated with news organizations.

The other fact-checkers are typically associated with non-governmental, non-profit and activist groups focused on civic engagement, government transparency and accountability. A handful are partisan, especially in conflict zones and in countries where the lines between independent media, activists and opposition parties are often blurry and where those groups are aligned against state-controlled media or other governmental and partisan entities….(More)

Another Tale of Two Cities: Understanding Human Activity Space Using Actively Tracked Cellphone Location Data


Paper by Yang Xu et al: “Activity space is an important concept in geography. Recent advancements of location-aware technologies have generated many useful spatiotemporal data sets for studying human activity space for large populations. In this article, we use two actively tracked cellphone location data sets that cover a weekday to characterize people’s use of space in Shanghai and Shenzhen, China. We introduce three mobility indicators (daily activity range, number of activity anchor points, and frequency of movements) to represent the major determinants of individual activity space. By applying association rules in data mining, we analyze how these indicators of an individual’s activity space can be combined with each other to gain insights of mobility patterns in these two cities. We further examine spatiotemporal variations of aggregate mobility patterns in these two cities. Our results reveal some distinctive characteristics of human activity space in these two cities: (1) A high percentage of people in Shenzhen have a relatively short daily activity range, whereas people in Shanghai exhibit a variety of daily activity ranges; (2) people with more than one activity anchor point tend to travel further but less frequently in Shanghai than in Shenzhen; (3) Shenzhen shows a significant north–south contrast of activity space that reflects its urban structure; and (4) travel distance in both cities is shorter around noon than in regular work hours, and a large percentage of movements around noon are associated with individual home locations. This study indicates the benefits of analyzing actively tracked cellphone location data for gaining insights of human activity space in different cities….(More)”

Donating Your Selfies to Science


Linda Poon at CityLab: “It’s not only your friends and family who follow your online selfies and group photos. Scientists are starting to look at them, too, though they’re more interested in what’s around you. In bulk, photos can reveal weather patterns across multiple locations, air quality of a place over time, the dynamics of a neighborhood—all sorts of information that helps researchers study cities.

At the Nanyang Technological University in Singapore, a research group is using crowdsourced photos to create a low-cost alternative to air-pollution sensors. Called AirTick, the smartphone app they’ve designed will collect photos from users and analyze how hazy the environment looks. It’ll then check each image against official air quality data, and through machine-learning the app will eventually be able to predict pollution levels based on an image alone.

AirTick creator Pan Zhengziang said in a promotional video last month that the growing concern among the public over air quality can make programs like this a success—especially in Southeast Asia, where smog has gotten so bad that governments have had to shut down schools and suspend outdoor activities.  “In Singapore’s recent haze episode, around 250,000 people [have] shared their concerns via Twitter,” he said. “This has made crowdsourcing-based air quality monitoring a possibility.”…(More)”

How Citizen Science Changed the Way Fukushima Radiation is Reported


Ari Beser at National Geographic: “It appears the world-changing event didn’t change anything, and it’s disappointing,”said Pieter Franken, a researcher at Keio University in Japan (Wide Project), the MIT Media Lab (Civic Media Centre), and co-founder of Safecast, a citizen-science network dedicated to the measurement and distribution of accurate levels of radiation around the world, especially in Fukushima. “There was a chance after the disaster for humanity to innovate our thinking about energy, and that doesn’t seem like it’s happened.  But what we can change is the way we measure the environment around us.”

Franken and his founding partners found a way to turn their email chain, spurred by the tsunami, into Safecast; an open-source network that allows everyday people to contribute to radiation-monitoring.

“We literally started the day after the earthquake happened,” revealed Pieter. “A friend of mine, Joi Ito, the director of MIT Media Lab, and I were basically talking about what Geiger counter to get. He was in Boston at the time and I was here in Tokyo, and like the rest of the world, we were worried, but we couldn’t get our hands on anything. There’s something happening here, we thought. Very quickly as the disaster developed, we wondered how to get the information out. People were looking for information, so we saw that there was a need. Our plan became: get information, put it together and disseminate it.”

An e-mail thread between Franken, Ito, and Sean Bonner, (co-founder of CRASH Space, a group that bills itself as Los Angeles’ first hackerspace), evolved into a network of minds, including members of Tokyo Hackerspace, Dan Sythe, who produced high-quality Geiger counters, and Ray Ozzie, Microsoft’s former Chief Technical Officer. On April 15, the group that was to become Safecast sat down together for the first time. Ozzie conceived the plan to strap a Geiger counter to a car and somehow log measurements in motion. This would became the bGeigie, Safecast’s future model of the do-it-yourself Geiger counter kit.

Armed with a few Geiger counters donated by Sythe, the newly formed team retrofitted their radiation-measuring devices to the outside of a car.  Safecast’s first volunteers drove up to the city of Koriyama in Fukushima Prefecture, and took their own readings around all of the schools. Franken explained, “If we measured all of the schools, we covered all the communities; because communities surround schools. It was very granular, the readings changed a lot, and the levels were far from academic, but it was our start. This was April 24, 6 weeks after the disaster. Our thinking changed quite a bit through this process.”

DSC_0358
With the DIY kit available online, all anyone needs to make their own Geiger counter is a soldering iron and the suggested directions.

Since their first tour of Koriyama, with the help of a successful Kickstarter campaign, Safecast’s team of volunteers have developed the bGeigie handheld radiation monitor, that anyone can buy on Amazon.com and construct with suggested instructions available online. So far over 350 users have contributed 41 million readings, using around a thousand fixed, mobile, and crowd-sourced devices….(More)

Open data and (15 million!) new measures of democracy


Joshua Tucker in the Washington Post: “Last month the University of Gothenberg’s V-Dem Institute released a new“Varieties of Democracy” dataset. It provides about 15 million data points on democracy, including 39 democracy-related indices. It can be accessed at v-dem.net along with supporting documentation. I asked Staffan I. Lindberg, Director of the V-Dem Institute and one of the directors of the project, a few questions about the new data. What follows is a lightly edited version of his answers.


Women’s Political Empowerment Index for Southeast Asia (Data: V-Dem data version 5; Figure V-Dem Institute, University of Gothenberg, Sweden)

Joshua Tucker: What is democracy, and is it even really to have quantitative measures on democracy?

Staffan Lindberg: There is no consensus on the definition of democracy and how to measure it. The understanding of what a democracy really is varies across countries and regions. This motivates the V-Dem approach not to offer one standard definition of the concept but instead to distinguish among five principles different versions of democracy: Electoral, Liberal, Participatory, Deliberative, and Egalitarian democracy. All of these principles have played prominent roles in current and historical discussions about democracy. Our measurement of these principles are based on two types of data, factual data collected by assisting researchers and survey responses by country experts, which are combined using a rather complex measurement model (which is a“custom-designed Bayesian ordinal item response theory model”, for details see the V-Dem Methodology document)….(More)

Big data’s big role in humanitarian aid


Mary K. Pratt at Computerworld: “Hundreds of thousands of refugees streamed into Europe in 2015 from Syria and other Middle Eastern countries. Some estimates put the number at nearly a million.

The sheer volume of people overwhelmed European officials, who not only had to handle the volatile politics stemming from the crisis, but also had to find food, shelter and other necessities for the migrants.

Sweden, like many of its European Union counterparts, was taking in refugees. The Swedish Migration Board, which usually sees 2,500 asylum seekers in an average month, was accepting 10,000 per week.

“As you can imagine, with that number, it requires a lot of buses, food, registration capabilities to start processing all the cases and to accommodate all of those people,” says Andres Delgado, head of operational control, coordination and analysis at the Swedish Migration Board.

Despite the dramatic spike in refugees coming into the country, the migration agency managed the intake — hiring extra staff, starting the process of procuring housing early, getting supplies ready. Delgado credits a good part of that success to his agency’s use of big data and analytics that let him predict, with a high degree of accuracy, what was heading his way.

“Without having that capability, or looking at the tool every day, to assess every need, this would have crushed us. We wouldn’t have survived this,” Delgado says. “It would have been chaos, actually — nothing short of that.”

The Swedish Migration Board has been using big data and analytics for several years, as it seeks to gain visibility into immigration trends and what those trends will mean for the country…./…

“Can big data give us peace? I think the short answer is we’re starting to explore that. We’re at the very early stages, where there are shining examples of little things here and there. But we’re on that road,” says Kalev H. Leetaru, creator of the GDELT Project, or the Global Database of Events, Language and Tone, which describes itself as a comprehensive “database of human society.”

The topic is gaining traction. A 2013 report, “New Technology and the Prevention of Violence and Conflict,” from the International Peace Institute, highlights uses of telecommunications technology, including data, in several crisis situations around the world. The report emphasizes the potential these technologies hold in helping to ease tensions and address problems.

The report’s conclusion offers this idea: “Big data can be used to identify patterns and signatures associated with conflict — and those associated with peace — presenting huge opportunities for better-informed efforts to prevent violence and conflict.”

That’s welcome news to Noel Dickover. He’s the director of PeaceTech Data Networks at the PeaceTech Lab, which was created by the U.S. Institute of Peace (USIP) to advance USIP’s work on how technology, media and data help reduce violent conflict around the world.

Such work is still in the nascent stages, Dickover says, but people are excited about its potential. “We have unprecedented amounts of data on human sentiment, and we know there’s value there,” he says. “The question is how to connect it.”

Dickover is working on ways to do just that. One example is the Open Situation Room Exchange (OSRx) project, which aims to “empower greater collective impact in preventing or mitigating serious violent conflicts in particular arenas through collaboration and data-sharing.”…(More)

Five ways tech is crowdsourcing women’s empowerment


Zara Rahman in The Guardian: “Around the world, women’s rights advocates are crowdsourcing their own data rather than relying on institutional datasets.

Citizen-generated data is especially important for women’s rights issues. In many countries the lack of women in positions of institutional power, combined with slow, bureaucratic systems and a lack of prioritisation of women’s rights issues means data isn’t gathered on relevant topics, let alone appropriately responded to by the state.

Even when data is gathered by institutions, societal pressures may mean it remains inadequate. In the case of gender-based violence, for instance, women often suffer in silence, worrying nobody will believe them or that they will be blamed. Providing a way for women to contribute data anonymously or, if they so choose, with their own details, can be key to documenting violence and understanding the scale of a problem, and thus deciding upon appropriate responses.

Crowdsourcing data on street harassment in Egypt

Using open source platform Ushahidi, HarassMap provides women with a way to document incidences of street harassment. The project, which began in 2010, is raising awareness of how common street harassment is, giving women’s rights advocates a concrete way to highlight the scale of the problem….

Documenting experiences of reporting sexual harassment and violence to the police in India

Last year, The Ladies Finger, a women’s zine based in India, partnered with Amnesty International to support its Ready to Report campaign, which aimed to make it easier for survivors of sexual violence to file a police complaint. Using social media and through word of mouth, it asked the community if they had experiences to share about reporting sexual assault and harassment to the police. Using these crowdsourced leads, The Ladies Finger’s reporters spoke to people willing to share their experiences and put together a series of detailed contextualised stories. They included a piece that evoked a national outcry and spurred the Uttar Pradesh government to make an arrest for stalking, after six months of inaction….

Reporting sexual violence in Syria

Women Under Siege is a global project by Women’s Media Centre that is investigating how rape and sexual violence is used in conflicts. Its Syria project crowdsources data on sexual violence in the war-torn country. Like HarassMap, it uses the Ushahidi platform to geolocate where acts of sexual violence take place. Where possible, initial reports are contextualised with deeper media reports around the case in question….

Finding respectful gynaecologists in India

After recognising that many women in her personal networks were having bad experiences with gynaecologists in India, Delhi-based Amba Azaad began – with the help of her friends – putting together a list of gynaecologists who had treated patients respectfully called Gynaecologists We Trust. As the site says, “Finding doctors who are on our side is hard enough, and when it comes to something as intimate as our internal plumbing, it’s even more difficult.”…

Ending tech-related violence against women

In 2011, Take Back the Tech, an initiative from the Association for Progressive Communications, started a map gathering incidences of tech-related violence against women. Campaign coordinator Sara Baker says crowdsourcing data on this topic is particularly useful as “victims/survivors are often forced to tell their stories repeatedly in an attempt to access justice with little to no action taken on the part of authorities or intermediaries”. Rather than telling that story multiple times and seeing it go nowhere, their initiative gives people “the opportunity to make their experience visible (even if anonymously) and makes them feel like someone is listening and taking action”….(More)

Iranian youth get app to dodge morality police


BBC Trending: “An anonymous team of Iranian app developers have come up with a solution to help young fashion conscious Iranians avoid the country’s notorious morality police known in Persian as “Ershad” or guidance.

Ershad’s mobile checkpoints which usually consist of a van, a few bearded menand one or two women in black chadors, are deployed in towns across Iran andappear with no notice.

Ershad personnel have a very extensive list of powers ranging from issuing warnings and forcing those they accuse of violating Iran’s Islamic code of conduct, to make a written statement pledging to never do so again, to fines or even prosecuting offenders.

The new phone app which is called “Gershad” (probably meaning get aroundErshad instead of facing them) however, will alert users to checkpoints and helpthem to avoid them by choosing a different route.

The data for the app is crowdsourced. It relies on users to point out the location of the Ershad vans on maps and when a sufficient number of users point out the same point, an alert will show up on the map for other users. When the number decreases, the alert will fade gradually from the map.

Screengrab of Tehran on Gershad

In a statement on their web page the app’s developers explain their motives in thisway: “Why do we have to be humiliated for our most obvious right which is the rightto wear what we want? Social media networks and websites are full of footage and photos of innocent women who have been beaten up and dragged on the ground by the Ershad patrol agents.”…

According to the designers of Gershad, in 2014 alone, around three million people were issued with official warnings, 18,000 were prosecuted and more than 200,000 were made to write formal pledges of repentance….

If the app, lives up to the claims made for it, Gershad will be a lifesaver for the growing numbers of young Iranians who are pushing the boundaries of what is allowed and finding themselves on the wrong side of what an Ershad agent sees as acceptable….(More)”

Dive Against Debris: Employing 25,600 scuba divers to collect data


DataDrivenJournalism: “In 2011, the team at Project AWARE launched the Dive Against Debris program with the objective of better documenting the amount of marine debris found in the world’s oceans. This global citizen science program trains volunteer scuba divers from across the globe to conduct underwater surveys, generating quantitative data on the debris they see. After cleaning this data for quality assurance, it is then published on their interactive Dive Against Debris Map. This data and visualization informs the team’s advocacy work, ultimately seeking to generate changes in policy.

The impact of marine debris is devastating, killing marine life and changing their habitats and ecosystems. Animals are extremely vulnerable to ingestion or entanglement which leads to death, as they are unable to distinguish between what is trash and what is not.

Beyond this, as microscopic pieces of plastic enter the food chain, most seafood ingested by humans also likely contains marine debris.

Project AWARE is a growing movement of scuba divers protecting the ocean, with a long history of working on the marine debris issue. Through its work, the Project AWARE team found that there was a significant lack of data available regarding underwater marine debris.

To remedy this, the Dive Against Debris program was launched in 2011. The programs seeks to collect and visualise data generated by their volunteers, then use this data to influence policy changes and raise social awareness around the world. This data collection is unique in that it focuses exclusively on yielding data about the types and quantities of marine debris items found beneath in the ocean, an issue Hannah Pragnell-Raasch, a Program Specialist with Project AWARE, told us “has previously been disregarded as out of sight, out of mind, as the everyday person is not exposed to the harmful impacts.”

To date, Dive Against Debris surveys have been conducted in over 50 countries, with the top reporting countries being the United States, Thailand and Greece. As more divers get involved with Dive Against Debris, Project AWARE continues to bring visibility to the problem of marine debris and helps to identify target areas for waste prevention efforts.

datadebrismap.png

 

….

Anyone can take part in a Dive Against Debris survey, as long as they are a certified diver. As described in their “Action Zone”, scuba divers can either “join” or “create” an action. To further support the program, Project AWARE launched the Dive Against Debris Distinctive Specialty, a course of divers, which “aims to equip students (scuba divers) with the skills and knowledge necessary to conduct their own Dive Against Debris Surveys.”

Before the data appears on the interactive Dive Against Debris Map, it goes through a quality review in order to ensure data integrity. The survey leader at Project AWARE corrects any data inconsistencies. Then, as the focus is exclusively on what is found underwater, all land data is removed. Project AWARE Aware aims to create “an accurate perspective about underwater marine debris, that policy-makers simply cannot ignore”…. Explore the Dive Against Debris project here…. (More)