Growing Data Collection Inspires Openness at NGA

Curated on Posted on by Stefaan Verhulst

at Secrecy News: “A flood of information from the ongoing proliferation of space-based sensors and ground-based data collection devices is promoting a new era of transparency in at least one corner of the U.S. intelligence community.

The “explosion” of geospatial information “makes geospatial intelligence increasingly transparent because of the huge number and diversity of commercial and open sources of information,” said Robert Cardillo, director of the National Geospatial-Intelligence Agency (NGA), in a speech last month.

Hundreds of small satellites are expected to be launched within the next three years — what Mr. Cardillo called a “darkening of the skies” — and they will provide continuous, commercially available coverage of the entire Earth’s surface.

“The challenges of taking advantage of all of that data are daunting for all of us,” Mr. Cardillo said.

Meanwhile, the emerging “Internet of Things” is “spreading rapidly as more people carry more handheld devices to more places” generating an abundance of geolocation data.

This is, of course, a matter of intelligence interest since “Every local, regional, and global challenge — violent extremism in the Middle East and Africa, Russian aggression, the rise of China, Iranian and North Korean nuclear weapons, cyber security, energy resources, and many more — has geolocation at its heart.”

Consequently, “We must open up GEOINT far more toward the unclassified world,” Director Cardillo said in another speech last week.

“In the past, we have excelled in our closed system. We enjoyed a monopoly on sources and methods. That monopoly has long since ended. Today and in the future, we must thrive and excel in the open.”

So far, NGA has already distinguished itself in the area of disaster relief, Mr. Cardillo said.

“Consider Team NGA’s response to the Ebola crisis. We are the first intelligence agency to create a World Wide Web site with access to our relevant unclassified content. It is open to everyone — no passwords, no closed groups.”

NGA provided “more than a terabyte of up-to-date commercial imagery.”

“You can imagine how important it is for the Liberian government to have accurate maps of the areas hardest hit by the Ebola epidemic as well as the medical and transportation infrastructure to combat the disease,” Mr. Cardillo said.

But there are caveats. Just because information is unclassified does not mean that it is freely available.

“Although 99 percent of all of our Ebola data is unclassified, most of that is restricted by our agreements [with commercial providers],” Mr. Cardillo said. “We are negotiating with many sources to release more data.”

Last week, Director Cardillo announced a new project called GEOINT Pathfinder that will attempt “to answer key intelligence questions using only unclassified data.”….(More)

New Desktop Application Has Potential to Increase Asteroid Detection, Now Available to Public

Curated on Posted on by Stefaan Verhulst

NASA Press Release: “A software application based on an algorithm created by a NASA challenge has the potential to increase the number of new asteroid discoveries by amateur astronomers.

Analysis of images taken of our solar system’s main belt asteroids between Mars and Jupiter using the algorithm showed a 15 percent increase in positive identification of new asteroids.

During a panel Sunday at the South by Southwest Festival in Austin, Texas, NASA representatives discussed how citizen scientists have made a difference in asteroid hunting. They also announced the release of a desktop software application developed by NASA in partnership with Planetary Resources, Inc., of Redmond, Washington. The application is based on an Asteroid Data Hunter-derived algorithm that analyzes images for potential asteroids. It’s a tool that can be used by amateur astronomers and citizen scientists.

The Asteroid Data Hunter challenge was part of NASA’s Asteroid Grand Challenge. The data hunter contest series, which was conducted in partnership with Planetary Resources under a Space Act Agreement, was announced at the 2014 South by Southwest Festival and concluded in December. The series offered a total of $55,000 in awards for participants to develop significantly improved algorithms to identify asteroids in images captured by ground-based telescopes. The winning solutions of each piece of the contest combined to create an application using the best algorithm that increased the detection sensitivity, minimized the number of false positives, ignored imperfections in the data, and ran effectively on all computer systems.

“The Asteroid Grand Challenge is seeking non-traditional partnerships to bring the citizen science and space enthusiast community into NASA’s work,” said Jason Kessler, program executive for NASA’s Asteroid Grand Challenge. “The Asteroid Data Hunter challenge has been successful beyond our hopes, creating something that makes a tangible difference to asteroid hunting astronomers and highlights the possibility for more people to play a role in protecting our planet.”…

The new asteroid hunting application can be downloaded at:

http://topcoder.com/asteroids

For information about NASA’s Asteroid Grand Challenge, visit:

http://www.nasa.gov/asteroidinitiative

Why governments need guinea pigs for policies

Curated on Posted on by Stefaan Verhulst

Jonathan Breckon in the Guardian:”People are unlikely to react positively to the idea of using citizens as guinea pigs; many will be downright disgusted. But there are times when government must experiment on us in the search for knowledge and better policy….

Though history calls into question the ethics of experimentation, unless we try things out, we will never learn. The National Audit Office says that £66bn worth of government projects have no plans to evaluate their impact. It is unethical to roll out policies in this arbitrary way. We have to experiment on a small scale to have a better understanding of how things work before rolling out policies across the UK. This is just as relevant to social policy, as it is to science and medicine, as set out in a new report by the Alliance for Useful Evidence.

Whether it’s the best ways to teach our kids to read, designing programmes to get unemployed people back to work, or encouraging organ donation – if the old ways don’t work, we have to test new ones. And that testing can’t always be done by a committee in Whitehall or in a university lab.

Experimentation can’t happen in isolation. What works in Lewisham or Londonnery, might not work in Lincoln – or indeed across the UK. For instance, there is a huge amount debate around the current practice of teaching children to read and spell using phonics, which was based on a small-scale study in Clackmannanshire, as well as evidence from the US. A government-commissioned review on the evidence for phonics led professor Carole Torgerson, then at York University, to warn against making national policy off the back of just one small Scottish trial.

One way round this problem is to do larger experiments. The increasing use of the internet in public services allows for more and faster experimentation, on a larger scale for lower cost – the randomised controlled trial on voter mobilisation that went to 61 million users in the 2010 US midterm elections, for example. However, the use of the internet doesn’t get us off the ethical hook. Facebook had to apologise after a global backlash to secret psychological tests on their 689,000 users.

Contentious experiments should be approved by ethics committees – normal practice for trials in hospitals and universities.

We are also not interested in freewheeling trial-and-error; robust and appropriate research techniques to learn from experiments are vital. It’s best to see experimentation as a continuum, ranging from the messiness of attempts to try something new to experiments using the best available social science, such as randomised controlled trials.

Experimental government means avoiding an approach where everything is fixed from the outset. What we need is “a spirit of experimentation, unburdened by promises of success”, as recommended by the late professor Roger Jowell, author of the 2003 Cabinet Office report, Trying it out [pdf]….(More)”

Big Data for Social Good

Curated on Posted on by Stefaan Verhulst

Introduction to a Special Issue of the Journal “Big Data” by Catlett Charlie and Ghani Rayid: “…organizations focused on social good are realizing the potential as well but face several challenges as they seek to become more data-driven. The biggest challenge they face is a paucity of examples and case studies on how data can be used for social good. This special issue of Big Data is targeted at tackling that challenge and focuses on highlighting some exciting and impactful examples of work that uses data for social good. The special issue is just one example of the recent surge in such efforts by the data science community. …

This special issue solicited case studies and problem statements that would either highlight (1) the use of data to solve a social problem or (2) social challenges that need data-driven solutions. From roughly 20 submissions, we selected 5 articles that exemplify this type of work. These cover five broad application areas: international development, healthcare, democracy and government, human rights, and crime prevention.

“Understanding Democracy and Development Traps Using a Data-Driven Approach” (Ranganathan et al.) details a data-driven model between democracy, cultural values, and socioeconomic indicators to identify a model of two types of “traps” that hinder the development of democracy. They use historical data to detect causal factors and make predictions about the time expected for a given country to overcome these traps.

“Targeting Villages for Rural Development Using Satellite Image Analysis” (Varshney et al.) discusses two case studies that use data and machine learning techniques for international economic development—solar-powered microgrids in rural India and targeting financial aid to villages in sub-Saharan Africa. In the process, the authors stress the importance of understanding the characteristics and provenance of the data and the criticality of incorporating local “on the ground” expertise.

In “Human Rights Event Detection from Heterogeneous Social Media Graphs,” Chen and Neil describe efficient and scalable techniques to use social media in order to detect emerging patterns in human rights events. They test their approach on recent events in Mexico and show that they can accurately detect relevant human rights–related tweets prior to international news sources, and in some cases, prior to local news reports, which could potentially lead to more timely, targeted, and effective advocacy by relevant human rights groups.

“Finding Patterns with a Rotten Core: Data Mining for Crime Series with Core Sets” (Wang et al.) describes a case study with the Cambridge Police Department, using a subspace clustering method to analyze the department’s full housebreak database, which contains detailed information from thousands of crimes from over a decade. They find that the method allows human crime analysts to handle vast amounts of data and provides new insights into true patterns of crime committed in Cambridge…..(More)

How to Fight the Next Epidemic

Curated on Posted on by Stefaan Verhulst

Bill Gates in the New York Times: “The Ebola Crisis Was Terrible. But Next Time Could Be Much Worse….Much of the public discussion about the world’s response to Ebola has focused on whether the World Health Organization, the Centers for Disease Control and Prevention and other groups could have responded more effectively. These are worthwhile questions, but they miss the larger point. The problem isn’t so much that the system didn’t work well enough. The problem is that we hardly have a system at all.

To begin with, most poor countries, where a natural epidemic is most likely to start, have no systematic disease surveillance in place. Even once the Ebola crisis was recognized last year, there were no resources to effectively map where cases occurred, or to use people’s travel patterns to predict where the disease might go next….

Data is another crucial problem. During the Ebola epidemic, the database that tracks cases has not always been accurate. This is partly because the situation is so chaotic, but also because much of the case reporting has been done on paper and then sent to a central location for data entry….

I believe that we can solve this problem, just as we’ve solved many others — with ingenuity and innovation.

We need a global warning and response system for outbreaks. It would start with strengthening poor countries’ health systems. For example, when you build a clinic to deliver primary health care, you’re also creating part of the infrastructure for fighting epidemics. Trained health care workers not only deliver vaccines; they can also monitor disease patterns, serving as part of the early warning systems that will alert the world to potential outbreaks. Some of the personnel who were in Nigeria to fight polio were redeployed to work on Ebola — and that country was able to contain the disease very quickly.

We also need to invest in disease surveillance. We need a case database that is instantly accessible to the relevant organizations, with rules requiring countries to share their information. We need lists of trained personnel, from local leaders to global experts, prepared to deal with an epidemic immediately. … (More)”

Crowdsourcing as a tool for political participation? – the case of Ugandawatch

Curated on Posted on by Stefaan Verhulst

Paper by Johan Hellström in the International Journal of Public Information Systems: “Uganda has democratic deficits where demand for democracy exceeds its supply. As a consequence it is argued that a segment of Ugandans might participate and honour the freedom to speak out, assemble, and associate given new opportunities outside the traditional political channels. With expanded mobile coverage and access to mobile devices and services in mind, and using the concept of open crowdsourcing, the platform UgandaWatch was launched prior to the 2011 general elections with the intention to meet the demand, to offer increased equality of political participation, and to advance efforts toward increased citizen engagement in Uganda. From a community informatics point of view, the study examined how and under what conditions access to ICT tools (mobile devices, networks, and a crowdsourcing platform) can be made usable and useful for people and communities for increased political participation in a given context. By combining the collection and analysis of quantitative (SMS-survey) and qualitative data (focus groups) through a mixed-method approach, this study answers the questions, What are the key factors that influence users’ willingness to use mobile phones and crowdsourcing platforms as a channel for political participation?, and What concerns do users have with respect to using mobile phones and crowdsourcing platforms in the participation process? The study shows that users participated because they hoped it would bring real change to Uganda’s electoral and political landscape, that it was a convenient channel to use (quick and easy) and that confidentiality was assured. The user concerns relate to costs, trust, and safety. Crowdsourcing offers an alternative channel and may substitute or supplement traditional means of political participation. It can increase participation in some groups, including among those who normally do not participate—something that increases equality of political participation in a positive direction….(More)”

Wittgenstein, #TheDress and Google’s search for a bigger truth

Curated on Posted on by Stefaan Verhulst

Robert Shrimsley at the Financial Times: “As the world burnt with a BuzzFeed-prompted debate over whether a dress was black and blue or white and gold, the BBC published a short article posing the question everyone was surely asking: “What would Wittgenstein say about that dress?”

Wittgenstein died in 1951, so we cannot know if the philosopher of language, truth and context would have been a devotee of BuzzFeed. (I guess it depends on whether we are talking of the early or the late Ludwig. The early Wittgenstein, it is well known, was something of an enthusiast for LOLs, whereas the later was more into WTFs and OMGs.)

The dress will now join the pantheon of web phenomena such as “Diet Coke and Mentos” and “Charlie bit my finger”. But this trivial debate on perceived truth captured in miniature a wider issue for the web: how to distil fact from noise when opinion drowns out information and value is determined by popularity.

At about the same time as the dress was turning the air blue — or was it white? — the New Scientist published a report on how one web giant might tackle this problem, a development in which Wittgenstein might have been very interested. The magazine reported on a Google research paper about how the company might reorder its search rankings to promote sites that could be trusted to tell the truth. (Google produces many such papers a year so this is a long way short of official policy.) It posits a formula for finding and promoting sites with a record of reliability.

This raises an interesting question over how troubled we should be by the notion that a private company with its own commercial interests and a huge concentration of power could be the arbiter of truth. There is no current reason to see sinister motives in Google’s search for a better web: it is both honourable and good business. But one might ask how, for example, Google Truth might determine established truths on net neutrality….

The paper suggests using fidelity to proved facts as a proxy for trust. This is easiest with single facts, such as a date or place of birth. For example, it suggests claiming Barack Obama was born in Kenya would push a site down the rankings. This would be good for politics but facts are not always neutral. Google would risk being depicted as part of “the mainstream media”. Fox Search here we come….(More)”

New million dollar fund for participatory budgeting in South Australia

Curated on Posted on by Stefaan Verhulst

Medha Basu at Future Gov: “A new programme in South Australia is allowing citizens to determine which community projects should get funding.

The Fund My Community programme has a pool of AU$1 million (US$782,130) to fund projects by non-profit organisations aimed at supporting disadvantaged South Australians.

Organisations can nominate their projects for funding from this pool and anyone in the state can vote for the projects on the YourSAy web site.

All information about the projects submitted by the organisations will be available online to make the process transparent. “We hope that by providing the community with the right information about grant applications, people will support projects that will have the biggest impact in addressing disadvantage across South Australia,” the Fund My Community web site says.

The window to nominate community projects for funding is open until 2 April. Eligible applications will be opened for community assessment from 23 April to 4 May. The outcome will be announced and grants will be given out in June. See the full timeline here:

Fund my Community South Australia

There is a catch here though. The projects that receive the most support from the community are suggested for funding, but due to “a legal requirement” the final decision and grant approval comes from the Board of the Charitable and Social Welfare Fund, according to the YourSAy web site….(More)”

Data-Driven Development Pathways for Progress

Curated on Posted on by Stefaan Verhulst
Report from the World Economic Forum: “Data is the lifeblood of sustainable development and holds tremendous potential for transformative positive change particularly for lower- and middle-income countries. Yet despite the promise of a “Data Revolution”, progress is not a certainty. Lack of clarity on privacy and ethical issues, asymmetric power dynamics and an array of entangled societal and commercial risks threaten to hinder progress.
Written by the World Economic Forum Global Agenda Council on Data-Driven Development, this report serves to clarify how big data can be leveraged to address the challenges of sustainable development. Providing a blueprint for balancing competing tensions, areas of focus include: addressing the data deficit of the Global South, establishing resilient governance and strengthening capacities at the community and individual level. (PDF)”

Amid Open Data Push, Agencies Feel Urge for Analytics

Curated on Posted on by Stefaan Verhulst

Jack Moore at NextGov: “Federal agencies, thanks to their unique missions, have long been collectors of valuable, vital and, no doubt, arcane data. Under a nearly two-year-old executive order from President Barack Obama, agencies are releasing more of this data in machine-readable formats to the public and entrepreneurs than ever before.
But agencies still need a little help parsing through this data for their own purposes. They are turning to industry, academia and outside researchers for cutting-edge analytics tools to parse through their data to derive insights and to use those insights to drive decision-making.
Take the U.S. Agency for International Development, for example. The agency administers U.S. foreign aid programs aimed at ending extreme poverty and helping support democratic societies around the globe.
Under the agency’s own recent open data policy, it’s started collecting reams of data from its overseas missions. Starting Oct. 1, organizations doing development work on the ground – including through grants and contracts – have been directed to also collect data generated by their work and submit it to back to agency headquarters. Teams go through the data, scrub it to remove sensitive material and then publish it.
The data spans the gamut from information on land ownership in South Sudan to livestock demographics in Senegal and HIV prevention activities in Zambia….The agency took the first step in solving that problem with a Jan. 20 request for information from outside groups for cutting-edge data analytics tools.
“Operating units within USAID are sometimes constrained by existing capacity to transform data into insights that could inform development programming,” the RFI stated.
The RFI queries industry on their capabilities in data mining and social media analytics and forecasting and systems modeling.
USAID is far from alone in its quest for data-driven decision-making.
A Jan. 26 RFI from the Transportation Department’s Federal Highway Administration also seeks innovative ideas from industry for “advanced analytical capabilities.”…(More)”