Northern America

Open Data Supply: Enriching the usability of information

Curated on April 27, 2016August 15, 2018 by Stefaan Verhulst

Report by Phoensight: “With the emergence of increasing computational power, high cloud storage capacity and big data comes an eager anticipation of one of the biggest IT transformations of our society today.

Open data has an instrumental role to play in our digital revolution by creating unprecedented opportunities for governments and businesses to leverage off previously unavailable information to strengthen their analytics and decision making for new client experiences. Whilst virtually every business recognises the value of data and the importance of the analytics built on it, the ability to realise the potential for maximising revenue and cost savings is not straightforward. The discovery of valuable insights often involves the acquisition of new data and an understanding of it. As we move towards an increasing supply of open data, technological and other entrepreneurs will look to better utilise government information for improved productivity.

This report uses a data-centric approach to examine the usability of information by considering ways in which open data could better facilitate data-driven innovations and further boost our economy. It assesses the state of open data today and suggests ways in which data providers could supply open data to optimise its use. A number of useful measures of information usability such as accessibility, quantity, quality and openness are presented which together contribute to the Open Data Usability Index (ODUI). For the first time, a comprehensive assessment of open data usability has been developed and is expected to be a critical step in taking the open data agenda to the next level.

With over two million government datasets assessed against the open data usability framework and models developed to link entire country’s datasets to key industry sectors, never before has such an extensive analysis been undertaken. Government open data across Australia, Canada, Singapore, the United Kingdom and the United States reveal that most countries have the capacity for improvements in their information usability. It was found that for 2015 the United Kingdom led the way followed by Canada, Singapore, the United States and Australia. The global potential of government open data is expected to reach 20 exabytes by 2020, provided governments are able to release as much data as possible within legislative constraints….(More)”

7 projects that state and local governments can reuse

Curated on April 24, 2016May 29, 2019 by Stefaan Verhulst

Melody Kramer at 18F: “We’re starting to see state and local governments adapt or use 18F products or tools. Nothing could make us happier; all of our code (and content) is available for anyone to use and reusable.

There are a number of open source projects that 18F has worked on that could work particularly well at any level of government. We’re highlighting seven below:

Public website analytics

We worked with the Digital Analytics Program, the U.S. Digital Service (USDS), and the White House to build and host a dashboard showing real-time U.S. federal government web traffic. This helps staff and the public learn about how people use government websites. The dashboard itself is open source and can be adapted for a state or local government. We recently interviewed folks from Philadelphia, Boulder, and the state of Tennessee about how they’ve adapted the analytics dashboard for their own use.

Quick mini-sites for content

We built a responsive, accessible website template (based on open source work by the Consumer Financial Protection Bureau) that we use primarily for documentation and guides. You can take the website template, adapt the colors and fonts to reflect your own style template, and have an easy way to release notes about a project. We’ve used this template to write a guide on accessibility in government, content guidelines, and a checklist for what needs to take place before we release software. You’re also welcome to take our content and adapt it for your own needs — what we write is in the public domain.

Insight into how people interact with government

Over the last several months, staff from General Service Administration’s USAGov and 18F teams have been talking to Americans around the country about their interactions with the federal government. The goal of the research was to identify and create cross-agency services and resources to improve how the government interacts with the public. Earlier this month, we published all of our research. You can read the full report with findings or explore what we learned on the 18F blog.

Market research for procurement

We developed a tool that helps you easily conduct market research across a number of categories for acquiring professional labor. You can read about how the city of Boston is using the tool to conduct market research.

Vocabulary for user-centered design

We released a deck of method cards that help research and design teams communicate a shared vocabulary across teams and agencies.

Task management

We recently developed a checklist program that help users manage complex to-do lists. One feature: checklist items deadlines can be set according to a fixed date or relative to completion of other items. This means you can create checklist for all new employees, for example, and say “Task five should be completed four days after task four,” whenever task four is completed by an employee.

Help small businesses find opportunities

FBOpen is a set of open source tools to help small businesses search for opportunities to work with the U.S. government. FBOpen presents an Application Programming Interface (API) to published Federal contracting opportunities, as well as implementing a beautiful graphical user interface to the same opportunities.

Anyone who wishes to may reuse this code to create their own website, free of charge and unencumbered by obligations….(More)”

The Open Data Barometer (3rd edition)

Curated on April 22, 2016August 3, 2018 by Stefaan Verhulst

The Open Data Barometer: “Once the preserve of academics and statisticians, data has become a development cause embraced by everyone from grassroots activists to the UN Secretary-General. There’s now a clear understanding that we need robust data to drive democracy and development — and a lot of it.

Last year, the world agreed the Sustainable Development Goals (SDGs) — seventeen global commitments that set an ambitious agenda to end poverty, fight inequality and tackle climate change by 2030. Recognising that good data is essential to the success of the SDGs, the Global Partnership for Sustainable Development Data and the International Open Data Charter were launched as the SDGs were unveiled. These alliances mean the “data revolution” now has over 100 champions willing to fight for it. Meanwhile, Africa adopted the African Data Consensus — a roadmap to improving data standards and availability in a region that has notoriously struggled to capture even basic information such as birth registration.

But while much has been made of the need for bigger and better data to power the SDGs, this year’s Barometer follows the lead set by the International Open Data Charter by focusing on how much of this data will be openly available to the public.

Open data is essential to building accountable and effective institutions, and to ensuring public access to information — both goals of SDG 16. It is also essential for meaningful monitoring of progress on all 169 SDG targets. Yet the promise and possibilities offered by opening up data to journalists, human rights defenders, parliamentarians, and citizens at large go far beyond even these….

At a glance, here are this year’s key findings on the state of open data around the world:

Open data is entering the mainstream.The majority of the countries in the survey (55%) now have an open data initiative in place and a national data catalogue providing access to datasets available for re-use. Moreover, new open data initiatives are getting underway or are promised for the near future in a number of countries, including Ecuador, Jamaica, St. Lucia, Nepal, Thailand, Botswana, Ethiopia, Nigeria, Rwanda and Uganda. Demand is high: civil society and the tech community are using government data in 93% of countries surveyed, even in countries where that data is not yet fully open.
Despite this, there’s been little to no progress on the number of truly open datasets around the world.Even with the rapid spread of open government data plans and policies, too much critical data remains locked in government filing cabinets. For example, only two countries publish acceptable detailed open public spending data. Of all 1,380 government datasets surveyed, almost 90% are still closed — roughly the same as in the last edition of the Open Data Barometer (when only 130 out of 1,290 datasets, or 10%, were open). What is more, much of the approximately 10% of data that meets the open definition is of poor quality, making it difficult for potential data users to access, process and work with it effectively.
“Open-washing” is jeopardising progress. Many governments have advertised their open data policies as a way to burnish their democratic and transparent credentials. But open data, while extremely important, is just one component of a responsive and accountable government. Open data initiatives cannot be effective if not supported by a culture of openness where citizens are encouraged to ask questions and engage, and supported by a legal framework. Disturbingly, in this edition we saw a backslide on freedom of information, transparency, accountability, and privacy indicators in some countries. Until all these factors are in place, open data cannot be a true SDG accelerator.
Implementation and resourcing are the weakest links.Progress on the Barometer’s implementation and impact indicators has stalled or even gone into reverse in some cases. Open data can result in net savings for the public purse, but getting individual ministries to allocate the budget and staff needed to publish their data is often an uphill battle, and investment in building user capacity (both inside and outside of government) is scarce. Open data is not yet entrenched in law or policy, and the legal frameworks supporting most open data initiatives are weak. This is a symptom of the tendency of governments to view open data as a fad or experiment with little to no long-term strategy behind its implementation. This results in haphazard implementation, weak demand and limited impact.
The gap between data haves and have-nots needs urgent attention.Twenty-six of the top 30 countries in the ranking are high-income countries. Half of open datasets in our study are found in just the top 10 OECD countries, while almost none are in African countries. As the UN pointed out last year, such gaps could create “a whole new inequality frontier” if allowed to persist. Open data champions in several developing countries have launched fledgling initiatives, but too often those good open data intentions are not adequately resourced, resulting in weak momentum and limited success.
Governments at the top of the Barometer are being challenged by a new generation of open data adopters. Traditional open data stalwarts such as the USA and UK have seen their rate of progress on open data slow, signalling that new political will and momentum may be needed as more difficult elements of open data are tackled. Fortunately, a new generation of open data adopters, including France, Canada, Mexico, Uruguay, South Korea and the Philippines, are starting to challenge the ranking leaders and are adopting a leadership attitude in their respective regions. The International Open Data Charter could be an important vehicle to sustain and increase momentum in challenger countries, while also stimulating renewed energy in traditional open data leaders….(More)”

Foreign Policy has lost its creativity. Design thinking is the answer.

Curated on April 21, 2016August 15, 2018 by Stefaan Verhulst

Elizabeth Radziszewski at The Wilson Quaterly: “Although the landscape of threats has changed in recent years, U.S. strategies bear striking resemblance to the ways policymakers dealt with crises in the past. Whether it involves diplomatic overtures, sanctions, bombing campaigns, or the use of special ops and covert operations, the range of responses suffers from innovation deficit. Even the use of drones, while a new tool of warfare, is still part of the limited categories of responses that focus mainly on whether or not to kill, cooperate, or do nothing. To meet the evolving nature of threats posed by nonstate actors such as ISIS, the United States needs a strategy makeover — a creative lift, so to speak.

Sanctions, diplomacy, bombing campaigns, special ops, covert operations — the range of our foreign policy responses suffers from an innovation deficit.

Enter the business world. Today’s top companies face an increasingly competitive marketplace where innovative approaches to product and service development are a necessity. Just as the market has changed for companies since the forces of globalization and the digital economy took over, so has the security landscape evolved for the world’s leading hegemon. Yet the responses of top businesses to these changes stand in stark contrast to the United States’ stagnant approaches to current national security threats. Many of today’s thriving businesses have embraced design thinking (DT), an innovative process that identifies consumer needs through immersive ethnographic experiences that are melded with creative brainstorming and quick prototyping.

What would happen if U.S. policymakers took cues from the business world and applied DT in policy development? Could the United States prevent the threats from metastasizing with more proactive rather than reactive strategies — by discovering, for example, how ideas from biology, engineering, and other fields could help analysts inject fresh perspective into tired solutions? Put simply, if U.S. policymakers want to succeed in managing future threats, then they need to start thinking more like business innovators who integrate human needs with technology and economic feasibility.

In his 1969 book The Sciences of the Artificial, Herbert Simon made the first connection between design and a way of thinking. But it was not until the 1980s and 1990s that Stanford scientists began to see the benefits of design practices used by industrial designers as a method for creative thinking. At the core of DT is the idea that solving a challenge requires a deeper understanding of the problem’s true nature and the processes and people involved. This approach contrasts greatly with more standard innovation styles, where a policy solution is developed and then resources are used to fit the solution to the problem. DT reverses the order.

DT encourages divergent thinking, the process of generating many ideas before converging to select the most feasible ones, including making connections between different-yet-related worlds. Finally, the top ideas are quickly prototyped and tested so that early solutions can be modified without investing many resources and risking the biggest obstacle to real innovation: the impulse to try fitting an idea, product, policy to the people, rather of the other way around…

If DT has reenergized the innovative process in the business and nonprofit sector, a systematic application of its methodology could just as well revitalize U.S. national security policies. Innovation in security and foreign policy is often framed around the idea of technological breakthroughs. Thanks toDefense Advanced Research Projects Agency (DARPA), the Department of Defense has been credited with such groundbreaking inventions as GPS, the Internet, and stealth fighters — all of which have created rich opportunities to explore new military strategies. Reflecting this infatuation with technology, but with a new edge, is Defense Secretary Ashton Carter’s unveiling of the Defense Innovation Unit Experimental, an initiative to scout for new technologies, improve outreach to startups, and form deeper relationships between the Pentagon and Silicon Valley. The new DIUE effort signals what businesses have already noticed: the need to be more flexible in establishing linkages with people outside of the government in search for new ideas.

Yet because the primary objective of DIUE remains technological prowess, the effort alone is unlikely to drastically improve the management of national security. Technology is not a substitute for an innovative process. When new invention is prized as the sole focus of innovation, it can, paradoxically, paralyze innovation. Once an invention is adopted, it is all too tempting to mold subsequent policy development around emergent technology, even if other solutions could be more appropriate….(More)”

Six of the Government’s Best Mobile Apps

Curated on April 19, 2016August 15, 2018 by Stefaan Verhulst

USA Gov: “There’s an app for everything in this digital age, including hundredsdeveloped by the federal government. Here are six apps that we foundespecially useful.

Smart Traveler – Planning a trip out of the country this year? SmartTraveler by the State Department is great for all your trips abroad. Getthe latest travel alerts and information on every country, includinghow to find and contact each U.S. Embassy.
FoodKeeper – Ever wonder how long you should cook chicken or howlong food can sit in the fridge before it goes bad? The U.S. Departmentof Agriculture’s FoodKeeper is the tool for you. Not only can you findresources on food safety and post reminders of how long food willremain safe to eat, you can also ask a food safety specialist questions 24/7.
FEMA App – The FEMA app helps you learn how to prepare for and respond to disasters. It includes weather alerts, tipsfor building a basic emergency supply kit, and contact information for applying for assistance and finding local sheltersand disaster recovery centers. Stay safe and know what to do when disasters happen.
IRS2GO – Tax season is here. This IRS app can help you track the status of your refund, make a payment, or find taxpreparation assistance, sometimes for free.
CDC Influenza App-Stay on top of the flu this season and get the latest updates from this official Centers for DiseaseControl and Prevention app. It’s great for health practitioners, teachers, and parents, and includes tips for avoiding the fluand maps of influenza activity.
Dwellr– Have you ever wondered what U.S. city might best suit you? Then the Dwellr app is just for you. When you firstopen the app, you’re guided through an interactive survey, to better understand your ideal places to live based on datagathered by the Census Bureau….(More)”

Data collection is the ultimate public good

Curated on April 7, 2016August 3, 2018 by Stefaan Verhulst

Lawrence H. Summers in the Washington Post: “I spoke at a World Bank conference on price statistics. … I am convinced that data is the ultimate public good and that we will soon have much more data than we do today. I made four primary observations.

First, scientific progress is driven more by new tools and new observations than by hypothesis construction and testing. I cited a number of examples: the observation that Jupiter was orbited by several moons clinched the case against the Ptolemaic system, the belief that all celestial objects circle around the Earth. We learned of cells by seeing them when the microscope was constructed. Accelerators made the basic structure of atoms obvious.

Second, if mathematics is the queen of the hard sciences then statistics is the queen of the social sciences. I gave examples of the power of very simple data analysis. We first learned that exercise is good for health from the observation that, in the 1940s, London bus conductors had much lower death rates than bus drivers. Similarly, data demonstrated that smoking was a major killer decades before the biological processes were understood. At a more trivial level, “Moneyball” shows how data-based statistics can revolutionize a major sport.

Third, I urged that what “you count counts” and argued that we needed much more timely and complete data. I noted the centrality of timely statistics to meaningful progress toward Sustainable Development Goals. In comparison to the nearly six-year lag in poverty statistics, it took the United States only about 3½ years to win World War II.

Fourth, I envisioned what might be possible in a world where there will soon be as many smartphones as adults. With the ubiquitous ability to collect data and nearly unlimited ability to process it will come more capacity to discover previously unknown relationships. We will improve our ability to predict disasters like famines, storms and revolutions. Communication technologies will allow us to better hold policymakers to account with reliable and rapid performance measures. And if history is any guide, we will gain capacities on dimensions we cannot now imagine but will come to regard as indispensable.

This is the work of both governments and the private sector. It is fantasy to suppose data, as the ultimate public good, will come into being without government effort. Equally, we will sell ourselves short if we stick with traditional collection methods and ignore innovative providers and methods such as the use of smartphones, drones, satellites and supercomputers. That is why something like the Billion Prices Project at MIT, which can provide daily price information, is so important. That is why I am excited to be a director and involved with Premise — a data company that analyzes information people collect on their smartphones about everyday life, like the price of local foods — in its capacity to mobilize these technologies as widely as possible. That is why Planet Labs, with its capacity to scan and monitor environmental conditions, represents such a profound innovation….(More)

What Should We Do About Big Data Leaks?

Curated on April 6, 2016October 10, 2018 by Stefaan Verhulst

Paul Ford at the New Republic: “I have a great fondness for government data, and the government has a great fondness for making more of it. Federal elections financial data, for example, with every contribution identified, connected to a name and address. Or the results of the census. I don’t know if you’ve ever had the experience of downloading census data but it’s pretty exciting. You can hold America on your hard drive! Meditate on the miracles of zip codes, the way the country is held together and addressable by arbitrary sets of digits.

You can download whole books, in PDF format, about the foreign policy of the Reagan Administration as it related to Russia. Negotiations over which door the Soviet ambassador would use to enter a building. Gigabytes and gigabytes of pure joy for the ephemeralist. The government is the greatest creator of ephemera ever.

Consider the Financial Crisis Inquiry Commission, or FCIC, created in 2009 to figure out exactly how the global economic pooch was screwed. The FCIC has made so much data, and has done an admirable job (caveats noted below) of arranging it. So much stuff. There are reams of treasure on a single FCIC web site, hosted at Stanford Law School: Hundreds of MP3 files, for example, with interviews with Jamie Dimonof JPMorgan Chase and Lloyd Blankfein of Goldman Sachs. I am desperate to find time to write some code that automatically extracts random audio snippets from each and puts them on top of a slow ambient drone with plenty of reverb, so that I can relax to the dulcet tones of the financial industry explaining away its failings. (There’s a Paul Krugman interview that I assume is more critical.)

The recordings are just the beginning. They’ve released so many documents, and with the documents, a finding aid that you can download in handy PDF format, which will tell you where to, well, find things, pointing to thousands of documents. That aid alone is 1,439 pages.

Look, it is excellent that this exists, in public, on the web. But it also presents a very contemporary problem: What is transparency in the age of massive database drops? The data is available, but locked in MP3s and PDFs and other documents; it’s not searchable in the way a web page is searchable, not easy to comment on or share.

Consider the WikiLeaks release of State Department cables. They were exhausting, there were so many of them, they were in all caps. Or the trove of data Edward Snowden gathered on aUSB drive, or Chelsea Manning on CD. And the Ashley Madison leak, spread across database files and logs of credit card receipts. The massive and sprawling Sony leak, complete with whole email inboxes. And with the just-released Panama Papers, we see two exciting new developments: First, the consortium of media organizations that managed the leak actually came together and collectively, well, branded the papers, down to a hashtag (#panamapapers), informational website, etc. Second, the size of the leak itself—2.5 terabytes!—become a talking point, even though that exact description of what was contained within those terabytes was harder to understand. This, said the consortia of journalists that notably did not include The New York Times, The Washington Post, etc., is the big one. Stay tuned. And we are. But the fact remains: These artifacts are not accessible to any but the most assiduous amateur conspiracist; they’re the domain of professionals with the time and money to deal with them. Who else could be bothered?

If you watched the movie Spotlight, you saw journalists at work, pawing through reams of documents, going through, essentially, phone books. I am an inveterate downloader of such things. I love what they represent. And I’m also comfortable with many-gigabyte corpora spread across web sites. I know how to fetch data, how to consolidate it, and how to search it. I share this skill set with many data journalists, and these capacities have, in some ways, become the sole province of the media. Organs of journalism are among the only remaining cultural institutions that can fund investigations of this size and tease the data apart, identifying linkages and thus constructing informational webs that can, with great effort, be turned into narratives, yielding something like what we call “a story” or “the truth.”

Spotlight was set around 2001, and it features a lot of people looking at things on paper. The problem has changed greatly since then: The data is everywhere. The media has been forced into a new cultural role, that of the arbiter of the giant and semi-legal database. ProPublica, a nonprofit that does a great deal of data gathering and data journalism and then shares its findings with other media outlets, is one example; it funded a project called DocumentCloud with other media organizations that simplifies the process of searching through giant piles of PDFs (e.g., court records, or the results of Freedom of Information Act requests).

At some level the sheer boredom and drudgery of managing these large data leaks make them immune to casual interest; even the Ashley Madison leak, which I downloaded, was basically an opaque pile of data and really quite boring unless you had some motive to poke around.

If this is the age of the citizen journalist, or at least the citizen opinion columnist, it’s also the age of the data journalist, with the news media acting as product managers of data leaks, making the information usable, browsable, attractive. There is an uneasy partnership between leakers and the media, just as there is an uneasy partnership between the press and the government, which would like some credit for its efforts, thank you very much, and wouldn’t mind if you gave it some points for transparency while you’re at it.

Pause for a second. There’s a glut of data, but most of it comes to us in ugly formats. What would happen if the things released in the interest of transparency were released in actual transparent formats?…(More)”

Drones Marshaled to Drop Lifesaving Supplies Over Rwandan Terrain

Curated on April 6, 2016August 3, 2018 by Stefaan Verhulst

John Markoff at the New York Times: “From a bluff overlooking the Pacific Ocean, aloud pop signals the catapult launch of a small fixed-wing drone that is designed to carry medical supplies to remote locations almost 40 miles away.

The drones are the brainchild of a small group of engineers at a SiliconValley start-up called Zipline, which plans to begin operating a service with them for the government of Rwanda in July. The fleet of robot planes will initially cover more than half the tiny African nation, creating a highly automated network to shuttle blood and pharmaceuticals to remote locations in hours rather than weeks or months.

Rwanda, one of the world’s poorest nations, was ranked 170th by gross domestic product in 2014 by the International Monetary Fund. And so it is striking that the country will be the first, company executives said, to establish a commercial drone delivery network — putting it ahead of places like the United States, where there have been heavily ballyhooed futuristicdrone delivery systems promising urban and suburban package delivery from tech giants such as Amazon and Google….

That Rwanda is set to become the first country with a drone delivery network illustrates the often uneven nature of the adoption of new technology. In the United States, drones have run into a wall of regulation and conflicting rules. But in Rwanda, the country’s master development plan has placed a priority on the use of the machines, first for medicine and then more broadly for economic development….

The new drone system will initially be capable of making 50 to 150 daily deliveries of blood and emergency medicine to Rwanda’s 21 transfusing facilities, mostly in hospitals and clinics in the western half of the nation.

The drone system is based on a fleet of 15 small aircraft, each with twin electric motors, a 3.5-pound payload and an almost eight-foot wingspan.The system’s speed makes it possible to maintain a “cold chain” —essentially a temperature-controlled supply chain needed to provide blood and vaccines — which is often not practical to establish in developing countries.

The Zipline drones will use GPS receivers to navigate and communicate via the Rwandan cellular network. They will be able to fly in rough weather conditions, enduring winds up to 30 miles per hour….(More)”

Website Seeks to Make Government Data Easier to Sift Through

Curated on April 5, 2016August 3, 2018 by Stefaan Verhulst

Steve Lohr at the New York Times: “For years, the federal government, states and some cities have enthusiastically made vast troves of data open to the public. Acres of paper records on demographics, public health, traffic patterns, energy consumption, family incomes and many other topics have been digitized and posted on the web.

This abundance of data can be a gold mine for discovery and insights, but finding the nuggets can be arduous, requiring special skills.

A project coming out of the M.I.T. Media Lab on Monday seeks to ease that challenge and to make the value of government data available to a wider audience. The project, called Data U S A, bills itself as “the most comprehensive visualization of U.S. public data.” It is free, and its software code is open source, meaning that developers can build custom applications by adding other data.

Cesar A. Hidalgo, an assistant professor of media arts and sciences at the M.I.T. Media Lab who led the development of Data USA, said the website was devised to “transform data into stories.” Those stories are typically presented as graphics, charts and written summaries….Type “New York” into the Data USA search box, and a drop-down menu presents choices — the city, the metropolitan area, the state and other options. Select the city, and the page displays an aerial shot of Manhattan with three basic statistics: population (8.49 million), median household income ($52,996) and median age (35.8).

Lower on the page are six icons for related subject categories, including economy, demographics and education. If you click on demographics, one of the so-called data stories appears, based largely on data from the American Community Survey of the United States Census Bureau.

Using colorful graphics and short sentences, it shows the median age of foreign-born residents of New York (44.7) and of residents born in the United States (28.6); the most common countries of origin for immigrants (the Dominican Republic, China and Mexico); and the percentage of residents who are American citizens (82.8 percent, compared with a national average of 93 percent).

Data USA features a selection of data results on its home page. They include the g ender w age g ap in Connecticut; the r acial b reakdown of po verty in Flint, Mich.; the wages of p hysicians and s urgeons across the United States; and the institutions that award the most c omputer s cience degrees….(More)

Data to the Rescue: Smart Ways of Doing Good

Curated on April 5, 2016May 29, 2019 by Stefaan Verhulst

Nicole Wallace in the Chronicle of Philanthropy: “For a long time, data served one purpose in the nonprofit world: measuring program results. But a growing number of charities are rejecting the idea that data equals evaluation and only evaluation.

Of course, many nonprofits struggle even to build the simplest data system. They have too little money, too few analysts, and convoluted data pipelines. Yet some cutting-edge organizations are putting data to work in new and exciting ways that drive their missions. A prime example: The Polaris Project is identifying criminal networks in the human-trafficking underworld and devising strategies to fight back by analyzing its data storehouse along with public information.

Other charities dive deep into their data to improve services, make smarter decisions, and identify measures that predict success. Some have such an abundance of information that they’re even pruning their collection efforts to allow for more sophisticated analysis.

The groups highlighted here are among the best nationally. In their work, we get a sneak peek at how the data revolution might one day achieve its promise.

House Calls: Living Goods…

Living Goods launched in eastern Africa in 2007 with an innovative plan to tackle health issues in poor families and reduce deaths among children. The charity provides loans, training, and inventory to locals in Uganda and Kenya — mostly women — to start businesses selling vitamins, medicine, and other health products to friends and neighbors.

Founder Chuck Slaughter copied the Avon model and its army of housewives-turned-sales agents. But in recent years, Living Goods has embraced a 21st-century data system that makes its entrepreneurs better health practitioners. Armed with smartphones, they confidently diagnose and treat major illnesses. At the same time, they collect information that helps the charity track health across communities and plot strategy….

Unraveling Webs of Wickedness: Polaris Project…

Calls and texts to the Polaris Project’s national human-trafficking hotline are often heartbreaking, terrifying, or both.

Relatives fear that something terrible has happened to a missing loved one. Trafficking survivors suffering from their ordeal need support. The most harrowing calls are from victims in danger and pleading for help.

Last year more than 5,500 potential cases of exploitation for labor or commercial sex were reported to the hotline. Since it got its start in 2007, the total is more than 24,000.

As it helps victims and survivors get the assistance they need, the Polaris Project, a Washington nonprofit, is turning those phone calls and texts into an enormous storehouse of information about the shadowy world of trafficking. By analyzing this data and connecting it with public sources, the nonprofit is drawing detailed pictures of how trafficking networks operate. That knowledge, in turn, shapes the group’s prevention efforts, its policy work, and even law-enforcement investigations….

Too Much Information: Year Up…

Year Up has a problem that many nonprofits can’t begin to imagine: It collects too much data about its program. “Predictive analytics really start to stink it up when you put too much in,” says Garrett Yursza Warfield, the group’s director of evaluation.

What Mr. Warfield describes as the “everything and the kitchen sink” problem started soon after Year Up began gathering data. The group, which fights poverty by helping low-income young adults land entry-level professional jobs, first got serious about measuring its work nearly a decade ago. Though challenged at first to round up even basic information, the group over time began tracking virtually everything it could: the percentage of young people who finish the program, their satisfaction, their paths after graduation through college or work, and much more.

Now the nonprofit is diving deeper into its data to figure out which measures can predict whether a young person is likely to succeed in the program. And halfway through this review, it’s already identified and eliminated measures that it’s found matter little. A small example: Surveys of participants early in the program asked them to rate their proficiency at various office skills. Those self-evaluations, Mr. Warfield’s team concluded, were meaningless: How can novice professionals accurately judge their Excel spreadsheet skills until they’re out in the working world?…

On the Wild Side: Wildnerness Society…Without room to roam, wild animals and plants breed among themselves and risk losing genetic diversity. They also fall prey to disease. And that’s in the best of times. As wildlife adapt to climate change, the chance to migrate becomes vital even to survival.

National parks and other large protected areas are part of the answer, but they’re not enough if wildlife can’t move between them, says Travis Belote, lead ecologist at the Wilderness Society.

“Nature needs to be able to shuffle around,” he says.

Enter the organization’s Wildness Index. It’s a national map that shows the parts of the country most touched by human activity as well as wilderness areas best suited for wildlife. Mr. Belote and his colleagues created the index by combining data on land use, population density, road location and size, water flows, and many other factors. It’s an important tool to help the nonprofit prioritize the locations it fights to protect.

In Idaho, for example, the nonprofit compares the index with information about known wildlife corridors and federal lands that are unprotected but meet the criteria for conservation designation. The project’s goal: determine which areas in the High Divide — a wild stretch that connects Greater Yellowstone with other protected areas — the charity should advocate to legally protect….(More)”