The Metric God That Failed


Jerry Muller in PS Long Reads: “Over the past few decades, formal institutions have increasingly been subjected to performance measurements that define success or failure according to narrow and arbitrary metrics. The outcome should have been predictable: institutions have done what they can to boost their performance metrics, often at the expense of performance itself.

…In 1986, the American management guru Tom Peters popularized the organizational theorist Mason Haire’s dictum that, “What gets measured gets done,” and with it a credo of measured performance that I call “metric fixation.” In time, the devotees of measured performance would arrive at a naive article of faith that is nonetheless appealing for its mix of optimism and scientism: “Anything that can be measured can be improved.”

In the intervening decades, this faith-based conceit has developed into a dogma about the relationship between measurement and performance. Evangelists of “disruption” and “best practices” have carried the new gospel to ever more distant shores. If you work in health care, education, policing, or the civil service, you have probably been subjected to the policies and practices wrought by metric-centrism.

There are three tenets to the metrical canon. The first holds that it is both possible and desirable to replace judgment – acquired through personal experience and talent – with numerical indicators of comparative performance based on standardized data. Second, making such metrics public and transparent ensures that institutions are held accountable. And, third, the best way to motivate people within organizations is to attach monetary or reputational rewards and penalties to their measured performance….(More)”.

Technology Landscape for Digital Identification


World Bank Report: “Robust, inclusive, and responsible identification systems can increase access to finance, healthcare, education, and other critical services and benefits. Identification systems are also key to improving efficiency and enabling innovation for public- and private-sector services, such as greater efficiency in the delivery of social safety nets and facilitating the development of digital economies. However, the World Bank estimates that more than 1.1 billion individuals do not have official proof of their identity.10 New technologies provide countries with the opportunity to leapfrog paper-based systems and rapidly establish a robust identification infrastructure. As a result, the countries are increasingly adopting nationwide digital identification (ID) programs and leveraging them in other sectors.

Whether a country is enhancing existing ID systems or implementing new systems from the ground up, technology choices are critical to the success of digital identification systems. A number of new technologies are emerging to enable various aspects of ID lifecycle. For some of these technologies, no large-scale studies have been done; for others, current speculation makes objective evaluations difficult.

This report is a first attempt to develop a comprehensive overview of the current technology landscape for digital identification. It is intended to serve as a framework for understanding the myriad options and considerations of technology in this rapidly advancing agenda and in no way is intended to provide advice on specific technologies, particularly given there are a number of other considerations and country contexts which need to be considered. This report also does not advocate the use of a certain technology from a particular vendor for any particular application.

While some technologies are relatively easy to use and affordable, others are costly or so complex that using them on a large scale presents daunting challenges. This report provides practitioners with an overview of various technologies and advancements that are especially relevant for digital identification systems. It highlights key benefits and challenges associated with each technology. It also provides a framework for assessing each technology on multiple criteria, including length of time it has been in use, its ease of integration with legacy and future systems, and its interoperability with other technologies. The practitioners and stakeholders who read this are reminded to bear in mind that the technologies associated with ID systems are rapidly evolving, and that this report, prepared in early 2018, is a snapshot in time. Therefore, technology limitations and challenges highlighted in this report today may not be applicable in the years to come….(More)”

Issuing Bonds to Invest in People


Tina Rosenberg at the New York Times: “The first social impact bond began in 2010 in Peterborough, England. Investors funded a program aimed at keeping newly released short-term inmates out of prison. It reduced reoffending by 9 percent compared to a control group, exceeding its target. So investors got their money back, plus interest.

Seldom has a policy idea gone viral so fast. There are now 108 such bonds, in 24 countries. The United States has 20, leveraging $211 million in investment capital, and at least 50 more are on the way. These bonds fund programs to reduce Oklahoma’s population of women in prison, help low-income mothers to have healthy pregnancies in South Carolina, teach refugees and immigrants English and job skills in Boston, house the homeless in Denver, and reduce storm water runoff in the District of Columbia. There’s a Forest Resilience Bond underway that seeks to finance desperately needed wildfire prevention.

Here’s how social impact bonds differ from standard social programs:

They raise upfront money to do prevention. Everyone knows most prevention is a great investment. But politicians don’t do “think ahead” very well. They hate to spend money now to create savings their successors will reap. Issuing a social impact bond means they don’t have to.

They concentrate resources on what works. Bonds build market discipline, since investors demand evidence of success.

They focus attention on outcomes rather than outputs. “Take work-force training,” said David Wilkinson, commissioner of Connecticut’s Office of Early Childhood. “We tend to pay for how many people receive training. We’re less likely to pay for — or even look at — how many people get good jobs.” Providers, he said, were best recognized for their work “when we reward them for outcomes they want to see and families they are serving want to achieve.”

They improve incentives.Focusing on outcomes changes the way social service providers think. In Connecticut, said Duryea, they now have a financial incentive to keep children out of foster care, rather than bring more in.

They force decision makers to look at data. Programs start with great fanfare, but often nobody then examines how they are doing. But with a bond, evaluation is essential.

They build in flexibility.“It’s a big advantage that they don’t prescribe what needs to be done,” said Cohen. The people on the ground choose the strategy, and can change it if necessary. “Innovators can think outside the box and tackle health or education in revolutionary ways,” he said.

…In the United States, social impact bonds have become synonymous with “pay for success” programs. But there are other ways to pay for success. For example, Wilkinson, the Connecticut official, has just started an Outcomes Rate Card — a way for a government to pay for home visits for vulnerable families. The social service agencies get base pay, but also bonuses. If a client has a full-term birth, the agency gets an extra $135 for a low-risk family, $170 for a hard-to-help one. A client who finds stable housing brings $150 or $220 to the agency, depending on the family’s situation….(More)”.

Why the web has challenged scientists’ authority – and why they need to adapt


Andrew J. Hoffman at The Conversation: “Academia is in the midst of a crisis of relevance. Many Americans are ignoring the conclusions of scientists on a variety of issues including climate change and natural selection. Some state governments are cutting funding for higher education; the federal government is threatening to cut funding for research. Resentful students face ever increasing costs for tuition.

And distrustful segments of society fear what academia does; one survey found that 58 percent of Republicans and Republican-leaning independents say colleges and universities have a negative effect on the way things are going in the country.

There are multiple causes for this existential crisis, but one in particular deserves special attention. The web is fundamentally changing the channels through which science is communicated – who can create it, who can access it and ultimately what it is. Society now has instant access to more news and information than ever before; knowledge is being democratized. And as a result, the role of the scientist in society is in flux.

But rather than facing this changing landscape head on, research shows that many in academia are resisting its inevitability. In many ways, this response has parallels to that of the Catholic Church in the wake of the invention of the printing press and its role in hastening the Protestant Reformation. I hope this comparison offers a compelling provocation for the scientific community to come to grips with the cataclysmic changes we are now living through and ignore at our peril….(More)”.

How Blockchain can benefit migration programmes and migrants


Solon Ardittis at the Migration Data Portal: “According to a recent report published by CB Insights, there are today at least 36 major industries that are likely to benefit from the use of Blockchain technology, ranging from voting procedures, critical infrastructure security, education and healthcare, to car leasing, forecasting, real estate, energy management, government and public records, wills and inheritance, corporate governance and crowdfunding.

In the international aid sector, a number of experiments are currently being conducted to distribute aid funding through the use of Blockchain and thus to improve the tracing of the ways in which aid is disbursed. Among several other examples, the Start Network, which consists of 42 aid agencies across five continents, ranging from large international organizations to national NGOs, has launched a Blockchain-based project that enables the organization both to speed up the distribution of aid funding and to facilitate the tracing of every single payment, from the original donor to each individual assisted.

As Katherine Purvis of The Guardian noted, “Blockchain enthusiasts are hopeful it could be the next big development disruptor. In providing a transparent, instantaneous and indisputable record of transactions, its potential to remove corruption and provide transparency and accountability is one area of intrigue.”

In the field of international migration and refugee affairs, however, Blockchain technology is still in its infancy.

One of the few notable examples is the launch by the United Nations (UN) World Food Programme (WFP) in May 2017 of a project in the Azraq Refugee Camp in Jordan which, through the use of Blockchain technology, enables the creation of virtual accounts for refugees and the uploading of monthly entitlements that can be spent in the camp’s supermarket through the use of an authorization code. Reportedly, the programme has contributed to a reduction by 98% of the bank costs entailed by the use of a financial service provider.

This is a noteworthy achievement considering that organizations working in international relief can lose up to 3.5% of each aid transaction to various fees and costs and that an estimated 30% of all development funds do not reach their intended recipients because of third-party theft or mismanagement.

At least six other UN agencies including the UN Office for Project Services (UNOPS), the UN Development Programme (UNDP), the UN Children’s Fund (UNICEF), UN Women, the UN High Commissioner for Refugees (UNHCR) and the UN Development Group (UNDG), are now considering Blockchain applications that could help support international assistance, particularly supply chain management tools, self-auditing of payments, identity management and data storage.

The potential of Blockchain technology in the field of migration and asylum affairs should therefore be fully explored.

At the European Union (EU) level, while a Blockchain task force has been established by the European Parliament to assess the ways in which the technology could be used to provide digital identities to refugees, and while the European Commission has recently launched a call for project proposals to examine the potential of Blockchain in a range of sectors, little focus has been placed so far on EU assistance in the field of migration and asylum, both within the EU and in third countries with which the EU has negotiated migration partnership agreements.

This is despite the fact that the use of Blockchain in a number of major programme interventions in the field of migration and asylum could help improve not only their cost-efficiency but also, at least as importantly, their degree of transparency and accountability. This at a time when media and civil society organizations exercise increased scrutiny over the quality and ethical standards of such interventions.

In Europe, for example, Blockchain could help administer the EU Asylum, Migration and Integration Fund (AMIF), both in terms of transferring funds from the European Commission to the eligible NGOs in the Member States and in terms of project managers then reporting on spending. This would help alleviate many of the recurrent challenges faced by NGOs in managing funds in line with stringent EU regulations.

As crucially, Blockchain would have the potential to increase transparency and accountability in the channeling and spending of EU funds in third countries, particularly under the Partnership Framework and other recent schemes to prevent irregular migration to Europe.

A case in point is the administration of EU aid in response to the refugee emergency in Greece where, reportedly, there continues to be insufficient oversight of the full range of commitments and outcomes of large EU-funded investments, particularly in the housing sector. Another example is the set of recent programme interventions in Libya, where a growing number of incidents of human rights abuses and financial mismanagement are being brought to light….(More)”.

Smarter New York City: How City Agencies Innovate


Book edited by André Corrêa d’Almeida: “Innovation is often presented as being in the exclusive domain of the private sector. Yet despite widespread perceptions of public-sector inefficiency, government agencies have much to teach us about how technological and social advances occur. Improving governance at the municipal level is critical to the future of the twenty-first-century city, from environmental sustainability to education, economic development, public health, and beyond. In this age of acceleration and massive migration of people into cities around the world, this book explains how innovation from within city agencies and administrations makes urban systems smarter and shapes life in New York City.
Using a series of case studies, Smarter New York City describes the drivers and constraints behind urban innovation, including leadership and organization; networks and interagency collaboration; institutional context; technology and real-time data collection; responsiveness and decision making; and results and impact. Cases include residential organic-waste collection, an NYPD program that identifies the sound of gunshots in real time, and the Vision Zero attempt to end traffic casualties, among others. Challenging the usefulness of a tech-centric view of urban innovation, Smarter New York City brings together a multidisciplinary and integrated perspective to imagine new possibilities from within city agencies, with practical lessons for city officials, urban planners, policy makers, civil society, and potential private-sector partners….(More)”.

Small Data for Big Impact


Liz Luckett at the Stanford Social Innovation Review: “As an investor in data-driven companies, I’ve been thinking a lot about my grandfather—a baker, a small business owner, and, I now realize, a pioneering data scientist. Without much more than pencil, paper, and extraordinarily deep knowledge of his customers in Washington Heights, Manhattan, he bought, sold, and managed inventory while also managing risk. His community was poor, but his business prospered. This was not because of what we celebrate today as the power and predictive promise of big data, but rather because of what I call small data: nuanced market insights that come through regular and trusted interactions.

Big data takes into account volumes of information from largely electronic sources—such as credit cards, pay stubs, test scores—and segments people into groups. As a result, people participating in the formalized economy benefit from big data. But people who are paid in cash and have no recognized accolades, such as higher education, are left out. Small data captures those insights to address this market failure. My grandfather, for example, had critical customer information he carefully gathered over the years: who could pay now, who needed a few days more, and which tabs to close. If he had access to a big data algorithm, it likely would have told him all his clients were unlikely to repay him, based on the fact that they were low income (vs. high income) and low education level (vs. college degree). Today, I worry that in our enthusiasm for big data and aggregated predictions, we often lose the critical insights we can gain from small data, because we don’t collect it. In the process, we are missing vital opportunities to both make money and create economic empowerment.

We won’t solve this problem of big data by returning to my grandfather’s shop floor. What we need is more and better data—a small data movement to supply vital missing links in marketplaces and supply chains the world over. What are the proxies that allow large companies to discern whom among the low income are good customers in the absence of a shopkeeper? At The Social Entrepreneurs’ Fund (TSEF), we are profitably investing in a new breed of data company: enterprises that are intentionally and responsibly serving low-income communities, and generating new and unique insights about the behavior of individuals in the process. The value of the small data they collect is becoming increasingly useful to other partners, including corporations who are willing to pay for it. It is a kind of dual market opportunity that for the first time makes it economically advantageous for these companies to reach the poor. We are betting on small data to transform opportunities and quality of life for the underserved, tap into markets that were once seen as too risky or too costly to reach, and earn significant returns for investors….(More)”.

How Universities Are Tackling Society’s Grand Challenges


Michelle Popowitz and Cristin Dorgelo in Scientific American: “…Universities embarking on Grand Challenge efforts are traversing new terrain—they are making commitments about research deliverables rather than simply committing to invest in efforts related to a particular subject. To mitigate risk, the universities that have entered this space are informally consulting with others regarding effective strategies, but the entire community would benefit from a more formal structure for identifying and sharing “what works.” To address this need, the new Community of Practice for University-Led Grand Challenges—launched at the October 2017 workshop—aims to provide peer support to leaders of university Grand Challenge programs, and to accelerate the adoption of Grand Challenge approaches at more universities supported by cross-sector partnerships.

The university community has identified extensive opportunities for collaboration on these Grand Challenge programs with other sectors:

  • Philanthropy can support the development of new Grand Challenge programs at more universities by establishing planning and administration grant programs, convening experts, and providing funding support for documenting these models through white papers and other publications and for evaluation of these programs over time.
  • Relevant associations and professional development organizations can host learning sessions about Grand Challenges for university leaders and professionals.
  • Companies can collaborate with universities on Grand Challenges research, act as sponsors and hosts for university-led programs and activities, and offer leaders, experts, and other personnel for volunteer advisory roles and tours of duties at universities.
  • Federal, State, and local governments and elected officials can provide support for collaboration among government agencies and offices and the research community on Grand Challenges.

Today’s global society faces pressing, complex challenges across many domains—including health, environment, and social justice. Science (including social sciences), technology, the arts, and humanities have critical roles to play in addressing these challenges and building a bright and prosperous future. Universities are hubs for discovery, building new knowledge, and changing understanding of the world. The public values the role universities play in education; yet as a sector, universities are less effective at highlighting their roles as the catalysts of new industries, homes for the fundamental science that leads to new treatments and products, or sources of the evidence on which policy decisions should be made.

By coming together as universities, collaborating with partners, and aiming for ambitious goals to address problems that might seem unsolvable, universities can show commitment to their communities and become beacons of hope….(More)”.

The Tyranny of Metrics


Book by Jerry Z. Muller on “How the obsession with quantifying human performance threatens our schools, medical care, businesses, and government…

Today, organizations of all kinds are ruled by the belief that the path to success is quantifying human performance, publicizing the results, and dividing up the rewards based on the numbers. But in our zeal to instill the evaluation process with scientific rigor, we’ve gone from measuring performance to fixating on measuring itself. The result is a tyranny of metrics that threatens the quality of our lives and most important institutions. In this timely and powerful book, Jerry Muller uncovers the damage our obsession with metrics is causing–and shows how we can begin to fix the problem.

Filled with examples from education, medicine, business and finance, government, the police and military, and philanthropy and foreign aid, this brief and accessible book explains why the seemingly irresistible pressure to quantify performance distorts and distracts, whether by encouraging “gaming the stats” or “teaching to the test.” That’s because what can and does get measured is not always worth measuring, may not be what we really want to know, and may draw effort away from the things we care about. Along the way, we learn why paying for measured performance doesn’t work, why surgical scorecards may increase deaths, and much more. But metrics can be good when used as a complement to—rather than a replacement for—judgment based on personal experience, and Muller also gives examples of when metrics have been beneficial…(More)”.

How AI Could Help the Public Sector


Emma Martinho-Truswell in the Harvard Business Review: “A public school teacher grading papers faster is a small example of the wide-ranging benefits that artificial intelligence could bring to the public sector. A.I could be used to make government agencies more efficient, to improve the job satisfaction of public servants, and to increase the quality of services offered. Talent and motivation are wasted doing routine tasks when they could be doing more creative ones.

Applications of artificial intelligence to the public sector are broad and growing, with early experiments taking place around the world. In addition to education, public servants are using AI to help them make welfare payments and immigration decisions, detect fraud, plan new infrastructure projects, answer citizen queries, adjudicate bail hearings, triage health care cases, and establish drone paths.  The decisions we are making now will shape the impact of artificial intelligence on these and other government functions. Which tasks will be handed over to machines? And how should governments spend the labor time saved by artificial intelligence?

So far, the most promising applications of artificial intelligence use machine learning, in which a computer program learns and improves its own answers to a question by creating and iterating algorithms from a collection of data. This data is often in enormous quantities and from many sources, and a machine learning algorithm can find new connections among data that humans might not have expected. IBM’s Watson, for example, is a treatment recommendation-bot, sometimes finding treatments that human doctors might not have considered or known about.

Machine learning program may be better, cheaper, faster, or more accurate than humans at tasks that involve lots of data, complicated calculations, or repetitive tasks with clear rules. Those in public service, and in many other big organizations, may recognize part of their job in that description. The very fact that government workers are often following a set of rules — a policy or set of procedures — already presents many opportunities for automation.

To be useful, a machine learning program does not need to be better than a human in every case. In my work, we expect that much of the “low hanging fruit” of government use of machine learning will be as a first line of analysis or decision-making. Human judgment will then be critical to interpret results, manage harder cases, or hear appeals.

When the work of public servants can be done in less time, a government might reduce its staff numbers, and return money saved to taxpayers — and I am sure that some governments will pursue that option. But it’s not necessarily the one I would recommend. Governments could instead choose to invest in the quality of its services. They can re-employ workers’ time towards more rewarding work that requires lateral thinking, empathy, and creativity — all things at which humans continue to outperform even the most sophisticated AI program….(More)”.