The Metric Tide


Book by James Wilsdon: Metrics evoke a mixed reaction from the research community. A commitment to using data and evidence to inform decisions makes many of us sympathetic, even enthusiastic, about the prospect of granular, real-time analysis of our own activities. Yet we only have to look around us, at the blunt use of metrics to be reminded of the pitfalls. Metrics hold real power: they are constitutive of values, identities and livelihoods.

How to exercise that power to positive ends is the focus of this book. Using extensive evidence-gathering, analysis and consultation, the authors take a thorough look at potential uses and limitations of research metrics and indicators. They explore the use of metrics across different disciplines, assess their potential contribution to the development of research excellence and impact and consider the changing ways in which universities are using quantitative indicators in their management systems. Finally, they consider the negative or unintended effects of metrics on various aspects of research culture.

Including an updated introduction from James Wilsdon, the book proposes a framework for responsible metrics and makes a series of targeted recommendations to show how responsible metrics can be applied in research management, by funders, and in the next cycle of the Research Excellence Framework.

The metric tide is certainly rising.  Unlike King Canute, we have the agency and opportunity – and in this book, a serious body of evidence – to influence how it washes through higher education and research….(More)”.

The rise of the citizen expert


Beth Noveck (The GovLab) at Policy Network: “Does the EU need to be more democratic? It is not surprising that Jürgen Habermas, Europe’s most famous democratic theorist, laments the dearth of mechanisms for “fulfilling the citizens’ political will” in European institutions. The controversial handling of the Greek debt crisis, according to Habermas, was clear evidence of the need for more popular input into otherwise technocratic decision-making. Incremental progress toward participation does not excuse a growing crisis of democratic legitimacy that, he says, is undermining the European project….

For participatory democrats like Habermas, opportunities for deliberative democratic input by citizens is essential to legitimacy. And, to be sure, the absence of such opportunities is no guarantee of more effective outcomes. A Greek referendum in July 2015 scuttled European austerity plans.

But pitting technocracy against citizenship is a false dichotomy resulting from the long-held belief, even among reformers, that only professional public servants or credentialed elites possess the requisite abilities to govern in a complex society. Citizens are spectators who can express opinions but cognitive incapacity, laziness or simply the complexity of modern society limit participation to asking people what they feel by means of elections, opinion polls, or social media.

Although seeing technocracy as the antinomy of citizenship made sense when expertise was difficult to pinpoint, now tools like LinkedIn, which make knowhow more searchable, are making it possible for public institutions to get more help from more diverse sources – including from within the civil service – systematically and could enable more members of the public to participate actively in governing based on what they know and care about. It is high time for institutions to begin to leverage such platforms to match the need for expertise to the demand for it and, in the process, increase engagement becoming more effective and more legitimate.

Such software does more than catalogue credentials. The internet is radically decreasing the costs of identifying diverse forms of expertise so that the person who has taken courses on an online learning platform can showcase those credentials with a searchable digital badge. The person who has answered thousands of questions on a question-and-answer website can demonstrate their practical ability and willingness to help. Ratings by other users further attest to the usefulness of their contributions. In short, it is becoming possible to discover what people know and can do in ever more finely tuned ways and match people to opportunities to participate that speak to their talents….

In an era in which it is commonplace for companies to use technology to segment customers in an effort to promote their products more effectively, the idea of matching might sound obvious. To be sure, it is common practice in business – but in the public sphere, the notion that participation should be tailored to the individual’s abilities and tethered to day-to-day practices of governing, not politicking, is new.  More accurately, it is a revival of Athenian life where citizen competence and expertise were central to economic and military success.

What makes this kind of targeted engagement truly democratic – and citizenship in this vision more active, robust, and meaningful – is that such targeting allows us to multiply the number and frequency of ways to engage productively in a manner consistent with each person’s talents. When we move away from focusing on citizen opinion to discovering citizen expertise, we catalyse participation that is also independent of geographical boundaries….(More)”

Moving from Open Data to Open Knowledge: Announcing the Commerce Data Usability Project


Jeffrey Chen, Tyrone Grandison, and Kristen Honey at the US Department of Commerce: “…in 2016, the DOC is committed to building on this momentum with new and expanded efforts to transform open data into knowledge into action.

DOC Open Data Graphic
Graphic Credit: Radhika Bhatt, Commerce Data Service

DOC has been in the business of open data for a long time. DOC’s National Oceanic and Atmospheric Administration (NOAA) alone collects and disseminates huge amounts of data that fuel the global weather economy—and this information represents just a fraction of the tens of thousands of datasets that DOC collects and manages, on topics ranging from satellite imagery to material standards to demographic surveys.

Unfortunately, far too many DOC datasets are either hard to find, difficult to use, and/or not yet publicly available on Data.gov, the home of U.S. government’s open data. This challenge is not exclusive to DOC; and indeed, under Project Open Data, Federal agencies are working hard on various efforts to make tax-payer funded data more easily discoverable.

CDUP screenshot

One of these efforts is DOC’s Commerce Data Usability Project (CDUP). To unlock the power of data, just making data open isn’t enough. It’s critical to make data easier to find and use—to provide information and tools that make data accessible and actionable for all users. That’s why DOC formed a public-private partnership to create CDUP, a collection of online data tutorials that provide students, developers, and entrepreneurs with the necessary context and code for them to start quickly extracting value from various datasets. Tutorials exist on topics such as:

  • NOAA’s Severe Weather Data Inventory (SWDI), demonstrating how to use hail data to save life and property. The tutorial helps users see that hail events often occur in the summer (late night to early morning), and in midwestern and southern states.
  • Security vulnerability data from the National Institute of Standards and Technology (NIST). The tutorial helps users see that spikes and dips in security incidents consistently occur in the same set of weeks each year.
  • Visible Infrared Imaging Radiometer Suite (VIIRS) data from the National Oceanic and Atmospheric Administration (NOAA). The tutorial helps users understand how to use satellite imagery to estimate populations.
  • American Community Survey (ACS) data from the U.S. Census Bureau. The tutorial helps users understand how nonprofits can identify communities that they want to serve based on demographic traits.

In the coming months, CDUP will continue to expand with a rich, diverse set of additional tutorials….(More)

Knowing Governance – The Epistemic Construction of Political Order


 

Book edited by Jan-Peter Voß and Richard Freeman: “This book is about the making of knowledge about governance and how it shapes political action. In a sense, doing politics has always turned on knowing governance, since political action builds on a certain understanding of what it is to act politically and how to do so effectively. Those seeking power have invariably wanted to know how collective order can be built and maintained: governing implies knowledge about the world to be governed and the resources available to do so, and about the interests and dispositions of the actors involved. What is more, while knowing governance has always been key to ruling effectively, it is at the same time a principal lever for those who seek to challenge authority. Shared knowledge is a precondition of collective action and of the imagined communities of modern politics, whether nations or social movements or issue-based constituencies….(Table of Contents) ”

 

Crowdsourcing City Government: Using Tournaments to Improve Inspection Accuracy


Edward Glaeser, Andrew Hillis, Scott Kominers and Michael Luca in American Economic Review: Papers and Proceedings:The proliferation of big data makes it possible to better target city services like hygiene inspections, but city governments rarely have the in-house talent needed for developing prediction algorithms. Cities could hire consultants, but a cheaper alternative is to crowdsource competence by making data public and offering a reward for the best algorithm. A simple model suggests that open tournaments dominate consulting contracts when cities can tolerate risk and when there is enough labor with low opportunity costs. We also report on an inexpensive Boston-based restaurant tournament, which yielded algorithms that proved reasonably accurate when tested “out-of-sample” on hygiene inspections….(More)”

 

Google Votes: A Liquid Democracy Experiment on a Corporate Social Network


Paper by Steve Hardt and Lia C. R. Lopes: “This paper introduces Google Votes, an experiment in liquid democracy built on Google’s internal corporate Google+ social network. Liquid democracy decision-making systems can scale to cover large groups by enabling voters to delegate their votes to other voters. This approach is in contrast to direct democracy systems where voters vote directly on issues, and representative democracy systems where voters elect representatives to vote on issues for them. Liquid democracy systems can provide many of the benefits of both direct and representative democracy systems with few of the weaknesses. Thus far, high implementation complexity and infrastructure costs have prevented widespread adoption. Google Votes demonstrates how the use of social-networking technology can overcome these barriers and enable practical liquid democracy systems. The case-study of Google Votes usage at Google over a 3 year timeframe is included, as well as a framework for evaluating vote visibility called the “Golden Rule of Liquid Democracy”….(More)”

How to use research evidence to improve your work


NESTA: “We’re pleased to announce the launch of the latest publication in our series of practice guides – Using Research Evidence. Created by the Alliance for Useful Evidence and Nesta, the guide has been designed to help you improve the way you work by using evidence effectively.

Evidence can help you make better decisions. Whether it’s in a police station, a school classroom or the boardroom of a charity, using research-based evidence can help improve outcomes. It is helpful not only in frontline service-delivery, but also in creating smarter organisations – charities, local authorities, government departments – and in developing national policies or charity campaigns.

It is also useful not only to you as a decision-maker, but to the citizens, voters, donors and wider public you are trying to support. Evidence can show if your services are working (or failing), save money, and align services with public needs.

The guide is aimed at those working in government, charities, voluntary organisations, professional membership bodies and local authorities. It will help you to:

  • Learn about evidence-informed decision-making, and why research is an essential element of it.

  • Understand the different scenarios in which using evidence can help you, as well as the types of evidence you might need at different stages of development.

  • Explore different types of evidence, how to choose the most appropriate and how to judge its quality.

  • Get advice on finding the right evidence to support your case, and how to get your message across once you have it….

Download the report here.”

Designing a toolkit for policy makers


 at UK’s Open Policy Making Blog: “At the end of the last parliament, the Cabinet Office Open Policy Making team launched the Open Policy Making toolkit. This was about giving policy makers the actual tools that will enable them to develop policy that is well informed, creative, tested, and works. The starting point was addressing their needs and giving them what they had told us they needed to develop policy in an ever changing, fast paced and digital world. In a way, it was the culmination of the open policy journey we have been on with departments for the past 2 years. In the first couple of months we saw thousands of unique visits….

Our first version toolkit has been used by 20,000 policy makers. This gave us a huge audience to talk to to make sure that we continue to meet the needs of policy makers and keep the toolkit relevant and useful. Although people have really enjoyed using the toolkit, user testing quickly showed us a few problems…

We knew what we needed to do. Help people understand what Open Policy Making was, how it impacted their policy making, and then to make it as simple as possible for them to know exactly what to do next.

So we came up with some quick ideas on pen and paper and tested them with people. We quickly discovered what not to do. People didn’t want a philosophy— they wanted to know exactly what to do, practical answers, and when to do it. They wanted a sort of design manual for policy….

How do we make user-centered design and open policy making as understood as agile?

We decided to organise the tools around the journey of a policy maker. What might a policy maker need to understand their users? How could they co-design ideas? How could they test policy? We looked at what tools and techniques they could use at the beginning, middle and end of a project, and organised tools accordingly.

We also added sections to remove confusion and hesitation. Our opening section ‘Getting started with Open Policy Making’ provides people with a clear understanding of what open policy making might mean to them, but also some practical considerations. Sections for limited timeframes and budgets help people realise that open policy can be done in almost any situation.

And finally we’ve created a much cleaner and simpler design that lets people show as much or little of the information as they need….

So go and check out the new toolkit and make more open policy yourselves….(More)”

Digital Decisions: Policy Tools in Automated Decision-Making


Ali Lange at CDT: “Digital technology has empowered new voices, made the world more accessible, and increased the speed of almost every decision we make as businesses, communities, and individuals. Much of this convenience is powered by lines of code that rapidly execute instructions based on rules set by programmers (or, in the case of machine learning, generated from statistical correlations in massive datasets)—otherwise known as algorithms. The technology that drives our automated world is sophisticated and obscure, making it difficult to determine how the decisions made by automated systems might fairly or unfairly, positively or negatively, impact individuals. It is also harder to identify where bias may inadvertently arise. Algorithmically driven outcomes are influenced, but not exclusively determined, by technical and legal limitations. The landscape of algorithmic decision-making is also shaped by policy choices in technology companies and by government agencies. Some automated systems create positive outcomes for individuals, and some threaten a fair society. By looking at a few case studies and drawing out the prevailing policy principle, we can draw conclusions about how to critically approach the existing web of automated decision-making. Before considering these specific examples, we will present a summary of the policy debate around data-driven decisions to give context to the examples raised. Then we will analyze three case studies from diverse industries to determine what policy interventions might be applied more broadly to encourage positive outcomes and prevent the risk of discrimination….(More)”

Platform for Mumbai’s slum entrepreneurs


Springwise: “We recently saw an initiative that empowered startup talent in a Finnish refugee camp, and now Design Museum Dharavi is a mobile museum that will provide a platform for makers in the Mumbai neighborhood.

The initiative is a brainchild of artist Jorge Rubio and Creative Industries Fund NL. Taking the model of a pop-up, it will stop at various locations throughout the neighborhood. Despite being an ‘informal settlement’, Dharavi is famed for producing very little waste due to a culture of recycling and repurposing. The mobile museum will showcase local makers, enable them to connect with potential clients and run workshops, ultimately elevating the global social perception towards life in the so-called ‘slums’. Home to over a million people, Dharavi has the additional tourism pull from appearing on the film Slumdog Millionaire…..(More)”