Counting down to ‘Evaluating Digital Citizen Engagement: A practical guide’

Curated on Posted on by Stefaan Verhulst

Matt Haikin at Aptiva: “Last year, Aptivate led a consortium of researchers and practitioners to explore the role of technology in citizen-engagement and participation in the development sector, and how to evaluate the success of such activities…

The guide was researched, developed and written by the multidisciplinary team of Matt Haikin (Aptivate), Savita Bailur (now at Caribou Digital), Evangelia Berdou (IDS), Claudia Lopes (now at Africa’s Voices), Jonathan Dudding (ICA:UK) and Martin Belcher (now at Palladium Group).

The result – ‘Evaluating Digital Citizen Engagement: A practical guide’ will be published in electronic form on the World Bank’s Open Knowledge Repository any day now.

The Guide forms part of the recommended reading for the World Bank’s high profile Coursera course Citizen Engagement : A game changer for development? …So what can you expect to find in the Guide…

  • Practical tools and guidelines for use in evaluating or designing activities in the expanding field of digital citizen engagement
  • Resources for anyone seeking to better understand the role of digital technology in citizen engagement.
  • Five ‘lenses’ you can use to explore different perspectives through which digital citizen engagement might be viewed (Objective, Control, Participation, Technology, Effects)Detailed advice and tips specific to technology and citizen engagement through every stage of a typical evaluation lifecycle (Scoping, Designing, Planning & Implementing, Analysing, Sharing, Reflecting & Learning)
  • Toolkits to help you design your own research questions and evaluation designs …(More)”

The city as platform

Curated on Posted on by Stefaan Verhulst

The report of the 2015 Aspen Institute Roundtable on Information Technology: “In the age of ubiquitous Internet connections, smartphones and data, the future vitality of cities is increasingly based on their ability to use digital networks in intelligent, strategic ways. While we are accustomed to thinking of cities as geophysical places governed by mayors, conventional political structures and bureaucracies, this template of city governance is under great pressure to evolve. Urban dwellers now live their lives in all sorts of hyper-connected virtual spaces, pulsating with real-time information, intelligent devices, remote-access databases and participatory crowdsourcing. Expertise is distributed, not centralized. Governance is not just a matter of winning elections and assigning tasks to bureaucracies; it is about the skillful collection and curation of information as a way to create new affordances for commerce and social life.

Except among a small class of vanguard cities, however, the far-reaching implications of the “networked city” for economic development, urban planning, social life and democracy, have not been explored in depth. The Aspen Institute Communications and Society Program thus convened an eclectic group of thirty experts to explore how networking technologies are rapidly changing the urban landscape in nearly every dimension. The goal was to learn how open networks, onlinecooperation and open data can enhance urban planning and administration, and more broadly, how they might improve economic opportunity and civic engagement. The conference, the 24th Annual Aspen Roundtable on Information Technology, also addressed the implications of new digital technologies for urban transportation, public health and safety, and socio-economic inequality….(Download the InfoTech 2015 Report)”

Sticky-note strategy: How federal innovation labs borrow from Silicon Valley

Curated on Posted on by Stefaan Verhulst

Carten Cordell in the Federal Times: “The framework for an integrated security solution in the Philippines is built on a bedrock of sticky notes. So is the strategy for combating piracy in East Africa and a handful of other plans that Zvika Krieger is crafting in a cauldron of collaboration within the State Department.

More specifically, Krieger, a senior adviser for strategy within the department’s Bureau of Political-Military Affairs, is working in the bureau’s Strategy Lab, just one pocket of federal government where a Silicon Valley-playbook for innovation is being used to develop policy solutions….

Krieger and a host of other policy thinkers learned a new way to channel innovation for policy solutions called human-centered design, or design thinking. While arguably new in government, the framework has long been in use by the tech sector to design products that will serve the needs of their customers. The strategy of group thinking towards a policy — which is more what these innovation labs seek to achieve — has been used before as well….Where the government has started to use HCD is in developing new policy solutions within a multifaceted group of stakeholders that can contribute a well-rounded slate of expertise. The product is a strategy that is developed from the creative thoughts of a team of experts, rather than a single specialized source….

The core tenet of HCD is to establish a meritocracy of ideas that is both empathetic of thought and immune to hierarchy. In order to get innovative solutions for a complex problem, Krieger forms a team of experts and stakeholders. He then mixes in outside thought leaders he calls “wild cards” to give the group outside perspective.

The delicate balance opens discussion and the mix of ideas ultimately form a strategy for handling the problem. That strategy might involve a technology; but it could also be a new partnership, a new function within an office, or a new acquisition program. Because the team is comprised of multiple experts, it can navigate the complexity more thoroughly, and the wild cards can offer their expertise to provide solutions the stakeholders may not have considered….

Human-centered design has been working its way through pockets of the federal government for a few years now. The Office of Personnel Management opened its Innovation Lab in 2012 and was tasked with improving the USAJobs website. The Department of Health and Human Services opened the IDEA Lab in 2013 to address innovation in its mission. The Department of Veteran Affairs has a Center of Innovation to identify new approaches to meet the current and future needs of veterans, and the departments of Defense and State both have innovation labs tackling policy solutions.

The concept is gaining momentum. This fall, the Obama administration released a strategy report calling for a network of innovation labs throughout federal agencies to develop new policy solutions through HCD.

“I think the word is spreading. It’s kind of like a whisper campaign, in the most positive way,” said an administration official with knowledge of innovation labs and HCD strategies, who was not authorized to speak to the press. “I think, again, the only constraint here is that we don’t have enough of them to be able to imbue this knowledge across government. We need many more people.”

A March 2014 GAO report said that the OPM Innovation Lab had not developed consistent performance targets that would allow it to assess the success of its projects. The report recommended more consistent milestones to assess progress, which the agency addressed through a series of pilot programs….

In the State Department’s Bureau of Educational and Cultural Affairs, an innovation lab called the Collaboratory is in its second year of existence, using HCD strategies to improve projects like the Fulbright program and other educational diplomacy efforts.

The Education Diplomacy initiative, for example, used HCD to devise ways to increase education access abroad using State resources. Defining U.S. embassies as the end user, the Collaboratory then analyzed the areas of need at the installations and began crafting policies.

“We identified a couple of area where we thought we could make substantial gains quite quickly and in a budget neutral way,” Collaboratory Deputy Director Paul Kruchoski said. The process allowed multiple stakeholders like the U.S. Agency for International Development, Peace Corps and the Department of Education to help craft the policy and create what Kruchoski called “feedback loops” to refine throughout the embassies…(More)”

 

How to Hold Governments Accountable for the Algorithms They Use

Curated on Posted on by Stefaan Verhulst

 in Slate: “In 2015 more than 59 million Americans received some form ofbenefit from the Social Security Administration, not just for retirement but also for disability or as a survivor of a deceased worker. It’s a behemoth of a government program, and keeping it solvent has preoccupied the Office of the Chief Actuary of theSocial Security Administration for years. That office makes yearly forecasts of key demographic (such as mortality rates) or economic (for instance, labor forceparticipation) factors that inform how policy can or should change to keep theprogram on sound financial footing. But a recent Harvard University study examinedseveral of these forecasts and found that they were systematically biased—underestimating life expectancy and implying that funds were on firmer financialground than warranted. The procedures and methods that the SSA uses aren’t openfor inspection either, posing challenges to replicating and debugging those predictivealgorithms.

Whether forecasting the solvency of social programs, waging a war, managingnational security, doling out justice and punishment, or educating the populace,government has a lot of decisions to make—and it’s increasingly using algorithms tosystematize and scale that bureaucratic work. In the ideal democratic state, theelectorate chooses a government that provides social goods and exercises itsauthority via regulation. The government is legitimate to the extent that it is heldaccountable to the citizenry. Though as the SSA example shows, tightly heldalgorithms pose issues of accountability that grind at the very legitimacy of thegovernment itself.

One of the immensely useful abilities of algorithms is to rank and prioritize hugeamounts of data, turning a messy pile of items into a neat and orderly list. In 2013 theObama administration announced that it would be getting into the business ofranking colleges, helping the citizens of the land identify and evaluate the “best”educational opportunities. But two years later, the idea of ranking colleges had beenneutered, traded in for what amounts to a data dump of educational statistics calledthe College Scorecard. The human influences, subjective factors, and methodologicalpitfalls involved in quantifying education into rankings would be numerous. Perhapsthe government sensed that any ranking would be dubious—that it would be riddledwith questions of what data was used and how various statistical factors wereweighted. How could the government make such a ranking legitimate in the eyes ofthe public and of the industry that it seeks to hold accountable?

That’s a complicated question that goes far beyond college rankings. But whatever theend goal, government needs to develop protocols for opening up algorithmic blackboxes to democratic processes.

Transparency offers one promising path forward. Let’s consider the new risk-assessment algorithm that the state of Pennsylvania is developing to help make criminal sentencing decisions. Unlike some other states that are pursuing algorithmiccriminal justice using proprietary systems, the level of transparency around thePennsylvania Risk Assessment Project is laudable, with several publicly available in-depth reports on the development of the system….(More)’

The Value of Open Governance: Adaptive Learning and Development

Curated on Posted on by Stefaan Verhulst

Alan Hudson at Global Policy: “Open governance is governance that puts into practice principles of transparency, participation and accountability. Proponents of open governance tend to make their case on the basis of two sets of arguments. Normative, or intrinsic value, arguments hold that open governance is a good thing in itself. The idea here is that people have a right to open governance, regardless of its outcomes. Instrumental, or extrinsic value, arguments make the case that open governance is important because it contributes to better outcomes; less corruption, lower poverty, greater prosperity, for instance.

Both sets of arguments have their weaknesses. My aim in this post is to outline these vulnerabilities and then to suggest an alternative way of thinking about the value of open governance, a conceptual framework that has practical implications for those of us working to harness the potential of open governance.

On the intrinsic side, the idea that open governance, or rights, are good things in themselves is questionable. Normative arguments can be useful, but not everyone thinks that the same things are “good” (see my post on moving “beyond the Good Governance mantra” for more). This is particularly problematic when normative arguments are made about the form that governance should take, rather than the functions that it should enable (see Matt Andrews on “hippos in the Sahara”). On the extrinsic side, the evidence about whether more open governance leads to better development outcomes remains decidedly patchy, despite substantial investments in exploring “what works”. We need, I would argue, to think not just harder, but also differently, about the ways in which open processes of governance can make a difference.

In some cases, organizations (including Global Integrity at times) hedge their bets, asserting both that citizens have a right to open governance and that open governance can lead to better development outcomes. This can be a reasonable argument to make, and may have some pragmatic benefits, but it has led to a situation where the theory of change about how open governance can contribute to better development outcomes remains unclear and under-examined. This has contributed to unrealistic expectations being placed on the open governance agenda, and complicates the task of marshaling the evidence to assess what works in order to inform more effective action. On the ground this can mean that investments in supporting the open governance agenda are misdirected and fail to deliver the expected benefits….

Open governance matters, not because it is a good thing in itself, or because it leads directly to better development outcomes (it rarely does). Instead, open governance matters because it enhances the ability of communities, to try, learn and adapt their way towards better development outcomes. This, it should be noted, is always about using evidence to navigate and engage with the prevailing political dynamics….(More)”

Research Consortium on the Impact of Open Government Processes

Curated on Posted on by Stefaan Verhulst
Image

“Mounting anecdotal evidence supports the case for open government. Sixty-nine national governments andcounting have signed on as participants in the Open Government Partnership, committing to rethinking theway they engage with citizens, while civil society organizations (CSOs) are increasingly demanding andbuilding mechanisms for this shift.Yet even as the open government agenda gains steam, relatively littlesystematic research has been done to examine the ways different types and sequences of reforms haveplayed out in various contexts, and with what impact. This is due in part to the newness of the field, but alsoto the challenges in attributing specific outcomes to any governance initiative. While acknowledging that thesearch for cookie-cutter “best practices” is of limited value, there is no doubt that reform-minded actorscould benefit from a robust analytical framework and more thorough understanding of experiences indifferent contexts to date.

To address these knowledge gaps, and to sharpen our ways of thinking about the difference that opengovernment processes can make, a range of public, academic, and advocacy organizations established aresearch consortium to convene actors, leverage support, and catalyze research. Its founding members areGlobal Integrity,The Governance Lab @ NYU (The GovLab), the World Bank’s Open Government GlobalSolutions Group, Open Government Partnership Support Unit, and Results for Development Institute. TheConsortium aims to build on existing research – including but not limited to the work of existing researchnetworks such as the MacArthur Foundation Research Network on Opening Governance – to improve ourunderstanding of the effectiveness and impact of open government reforms. That is, to what extent andthrough which channels do such reforms actually improve transparency, accessibility, and accountability; how does this play out differently in different contexts; and can we trace tangible improvements in the livesof citizens to these measures…..

Countries participating in the Open Government Partnership have signed on to the view that opengovernment is intrinsically good in terms of strengthening civic participation and democratic processes.Governments are also increasingly looking at such initiatives through a return-on-investment (ROI) lens: dosuch reforms lead to cost savings that allow them to allocate and spend resources more efficiently on publicservices? Does the availability and accessibility of open government data create economic opportunities,including jobs and new businesses? The Consortium is excited to support innovative research aimed atunderstanding the extent to which reforms deliver, not only in terms of open governance itself, but also interms of improved public sector performance and service delivery gains. This focus will also help theConsortium identify research-driven stories of the impact that open governance reforms are having….(More)”

A Government of the Future

Curated on Posted on by Stefaan Verhulst

White House Fact Sheet on The President’s Fiscal Year 2017 Budget: “…The President is committed to driving last­ing change in how Government works – change that makes a significant, tangible, and positive difference in the economy and the lives of the American people. Over the past seven years, the Administration has launched successful efforts to modernize and improve citizen-facing services, eliminate wasteful spending, reduce the Federal real property footprint, improve the use of evidence to improve program performance, and spur innova­tion in the private sector by opening to the public tens of thousands of Federal data sets and inno­vation assets at the national labs.

Supporting the President’s Management Agenda. The Budget includes investments to continue driving the President’s Management Agenda by improving the service we provide to the American public; leveraging the Federal Government’s buying power to bring more value and efficiency to how we use taxpayer dollars; opening Government data and research to the private sector to drive innovation and economic growth; promoting smarter information technology; modernizing permitting and environmental review processes; creating new Idea Labs to support employees with promising ideas; and, attracting and retaining the best talent in the Federal workforce.

Supporting Digital Service Delivery for Citizens. In 2014 the Administration piloted the U.S. Digital Service, a unit of innovators, entrepreneurs, and engineers. This team of America’s best digital experts has worked in collaboration with Federal agencies to implement streamlined and effective digital technology practices on the Nation’s highest priority programs. This work includes collaborating with the Department of Education to launch the new College Scorecard to give stu­dents, parents, and their advisors most reliable national data to help with college choice and supporting the U.S. Citizenship and Immigration Services (USCIS) transition to launch the new myUSCIS which makes it easier for users to access information about the immigration process and immigration ser­vices. To institutionalize the dramatic improve­ments that this approach has demonstrated, the Budget supports the Administration’s aggressive goal of hiring and placing 500 top technology and design experts to serve in the Government by January 2017.

Strengthening Federal Cybersecurity. As outlined above, the Budget provides $19 billion in resources for cybersecurity. This includes the creation of a new $3.1 billion revolving fund, the Information Technology Modernization Fund (ITMF), to retire the Government’s antiquated IT systems and transition to more secure and efficient modern IT systems, funding to streamline governance and secure Federal networks, and investments to strengthen the cybersecurity workforce and cybersecurity education across society.

Building Evidence and Encouraging Innovation. The President has made it clear that policy decisions should be driven by evidence so that the Federal government can do more of what works and less of what does not. The Administration’s evidence-based approaches have resulted in important gains in areas ranging from reducing veteran homelessness, to improving educational outcomes, to enhancing the effectiveness of international development programs. The Budget invests in expanding evidence-based approaches, developing and testing effective practices, and enhancing government’s capacity to build and use evidence, in particular by expanding access to administrative data and further developing Federal, State, local, and tribal data infrastructure.

Reorganizing Government to Succeed in the Global Economy. The Budget also includes proposals to consolidate and reorganize Government agencies to make them leaner and more efficient, and it increases the use of evidence and evaluation to ensure that taxpayer dollars are spent wisely on programs that work….(More). See also President Barack Obama’s FY 2017 Budget for the U.S. Government

New Tools for Collaboration: The Experience of the U.S. Intelligence Community

Curated on Posted on by Stefaan Verhulst

IBM Center for Business of Government: “This report is intended for an audience beyond the U.S. Intelligence Community—senior managers in government, their advisors and students of government performance who are interested in the progress of collaboration in a difficult environment. …

The purpose of this report is to learn lessons by looking at the use of internal collaborative tools across the Intelligence Community. The initial rubric was tools, but the real focus is collaboration, for while the tools can enable, what ultimately matters are policies and practices interacting with organizational culture. It looks for good practices to emulate. The ultimate question is how and how much could, and should, collaborative tools foster integration across the Community. The focus is analysis and the analytic process, but collaborative tools can and do serve many other functions in the Intelligence Community—from improving logistics or human resources, to better connecting collection and analysis, to assisting administration and development, to facilitating, as one interlocutor put it, operational “go” decisions. Yet it is in the analytic realm that collaboration is both most visible and most rubs against traditional work processes that are not widely collaborative.

The report defines terms and discusses concepts, first exploring collaboration and coordination, then defining collaborative tools and social media, then surveying the experience of the private sector. The second section of the report uses those distinctions to sort out the blizzard of collaborative tools that have been created in the various intelligence agencies and across them. The third section outlines the state of collaboration, again both within agencies and across them. The report concludes with findings and recommendations for the Community. The recommendations amount to a continuum of possible actions in making more strategic what is and will continue to be more a bottom-up process of creating and adopting collaborative tools and practices….(More)”

The Metric Tide

Curated on Posted on by Stefaan Verhulst

Book by James Wilsdon: Metrics evoke a mixed reaction from the research community. A commitment to using data and evidence to inform decisions makes many of us sympathetic, even enthusiastic, about the prospect of granular, real-time analysis of our own activities. Yet we only have to look around us, at the blunt use of metrics to be reminded of the pitfalls. Metrics hold real power: they are constitutive of values, identities and livelihoods.

How to exercise that power to positive ends is the focus of this book. Using extensive evidence-gathering, analysis and consultation, the authors take a thorough look at potential uses and limitations of research metrics and indicators. They explore the use of metrics across different disciplines, assess their potential contribution to the development of research excellence and impact and consider the changing ways in which universities are using quantitative indicators in their management systems. Finally, they consider the negative or unintended effects of metrics on various aspects of research culture.

Including an updated introduction from James Wilsdon, the book proposes a framework for responsible metrics and makes a series of targeted recommendations to show how responsible metrics can be applied in research management, by funders, and in the next cycle of the Research Excellence Framework.

The metric tide is certainly rising.  Unlike King Canute, we have the agency and opportunity – and in this book, a serious body of evidence – to influence how it washes through higher education and research….(More)”.

How to use research evidence to improve your work

Curated on Posted on by Stefaan Verhulst

NESTA: “We’re pleased to announce the launch of the latest publication in our series of practice guides – Using Research Evidence. Created by the Alliance for Useful Evidence and Nesta, the guide has been designed to help you improve the way you work by using evidence effectively.

Evidence can help you make better decisions. Whether it’s in a police station, a school classroom or the boardroom of a charity, using research-based evidence can help improve outcomes. It is helpful not only in frontline service-delivery, but also in creating smarter organisations – charities, local authorities, government departments – and in developing national policies or charity campaigns.

It is also useful not only to you as a decision-maker, but to the citizens, voters, donors and wider public you are trying to support. Evidence can show if your services are working (or failing), save money, and align services with public needs.

The guide is aimed at those working in government, charities, voluntary organisations, professional membership bodies and local authorities. It will help you to:

  • Learn about evidence-informed decision-making, and why research is an essential element of it.

  • Understand the different scenarios in which using evidence can help you, as well as the types of evidence you might need at different stages of development.

  • Explore different types of evidence, how to choose the most appropriate and how to judge its quality.

  • Get advice on finding the right evidence to support your case, and how to get your message across once you have it….

Download the report here.”