Collection

Randomized control trials (RCTs): interesting, but a marginal tool for governments

Curated on March 27, 2014May 29, 2019 by Stefaan Verhulst

ODI Researcher Philipp Krause at BeyondBudgets: “Randomized control trials (RCTs) have had a great decade. The stunning line-up of speakers who celebrated J-PAL’s tenth anniversary in Boston last December gives some indication of just how great. They are the shiny new tool of development policy, and a lot of them are pretty cool. Browsing through J-PAL’s library of projects, it’s easy to see how so many of them end up in top-notch academic journals.
So far, so good. But the ambition of RCTs is not just to provide a gold-standard measurement of impact. They aim to actually have an impact on the real world themselves. The scenario goes something like this: researchers investigate the effect of an intervention and use the findings to either get out of that mess quickly (if the intervention doesn’t work) or scale it up quickly (if it does). In the pursuit of this impact-seeker’s Nirvana, it’s easy to conflate a couple of things, notably that an RCT is not the only way to evaluate impact; and evaluating impact is not the only way to use evidence for policy. Unfortunately, it is now surprisingly common to hear RCTs conflated with evidence-use, and evidence-use equated with the key ingredient for better public services in developing countries. The reality of evidence use is different.
Today’s rich countries didn’t get rich by using evidence systematically. This is a point that we recently discussed at a big World Bank – ODI conference on the (coincidental?) tenth anniversary of the WDR 2004. Lant Pritchett made it best when describing Randomistas as engaging in faith-based activity: nobody could accuse the likes of Germany, Switzerland, Sweden or the US of achieving human development by systematically scaling up what works.
What these countries do have in spades is people noisily demanding stuff, and governments giving it to them. In fact, some of the greatest innovations in providing health, unemployment benefits and pensions to poor people (and taking them to scale) happened because citizens seemed to want them, and giving them stuff seemed like a good way to shut them up. Ask Otto Bismarck. It’s not too much of a stretch to call this the history of public spending in a nutshell….
The bottom line is governments s that care about impact have plenty of cheaper, timelier and more appropriate tools and options available to them than RCTs. That doesn’t mean RCTs shouldn’t be done, of course. And the evaluation of aid is a different matter altogether, where donors are free to be as inefficient about evidence-basing as they wish without burdening poor countries.
But for governments the choice of how to go about using systematic evidence is theirs to make. And it’s a tough capability to pick up. Many governments choose not to do it, and there’s no evidence that they suffer for it. It would be wrong for donors to suggest to low-income countries that RCTs are in any way critical for their public service capability. Better call them what they are: interesting, but marginal.”

Pursuing adoption of free and open source software in governments

Curated on March 27, 2014May 29, 2019 by Stefaan Verhulst

Andy Oram at O’Reilly Radar: “Reasons for government agencies to adopt free and open source software have been aired repeatedly, including my article mentioned earlier. A few justifications include:

Access: Document formats must allow citizens to read and submit documents without purchasing expensive tools.
Participation: Free software allows outside developers to comment and contribute.
Public ownership: Whatever tools are developed or purchased by the government should belong to the public, as long as no security issues are involved.
Archiving: Proprietary formats can be abandoned by their vendors after only two or three years.
Transparency: Free software allows the public to trust that the tools are accurate and have no security flaws.
Competition: The government has an interest in avoiding lock-in and ensuring that software can be maintained or replaced.
Cost: In the long run, an agency can save a lot of money by investing in programming or system administration skills, or hiring a firm to maintain the free software.

Obviously, though, government agencies haven’t gotten the memo. I’m not just talking metaphorically; there have been plenty of memos urging the use of open source, ranging from the US Department of Defense to laws passed in a number of countries.
And a lot of progress has taken place. Munich, famously, has switched its desktops to GNU/Linux and OpenOffice.org — but the process took 13 years. Elsewhere in Europe, Spain has been making strides, and the UK promises to switch. In Latin America, Brazil has made the most progress. Many countries that could benefit greatly from using free software — and have even made commitments to do so — are held back by a lack of IT staff with the expertise to do so.
Key barriers include:

Procurement processes: General consensus among knowledgeable software programmers holds that age-old rules for procurement shouldn’t be tossed out, but could be tweaked to admit bids from more small businesses that want to avoid the bureaucracy of registering with the government and answering Requests for Proposals (RFPs).
Habits of passivity: Government managers are well aware of how little they understand the software development process — in fact, if you ask them what they would need to adopt more open source software, they rarely come up with useful answers. They prefer to hand all development and maintenance to an outside firm, which takes full advantage of this to isolate agencies from one another and lock in expensive rates.
Lack of knowledgeable IT staff: The government managers have reason to keep hands off free software. One LibrePlanet audience member reported that he installed a good deal of free software at his agency, but that when he left, they could not find knowledgeable IT hires to take over. Bit by bit, the free software was replaced with proprietary products known to the new staff.
Political pressure: The urge to support proprietary companies doesn’t just come from their sales people or lobbyists. Buying software, like other products, is seen by politicians as a way of ensuring that jobs remain in their communities.
Lack of information: Free software is rarely backed by a marketing and sales organization, and even if managers have the initiative to go look for the software, they don’t know how to evaluate its maturity and readiness.

Thoroughgoing change in the area of software requires managers to have a certain consciousness at a higher level: they need to assert control over their missions and adopt agile workflows. That will inevitably spawn a desire for more control over the software that carries out these missions. A posting by Matthew Burton of the Consumer Financial Protection Bureau shows that radical redirections like this are possible.
In the meantime, here are some ideas that the panelists and audience came up with:

Tweaking procurement: If projects can be kept cheap — as Code for America does using grants and other stratagems — they don’t have to navigate the procurement process. Hackathons and challenges can also produce results — but they have a number of limitations, particularly the difficulty developers have in understanding the requirements of the people they want to serve. Some agencies can also bypass procurement by forming partnerships with community groups who produce the software. Finally, a possibly useful model is to take a cut of income from a project instead of charging the government for it.
Education: Managers have heard of open source software by now — great progress from just a few years ago — and are curious about it. On the production side, we need to help them see the benefits of releasing code, and how to monitor their software vendors to make sure the code is really usable. On the consumption side, we need to teach them maturity models and connect them to strong development projects.
Sharing: Most governments have familiar tasks that can be met by the same software base, but end up paying to reinvent (or just reinstall) the wheel. Code for America started a peer network to encourage managers to talk to one another about solutions. The Brazilian government has started a Public Software Portal. The European Union has an open source database and the US federal government has posted a list of government software released as open source.”

European Commission launches network to foster web talent through Massive Open Online Courses (MOOCs)

Curated on March 27, 2014July 18, 2019 by Stefaan Verhulst

Press Release: “The Commission is launching a network of providers of MOOCs related to web and apps skills. MOOCs are online university courses which enable people to access quality education without having to leave their homes. The new network aims to map the demand for web-related skills across Europe and to promote the use of Massive Open Online Courses (MOOCs) for capacity-building in those fields.
Web-related industry is generating more economic growth than any other part of the European economy, but hundreds of thousands of jobs remain unfilled due to the lack of qualified staff.
European Commission Vice President Neelie Kroes, responsible for the Digital Agenda, said:
“By 2020, 90% of jobs will need digital skills. That is just around the corner, and we aren’t ready! Already businesses in Europe are facing a shortage of skilled ICT workers. We have to fill that gap, and this network we are launching will help us identify where the gaps are. This goes hand in hand with the work being done through the Grand Coalition for Digital Jobs”.
The Commission calls upon web entrepreneurs, universities, MOOC providers and online learners to join the network, which is part of the “Startup Europe” initiative.
Participants in the network benefit from the exchange of experiences and best practices, opportunities for networking, news updates, and the chance to participate in a conference dedicated to MOOCs for web and apps skills scheduled for the second half of 2014. In addition, the network offers a discussion group that can be found on the European Commission’s portal Open Education Europa. The initiative is coordinated by p.a.u. education and in partnership with Iversity.
Useful links
Link to EC press release on the launch of first pan-European university MOOCs
Open Education Europa website
Startup Europe website
Grand Coalition for Digital Jobs website”

The Use of ICT for Open Government in U. S. Municipalities Perceptions of Chief Administrative Officers

Curated on March 27, 2014August 3, 2018 by Stefaan Verhulst

Paper by Sukumar Ganapatiand Christopher G. Reddick in Public Performance & Management Review: “The extent to which U. S. municipal governments have adopted open e-government initiatives is examined through a survey and interviews with chief administrative officers (CAOs) along the three dimensions of open government: transparency, participation, and collaboration. A very high share of CAOs reported satisfaction with implementing open government overall. A majority indicated achievement along each of the open government dimensions. Whereas the CAOs had a significantly positive view of collaboration, their view on challenges was negatively significant for achievement and satisfaction with open government. The interviews indicated that the CAOs do not view open government as a fad and place it high on their respective agendas.”

Artists Show How Anyone Can Fight the Man with Open Data

Curated on March 26, 2014May 29, 2019 by Stefaan Verhulst

MotherBoard: “The UK’s Open Data Institute usually looks, as you’d probably expect, like an office full of people staring at screens. But visit at the moment and you might see a potato gun among the desks or a bunch of drone photos on the wall—all in the name of encouraging public discussion around and engagement with open data.
The ODI was set up by World Wide Web inventor Tim Berners-Lee and interdisciplinary researcher Nigel Shadbolt in London to push for an open data culture, and from Monday it will be hosting the second Data as Culture exhibition, which presents a more artistic take on questions surrounding the practicalities of open data. In doing so, it shows quite how the general public can (and probably really should) use data to inform their own lives and to engage with political issues.
All of the exhibits are based on freely available data, which is made lot more animated and accessible than numbers in a spreadsheet. “I made the decision straight away to move away from anything screen-based,” curator Shiri Shalmy told me as she gave me a tour, winding through office workers tapping away on keyboards. “Everything had to be physical.”…
James Bridle’s work on drone warfare touches a similar theme, though in this case the data are not hidden: his images of military UAVs come from Google Maps. “They’re there for anybody to look at, they’re kind of secret but available,” said Shalmy, who added that with the data out there, we can’t pretend we don’t know what’s going on. “They can do things in secret as long as we pretend it’s a secret.”
We’ve looked at Bridle’s work before, from his Dronestagram photos to his chalk outlines of drones, and he’s been commissioned to do something new for the Data as Culture show: Shalmy has asked him to compare the open data on military drones against that of London’s financial centre. He’ll present what he digs up in summer.

From the series ‘Watching the Watchers.’ Image: James Bridle/ODI

Using this kind of government data—from local council expenses to military movements—shows quite how much information is available and how it can be used to hold politicians to account. In essence, anyone can do surveillance to some level. While activists including Berners-Lee push for more data to be made accessible, it’s only useful if we actually bother to engage with it, and work like Bridle’s pose the uneasy suggestion that sometimes it’s more comfortable to remain ignorant.
And in addition to reading data, we can collect it. Rather than delving into government files, a knitted banner by artist Sam Meech uses publicly generated data to make a political point. The banner bears the phrase “8 hour labour,” a reference to the eight-hour workday movement that sprang up in Britain’s Industrial Revolution. The idea was that people would have eight hours work, eight hours rest, and eight hours recreation.

A detail from Sam Meechan’s Punchcard Economy. Image: Sam Meechan/ODI

But the black-and-white pattern in the banner is made up of much less regular working hours: those logged by self-employed creatives, who can take part by entering their own timesheet data via virtual punchcards. Shalmy pointed out her own schedule in a week when she was setting up the exhibition: a 70-hour block woven into the knit. It’s an example of how individuals can use data to make a political point—the work is reminiscent of trade union banners and seems particularly relevant at a time when controversial zero hours contracts are on the rise.
Also garnering data from the public, artist collective Thickear are asking people to fill in data forms on their arrival, which they’ll file on an old-fashioned spike. I took one of the forms, only to be confronted with nonsensical bureaucratic-type boxes. “The data itself is not informative in any way,” said Shalmy. It’s more about the idea of who we trust to give our data to. How often do we accept privacy policies without even giving ourselves the chance to even blink at the small print?…”

Crowdsourced transit app shows what time the bus will really come

Curated on March 26, 2014October 31, 2018 by Stefaan Verhulst

Springwise: “The problem with most transport apps is that they rely on fixed data from transport company schedules and don’t truly reflect exactly what’s going on with the city’s trains and buses at any given moment. Operating like a Waze for public transport, Israel’s Ototo app crowdsources real-time information from passengers to give users the best suggestions for their commute.
The app relies on a community of ‘Riders’, who allow anonymous location data to be sent from their smartphone whenever they’re using public transport. By collating this data together, Ototo offers more realistic information about bus and train routes. While a bus may be due in five minutes, a Rider currently on that bus might be located more than five minutes away, indicating that the bus isn’t on time. Ototo can then suggest a quicker route for users. According to Fast Company, the service currently has a 12,000-strong global Riders community that powers its travel recommendations. On top of this, the app is designed in an easy-to-use infographic format that quickly and efficiently tells users where they need to be going and how long it will take. The app is free to download from the App Store, and the video below offers a demonstration:

Ototo faces competition from similar services such as New York City’s Moovit, which also details how crowded buses are.”

Sinkhole of bureaucracy

Curated on March 26, 2014August 3, 2018 by Stefaan Verhulst

First article in a Washington Post series “examining the failures at the heart of troubled federal systems” by David A. Fahrenthold: “The trucks full of paperwork come every day, turning off a country road north of Pittsburgh and descending through a gateway into the earth. Underground, they stop at a metal door decorated with an American flag.

Behind the door, a room opens up as big as a supermarket, full of five-drawer file cabinets and people in business casual. About 230 feet below the surface, there is easy-listening music playing at somebody’s desk.
This is one of the weirdest workplaces in the U.S. government — both for where it is and for what it does.
Here, inside the caverns of an old Pennsylvania limestone mine, there are 600 employees of the Office of Personnel Management. Their task is nothing top-secret. It is to process the retirement papers of the government’s own workers.
But that system has a spectacular flaw. It still must be done entirely by hand, and almost entirely on paper.

The employees here pass thousands of case files from cavern to cavern and then key in retirees’ personal data, one line at a time. They work underground not for secrecy but for space. The old mine’s tunnels have room for more than 28,000 file cabinets of paper records.
This odd place is an example of how hard it is to get a time-wasting bug out of a big bureaucratic system.
Held up by all that paper, work in the mine runs as slowly now as it did in 1977….”
See also Data mining. The old-fashioned way: View the full graphic.

Exploration, Extraction and ‘Rawification’. The Shaping of Transparency in the Back Rooms of Open Data

Curated on March 26, 2014November 28, 2018 by Stefaan Verhulst

Paper by Denis, Jerome and Goëta, Samuel: “With the advent of open data initiatives, raw data has been staged as a crucial element of government transparency. If the consequences of such data-driven transparency have already been discussed, we still don’t know much about its back rooms. What does it mean for an administration to open its data? Following information infrastructure studies, this communication aims to question the modes of existence of raw data in administrations. Drawing on an ethnography of open government data projects in several French administrations, it shows that data are not ready-at-hand resources. Indeed, three kinds of operations are conducted that progressively instantiate open data. The first one is exploration. Where are, and what are, the data within the institution are tough questions, the response to which entails organizational and technical inquiries. The second one is extraction. Data are encapsulated in databases and its release implies a sometimes complex disarticulation process. The third kind of operations is ‘rawification’. It consists in a series of tasks that transforms what used to be indexical professional data into raw data. To become opened, data are (re)formatted, cleaned, ungrounded. Though largely invisible, these operations foreground specific ‘frictions’ that emerge during the sociotechnical shaping of transparency, even before data publication and reuses.”