Measuring Internet Activity: a (Selective) Review of Methods and Metrics


Paper by Faris, Robert and Heacock, Rebekah (part of the Internet Monitor series): “Two Decades after the birth of the World Wide Web, more than two billion people around the world are Internet users. The digital landscape is littered with hints that the affordances of digital communications are being leveraged to transform life in profound and important ways. The reach and influence of digitally mediated activity grow by the day and touch upon all aspects of life, from health, education, and commerce to religion and governance. This trend demands that we seek answers to the biggest questions about how digitally mediated communication changes society and the role of different policies in helping or hindering the beneficial aspects of these changes. Yet despite the profusion of data the digital age has brought upon us—we now have access to a flood of information about the movements, relationships, purchasing decisions, interests, and intimate thoughts of people around the world—the distance between the great questions of the digital age and our understanding of the impact of digital communications on society remains large. A number of ongoing policy questions have emerged that beg for better empirical data and analyses upon which to base wider and more insightful perspectives on the mechanics of social, economic, and political life online. This paper seeks to describe the conceptual and practical impediments to measuring and understanding digital activity and highlights a sample of the many efforts to fill the gap between our incomplete understanding of digital life and the formidable policy questions related to developing a vibrant and healthy Internet that serves the public interest and contributes to human wellbeing. Our primary focus is on efforts to measure Internet activity, as we believe obtaining robust, accurate data is a necessary and valuable first step that will lead us closer to answering the vitally important questions of the digital realm. Even this step is challenging: the Internet is difficult to measure and monitor, and there is no simple aggregate measure of Internet activity—no GDP, no HDI. In the following section we present a framework for assessing efforts to document digital activity. The next three sections offer a summary and description of many of the ongoing projects that document digital activity, with two final sections devoted to discussion and conclusions.”

Crisis response needs to be a science, not an art


Jimmy Whitworth in the Financial Times:”…It is an imperative to offer shelter, nutrition, sanitation and medical care to those suddenly bereft of it. Without aid, humanitarian crises would cause still greater suffering. Yet admiration for the agencies that deliver relief should not blind us to the need to ensure that it is well delivered. Humanitarian responses must be founded on good evidence.
The evidence base, unfortunately, is weak. We know that storms, earthquakes and conflicts have devastating consequences for health and wellbeing, and that not responding is not an option, but we know surprisingly little about how best to go about it. Not only is evidence-based practice rare in humanitarian relief operations, it is often impossible.
Questions about how best to deliver clean water or adequate shelter, and even about which health needs should be prioritised as the most pressing, have often been barely researched. Indeed, the evidence gap is so great that the Humanitarian Practice Network has highlighted a “dire lack of credible data to help us understand just how much populations in crisis suffer, and to what extent relief operations are able to relieve that suffering”. No wonder aid responses are often characterised as messy.
Good practice often rests on past practice rather than research. The Bible of humanitarian relief is a document called the Sphere handbook, an important initiative to set minimum standards for provision of health, nutrition, sanitation and shelter. Yet analysis of the 2004 handbook has revealed that just 13 per cent of its 346 standards were supported by good evidence of relevance to health. The handbook, for example, recommended that refugee camps should prioritise measles vaccination – a worthwhile goal, but not one that should clearly be favoured over control of other infectious diseases.

Also under-researched is the question of how best to provide types of relief that everybody agrees meet essential needs. Access to clean water is a clear priority for almost all populations in crisis but little is understood about how this is most efficiently delivered. Is it best to ship bottled water to stricken areas? Are tankers of clean water more effective? Or can water purification tablets do the job? The summer floods in northern India made it clear that there is little good evidence one way or another.

Adequate shelter, too, is a human essential in all but the most benign environments but, once again, the evidence base about how best to provide it is limited. There is a school of thought that building transitional shelter from locally available materials is better in the long run than housing people under tents, tarpaulins and plastic, which if accurate would have far-reaching consequences for standard practice. But too little research has been done…
Researchers also face significant challenges to building a better evidence base. They can struggle to secure access to disaster zones when getting relief in is the priority. The timescales involved in applying for funding and ethical approval, too, make it difficult for them to move quickly enough to set up a study in the critical post-disaster period.
It is to address this that Enhancing Learning and Research for Humanitarian Assistance, with the support of the Wellcome Trust and the UK Department for International Development, recently launched an £8m research programme that investigates these issues.”

White House Unveils Big Data Projects, Round Two


Information Week: “The White House Office of Science and Technology Policy (OSTP) and Networking and Information Technology R&D program (NITRD) on Tuesday introduced a slew of new big-data collaboration projects aimed at stimulating private-sector interest in federal data. The initiatives, announced at the White House-sponsored “Data to Knowledge to Action” event, are targeted at fields as varied as medical research, geointelligence, economics, and linguistics.
The new projects are a continuation of the Obama Administration’s Big Data Initiative, announced in March 2012, when the first round of big-data projects was presented.
Thomas Kalil, OSTP’s deputy director for technology and innovation, said that “dozens of new partnerships — more than 90 organizations,” are pursuing these new collaborative projects, including many of the best-known American technology, pharmaceutical, and research companies.
Among the initiatives, Amazon Web Services (AWS) and NASA have set up the NASA Earth eXchange, or NEX, a collaborative network to provide space-based data about our planet to researchers in Earth science. AWS will host much of NASA’s Earth-observation data as an AWS Public Data Set, making it possible, for instance, to crowdsource research projects.
An estimated 4.4 million jobs are being created between now and 2015 to support big-data projects. Employers, educational institutions, and government agencies are working to build the educational infrastructure to provide students with the skills they need to fill those jobs.
To help train new workers, IBM, for instance, has created a new assessment tool that gives university students feedback on their readiness for number-crunching careers in both the public and private sector. Eight universities that have a big data and analytics curriculum — Fordham, George Washington, Illinois Institute of Technology, University of Massachusetts-Boston, Northwestern, Ohio State, Southern Methodist, and the University of Virginia — will receive the assessment tool.
OSTP is organizing an initiative to create a “weather service” for pandemics, Kalil said, a way to use big data to identify and predict pandemics as early as possible in order to plan and prepare for — and hopefully mitigate — their effects.
The National Institutes of Health (NIH), meanwhile, is undertaking its ” Big Data to Knowledge” (BD2K) initiative to develop a range of standards, tools, software, and other approaches to make use of massive amounts of data being generated by the health and medical research community….”
See also:
November 12, 2013 – Fact Sheet: Progress by Federal Agencies: Data to Knowledge to Action
November 12, 2013 – Fact Sheet: New Announcements: Data to Knowledge to Action
November 12, 2013 – Press Release: Data to Knowledge to Action Event

13 ways to unlock the potential of open government


The Guardian: “Nine experts offer their thoughts on making open data initiatives work for all citizens…
Tiago Peixoto, open government specialist, The World Bank, Washington DC, US. @participatory
Open data is an enabler – not a guarantee – of good participation: Participation implies creating legitimate channels of communication between citizens and governments, and opening up data does not create that channel. We need to consider which structures enable us to know about citizens’ needs and preferences.
Both governments and civil society are responsible for connecting governments to the people: If we assume institutional or regulatory reforms are needed, then clearly governments (at both the legislative and executive level) should take a big part of the responsibility. After that, it is civil society’s role (and individual citizens) to further promote and strengthen those institutions….
Ben Taylor, open data consultant, Twaweza, UK and Tanzania. @mtega
We need to put people before data: The OGP Summit raised some interesting questions on open data and open government in developing countries. In a particular session discussing how to harness data to drive citizens engagement, the consensus was that this was the wrong way around. It should instead be reversed, putting the real, everyday needs of citizens first, and then asking how can we use data to help meet these.
Open government is not all about technology: Often people assume that open government means technology, but I think that’s wrong. For me, open government is a simple idea: it’s about making the nuts and bolts of how government works visible to citizens. Even open data isn’t always just about technology, for example postings on noticeboards and in newspapers are also valuable. Technology has a lot to offer, but it has limitations as well…
Juan M Casanueva, director, SocialTIC, Mexico City, Mexico. @jm_casanueva
Closed working cultures stifle open government initiatives: It is interesting to think about why governments struggle to open up. While closed systems tend to foster corruption and other perverse practices, most government officials also follow a pre-established closed culture that has become ingrained in their working practices. There are sometimes few incentives and high risks for government officials that want to make career in the public service and some also lack capacities to handle technology and citizen involvement. It is very interesting to see government officials that overcome these challenges actually benefiting politically for doing innovative citizen-centered actions. Unfortunately, that is too much of a risk at higher levels of government.
NGOs in Mexico are leading the way with access to information and citizen involvement: Sonora Ciudana recently opened the state’s health payroll and approached the public staff so that they could compare what they earn with the state expense reports. Pacto por Juarez has created grassroots transparency and accountability schools and even have a bus tour that goes around the city explaining the city’s budget and how it is being spent….”

The trouble with democracy


author of The Confidence Trap: A History of Democracy in Crisis from World War I to the Present in The Guardian: “Government shutdowns, petty policy squabbles, voter disaffection – democracy doesn’t seem to work very well. But what’s the alternative? And can we rely on muddling through?…Those of us who live in the western democracies might sometimes be tempted to agree. Dictator envy is a habitual feature of democratic politics. We don’t actually want to live under a dictatorship – we still have a horror of what that would entail – but we do envy dictators their ability to act decisively in a crisis….
The irony of dictator envy is that it goes against the historical evidence. Over the last 100 years, democracies have shown that they are better than dictatorships at dealing with the most serious crises that any political system has to face. Democracies win wars. They survive economic disasters. They adapt to meet environmental challenges. Precisely because they are able to act decisively without having to square public opinion first, dictators are the ones who end up making the catastrophic mistakes. When dictators get things wrong, they can take the whole state over the cliff with them. When democratic leaders get things wrong, we kick them out before they can do terminal damage.
Yet that is little consolation in the middle of a crisis. The reason we keep succumbing to dictator envy is that it requires steady nerves to take the long view when things are going wrong. The qualities that give democracies the advantage in the long run – their restlessness and impatience with failure – are the same qualities that make it hard for them to take the long view. They look with envy on political systems that can seize the moment. Democracies are very bad at seizing the moment. Their survival technique is muddling through. The curse of democracy is that we are condemned to want the thing we can’t have.
The person who first noticed this deeply conflicted character of democratic life was a French aristocrat. When he travelled to the US to study its prisons in 1831, Alexis de Tocqueville shared the common 19th-century prejudice against democracy. He thought it was a chaotic and stupid system of government. By the time he finished his journey a year later, he had changed his mind. He decided that American democracy was a lot better than it looks. On the surface, everything appeared a mess: bickering politicians, vituperative and ill-informed newspapers (“The job of the journalist in America”, Tocqueville wrote, “is to attack coarsely, without preparation and without art, to set aside principles in order to grab men”), distracted citizens. No one was able to exert a grip. There was far too much noise, not enough signal. But over time this surfeit of noise produced an adaptable politics that never sat still for long enough to get stuck. The raucousness of American politics was a sign of its essential health. Americans kept stumbling into holes and then back out of them. More mistakes are made in a democracy, Tocqueville wrote, but more mistakes are corrected as well. More fires get started by Americans. More fires get put out by them too….
It has always been like this. The history of democracy throughout the 20th century is a story of repeated crises during which politicians and publics have been torn between the twin impulses to overreact and to underreact to the dangers, without ever finding the balance between them. Dictator envy is never far from the surface….The pattern of democratic life is to drift into impending disaster and then to stumble out of it. Undemocratic practices creep up on us unawares, until the routine practices of democracy – a free press, a few unbiddable politicians – expose them. When that happens, democracies do not get a grip; they simply make the minimum of necessary adjustments until they drift into the next disaster. What is hard for any democracy is to exert the constant, vigilant pressure needed to rein in the forces that produce the crises. It is so much easier to wait for the crisis to reveal itself before trying to do something about it. The new information technology, far from solving this problem, has made it worse. We are more distracted than ever. The surfeit of information flowing around the world makes it practically impossible for anyone to keep secrets for long. But it also makes it practically impossible to secure broad democratic agreement for wide-ranging reform of public life. There is far too much noise, not enough signal. So we keep our fingers crossed in the hope we will muddle through.”

Behavioural Public Policy


New book by Adam Oliver (Cambridge University Press): “How can individuals best be encouraged to take more responsibility for their well-being and their environment or to behave more ethically in their business transactions? Across the world, governments are showing a growing interest in using behavioural economic research to inform the design of nudges which, some suggest, might encourage citizens to adopt beneficial patterns of behaviour. In this fascinating collection, leading academic economists, psychologists and philosophers reflect on how behavioural economic findings can be used to help inform the design of policy initiatives in the areas of health, education, the environment, personal finances and worker remuneration. Each chapter is accompanied by a shorter ‘response’ that provides critical commentary and an alternative perspective. This accessible book will interest academic researchers, graduate students and policy-makers across a range of disciplinary perspectives.”

Index: Trust in Institutions


The Living Library Index – inspired by the Harper’s Index – provides important statistics and highlights global trends in governance innovation. This installment focuses on trust in institutions and was originally published in 2013.

Trust in Government

  • How many of the global public feel that their governments listen to them: 17%
  • How much of the global population trusts in institutions: almost half 
  • The number of Americans who trust institutions: less than half
  • How many people globally believe that business leaders and government officials will tell the truth when confronted with a difficult issue: Less than one-fifth
  • The average level of confidence amongst citizens in 25 OECD countries:
    • In national government: 40%, down from 45% in 2007
    • In financial institutions: 43%
    • In public services such as local police and healthcare: 72% and 71% respectively

Executive Government

  • How many Americans trust the government in Washington to do what is right “just about always or most of the time” in September 2013: 19%
  • Those who trust the “men and women … who either hold or are running for public office”: 46%
  • Number of Americans who express a great deal or fair amount of trust in:
    • Local government: 71%
    • State government: 62%
    • Federal government: 52%
  • How many Americans trust in the ability of “the American people” to make judgments about political issues facing the country:  61%, declining every year since 2009
  • Those who have trust and confidence in the federal government’s ability to handle international problems: 49%
  • Number of Americans who feel “angry” at the federal government: 3 in 10, all-time high since first surveyed in 1997

Congress

  • Percentage of Americans who say “the political system can work fine, it’s the members of Congress that are the problem” in October 2013: 58%
  • Following the government shutdown, number of Americans who stated that Congress would work better if nearly every member was replaced next year: nearly half
  • Those who think that even an entire overhaul of Congress would not make much difference: 4 in 10 
  • Those who think that “most members of Congress have good intentions, it’s the political system that is broken” in October 2013: 32%

Trust in Media

  • Global trust in media (traditional, social, hybrid, owned, online search): 57% and rising
  • The percentage of Americans who say they have “a great deal or fair amount of trust and confidence in the mass media”: 44% – the lowest level since first surveyed in 1997
  • How many Americans see the mass media as too liberal: 46%
    • As too conservative: 13%
    • As “just about right”: 37%
  • The number of Americans who see the press as fulfilling the role of political watchdog and believe press criticism of political leaders keeps them from doing things that should not be done: 68%
  • The proportion of Americans who have “only a little/not at all” level of trust in Facebook to protect privacy and personal information: three in four
    • In Google: 68%
    • In their cell phone provider: 63%

Trust in Industry

  • Global trust in business: 58%
  • How much of the global public trusts financial institutions: 50%
  • Proportion of the global public who consider themselves informed about the banking scandals: more than half
  • Of those, how many Americans report they now trust banks less: almost half
  • Number of respondents globally who say they trust tech companies to do what’s right: 77%, most trusted industry
  • Number of consumers across eight markets who were “confident” or “somewhat confident” that the tech sector can provide long-term solutions to meet the world’s toughest challenges: 76%

Sources

New Report: Federal Ideation Program: Challenges and Best Practices


New Report by Professor Gwanhoo Lee for the IBM Center for The Business of Government: “Ideation is the process of generating new ideas or solutions using crowdsourcing technologies, and it is changing the way federal government agencies innovate and solve problems. Ideation tools use online brainstorming or social voting platforms to submit new ideas, search previously submitted ideas, post questions and challenges, discuss and expand on ideas, vote them up or down and flag them.
This report examines the current status, challenges, and best practices of federal internal ide­ation programs made available exclusively to employees. Initial experiences from a variety of agencies show that these ideation tools hold great promise in engaging employees and stake­holders in problem-solving.
While ideation programs offer promising benefits, making innovation an aspect of everyone’s job is very hard to achieve. Given that these ideation tools and programs are still relatively new, agencies have not yet figured out the best practices and often do not know what to expect during the implementation process. This report seeks to fill this gap.
Based on field research and a literature review, the report describes four federal internal ideation programs, including IdeaHub (Department of Transportation), the Sounding Board (the Department of State), IdeaFactory (Department of Homeland Security), and CDC IdeaLab (Centers for Disease Control and Prevention, Department of Health and Human Services).
Four important challenges are associated with the adoption and implementation of federal internal ideation programs. These are: managing the ideation process and technology; managing cultural change; managing privacy, security and transparency; and managing use of the ideation tool.
Federal government agencies have been moving in the right direction by embracing these tools and launching ideation programs in boosting employee-driven innovation. However, many daunting challenges and issues remain to be addressed. For a federal agency to sustain its internal ideation program, it should note the following:
Recommendation One: Treat the ideation program not as a management fad but as a vehicle to reinvent the agency.
Recommendation Two: Institutionalize the ideation program.
Recommendation Three: Make the ideation team a permanent organizational unit.
Recommendation Four: Document ideas that are implemented.Quantify their impact and demonstrate the return on investment.Share the return with the employees through meaningful rewards.
Recommendation Five: Assimilate and integrate the ideation program into the mission-critical administrative processes.
Recommendation Six: Develop an easy-to-use mobile app for the ideation system.
Recommendation Seven: Keep learning from other agencies and even from commercial organizations.”

Why Research is Key to mySociety’s Future


Paul Lenz – Head of Finance and International Projects, mySociety: “mySociety operates in a field that we term the Civic Power sector. This sector includes a wide range of organizations, including non-profits like Ushahidi, The Sunlight Foundation, Avaaz and other companies like Change and Nationbuilder. There are many differences between these organizations, but they do share one thing in common: in the context of the wider civil society & development world in which we are situated, they are very young indeed.  mySociety, celebrating it’s tenth birthday this year, is one of the oldest in this sector –  but we are a spring chicken compared to the likes of Oxfam, Amnesty International and Plan
Theory of change
Our underlying philosophy – our theory of change – is that enabling (and encouraging) politically inexperienced people to take actions like reporting broken street lights or asking for government information will make people more aware of their own power to get things changed, and that will benefit both them and the communities they live in. But just because lots of people perform these actions doesn’t mean we have affected those users in any profound way.
As we have matured we have started to ask ourselves some tough questions, including:
– Does the use of our sites and services (and those of our partners) make people more powerful in the civic and democratic aspects of their lives?
– Does this power genuinely deliver tangible beneficial impacts (particularly in the face of potentially unresponsive or corrupt governments)?
– Do our tools risk increasing the power of the relatively richer, better educated and technically adept minority at the expense of the majority?

Theoretical challenges 
One of the challenges we face is that within our field there is not an easy or categorical connection between action and impact.  If you immunize a child against disease, then you can be certain that the child has a materially higher chance of remaining healthy.  There are of course wider discussions around whether immunization should be carried out by foreign NGOs or whether governments should work to improve their own health provisioning, but there is no doubt that the immunization itself is a good thing.
What about writing to a politician?  Is that a good thing?  We believe that it is.  We believe that it drives engagement and accountability and strengthens democracy.  But we can’t prove it, and we might be wrong. We must find out.
We have a great deal of data – page impressions, unique visitors, Freedom of Information requests raised, international re-uses of our code bases, messages sent to politicians, etc. – but no way of linking this to true impact.  In order to address this gap we will conduct methodologically rigorous, experimentally-driven research on both UK and international deployments of our technologies. We will then use the findings and the method we develop to encourage increased rigor in impact assessment by other organizations working in the Civic Power sector.
It is quite likely that some of these outcomes will be challenging for us, potentially suggesting that some of our workstreams have little or no true impact as things currently stand.  Nonetheless, we are committed to sharing the all of the results – good and ill – as they start to come through.”

Findings from the emerging field of Transparency Research


Tiago Peixoto: “HEC Paris has just hosted the 3rd Global Conference on Transparency Research, and they have made the list of accepted papers available. …
As one goes through the papers,  it is clear that unlike most of the open government space, when it comes to research, transparency is treated less as a matter of technology and formats and more as a matter of social and political institutions.  And that is a good thing.”
This year’s papers are listed below: