The Living Library Index – inspired by the Harper’s Index – provides important statistics and highlights global trends in governance innovation. This installment focuses on the data universe and was originally published in 2013.
- How much data exists in the digital universe as of 2012: 2.7 zetabytes*
- Increase in the quantity of Internet data from 2005 to 2012: +1,696%
- Percent of the world’s data created in the last two years: 90
- Number of exabytes (=1 billion gigabytes) created every day in 2012: 2.5; that number doubles every month
- Percent of the digital universe in 2005 created by the U.S. and western Europe vs. emerging markets: 48 vs. 20
- Percent of the digital universe in 2012 created by emerging markets: 36
- Percent of the digital universe in 2020 predicted to be created by China alone: 21
- How much information in the digital universe is created and consumed by consumers (video, social media, photos, etc.) in 2012: 68%
- Percent of which enterprises have liability or responsibility for (copyright, privacy, compliance with regulations, etc.): 80
- Amount included in the Obama Administration’s 2-12 Big Data initiative: over $200 million
- Amount the Department of Defense is investing annually on Big Data projects as of 2012: over $250 million
- Data created per day in 2012: 2.5 quintillion bytes
- How many terabytes* of data collected by the U.S. Library of Congress as of April 2011: 235
- How many terabytes of data collected by Walmart per hour as of 2012: 2,560, or 2.5 petabytes*
- Projected growth in global data generated per year, as of 2011: 40%
- Number of IT jobs created globally by 2015 to support big data: 4.4 million (1.9 million in the U.S.)
- Potential shortage of data scientists in the U.S. alone predicted for 2018: 140,000-190,000, in addition to 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions
- Time needed to sequence the complete human genome (analyzing 3 billion base pairs) in 2003: ten years
- Time needed in 2013: one week
- The world’s annual effective capacity to exchange information through telecommunication networks in 1986, 2007, and (predicted) 2013: 281 petabytes, 65 exabytes, 667 exabytes
- Projected amount of digital information created annually that will either live in or pass through the cloud: 1/3
- Increase in data collection volume year-over-year in 2012: 400%
- Increase in number of individual data collectors from 2011 to 2012: nearly double (over 300 data collection parties in 2012)
*1 zetabyte = 1 billion terabytes | 1 petabyte = 1,000 terabytes | 1 terabyte = 1,000 gigabytes | 1 gigabyte = 1 billion bytes
- “Open Data Sites,” data.gov, accessed August 19, 2013.
- “Obama Administration Unveils “Big Data’ Initiative: Announces $200 Million in New R&D Investments,” Office of Science and Technology Policy, March 29, 2012.
- Joan Gantz and David Reinsel, “The Digital Universe Decade – Are you Ready?” International Data Corporation, iview, May 2010
- Andrew McAfee and Erik Brynjolfsson,“Big Data: The Management Revolution,” Harvard business Review, October 2012.
- “Big data: The next frontier for innovation, competition and productivity,” Mckinsey Global Institute, June 2011.
- “Big Data, Official statistics and Social Science Research,” The World Bank, December 12, 2012.
- “Data, data everywhere,” The Economist, February 25, 2010.
- Jimmy Daly, “18 Incredible Internet-Usage Statistics,” FedTech, June 12, 2013.
- Douglas Karr, “Infographic: Big Data Brings Marketing Big Numbers,” Marketing Tech Blog, May 9, 2012.
- “2012 Krux Cross-Industry Study,” Krux Research, June 12, 2012.
- “Gartner Says Big Data Creates Big Jobs,” Gartner, October 22, 2012.
- “Big Data Universe Beginning to Explode,” CSC, accessed August 19, 2013.
- “The Digital Universe in 2020,” IDC, December 2012
- “Big Data, for Better or Worse: 90 % of World’s Data Generated Over Last Two Years”, Science Faily, May 22, 2013.
New paper by Vint Cerf, Patrick Ryan and Max Senges Senges: “This essay looks at the the different roles that multistakeholder institutions play in the Internet governance ecosystem. We propose a model for thinking of Internet governance within the context of the Internet’s layered model. We use the example of the negotiations in Dubai in 2102 at the World Conference on International Telecommunications as an illustration for why it is important for different institutions within the governance system to focus on their respective areas of expertise (e.g., the ITU, ICANN, and IGF). Several areas of conflict (a “tussle”) are reviewed, such as the desire to promote more broadband infrastructure, a topic that is in the remit of the International Telecommunications Union, but also the recurring desire of countries like Russia and China to use the ITU to regulate content and restrict free expression on the Internet through onerous cybersecurity and spam provisions. We conclude that it is folly to try and regulate all these areas through an international treaty, and encourage further development of mechanisms for global debate like the Internet Governance Forum (IGF).”
“CLT allows many users to translate small pieces of legal texts between Chinese and English, to promote mutual understanding and provide a resource to legal professionals around the world. The sum of these small pieces, contributed in any order or no order at all, gradually creates a completed translation. You can translate as little or as much as you like, or leave comments to discuss the work of others or suggest better translations.
Quick Start: 1. open a post 2. Select Language you want to translate INTO 3. Open Tranlator Mode. 4. Translate…
We aim to create complete translations of important Chinese laws and articles, but some of the articles you view may still be incomplete or translated less than fluently. Bear with us, we are a new resource and getting there slowly. If you are able,, help us improve the translations you see!”
New paper in Development Policy Review: “Analysis of the impact and effectiveness of Freedom of Information (FOI) legislation has been hampered by lack of systematic evidence and conceptual confusion about what kind of right it represents. This article discusses some of the main conceptual parameters of FOI theory, before reviewing the available evidence from a range of studies. It presents case studies of civil-society activism on FOI in India and South Africa to illustrate the extent to which access to information is having an impact, in particular on socio-economic conditions. After reviewing the range of approaches used, it concludes that the academic community and the FOI community of practice need to come together to devise robust and rigorous methodologies.”
Tom Simonite in MIT Technology Review: “…as Facebook’s user base continues to expand, a growing proportion of its users think of it quite differently, as a luxury brand, badge of status, and or even a place to make a little extra money. That’s due to the rapid growth in the number of Facebook users signing on from developing countries, a trend underscored by news from the company today that more than 100 million people use a mobile app the company makes for feature phones…
Little research has been done on Facebook’s growth in developing countries (and a lot would be needed to capture even some of the diversity included under the blanket term “developing world”). Two small, recent studies of Kenyan Facebook users in poor areas by Susan Wyche of Michigan State University are among the first to be published, and they provide some interesting insights.
One of Wyche’s ethnographic studies took place in rural Internet cafes, where the researchers were told that “Facebook is a luxury,” only to be indulged if someone had money to spare (here’s a PDF of Wyche’s paper). When study participants thought about social networking, the challenges of low bandwidth and sometimes unreliable electricity supplies were foremost in their minds.
The barriers of cost and infrastructure associated with Facebook led people in another community Wyche and colleagues visited, a slum of Nairobi, to see the service as for more than just socializing. They used it—with mixed success—as a way to make a little money, look for jobs, market themselves, and seek remittances from friends and family overseas. (This reminded me of a recent report on people in Kuwait using Instagram to sell things and run retail businesses.)…
Should it want to, Facebook could even become a powerful tool for efforts to improve the lives of people in poor areas, where the site is gaining traction. The company has already dabbled with using social engineering to boost organ donations in the U.S. (see “Thank God for Facebook: When Platforms Proselytize”). There’s no shortage of similar experiments that could be run in places with more fundamental health problems, where Facebook’s status as a luxury could make it very influential.”
The Physics arXiv Blog “We’ve all experienced the sense of being familiar with somebody without knowing their name or even having spoken to them. These so-called “familiar strangers” are the people we see every day on the bus on the way to work, in the sandwich shop at lunchtime, or in the local restaurant or supermarket in the evening.
These people are the bedrock of society and a rich source of social potential as neighbours, friends, or even lovers.
But while many researchers have studied the network of intentional links between individuals—using mobile-phone records, for example—little work has been on these unintentional links, which form a kind of hidden social network.
Today, that changes thanks to the work of Lijun Sun at the Future Cities Laboratory in Singapore and a few pals who have analysed the passive interactions between 3 million residents on Singapore’s bus network (about 55 per cent of the city’s population). ”This is the first time that such a large network of encounters has been identied and analyzed,” they say.
The results are a fascinating insight into this hidden network of familiar strangers and the effects it has on people….
Perhaps the most interesting result involves the way this hidden network knits society together. Lijun and co say that the data hints that the connections between familiar strangers grows stronger over time. So seeing each other more often increases the chances that familiar strangers will become socially connected.
That’s a fascinating insight into the hidden social network in which we are all embedded. It’s important because it has implications for our understanding of the way things like epidemics can spread through cities.
Perhaps a more interesting is the insight it gives into how links form within communities and how these can strengthened. With the widespread adoption of smart cards on transport systems throughout the world, this kind of study can easily be repeated in many cities, which may help to tease apart some of the factors that make them so different.”
Ref: arxiv.org/abs/1301.5979: Understanding Metropolitan Patterns of Daily Encounters
Press Release: “Knight Foundation today named eight projects as winners of the Knight News Challenge on Open Gov, awarding the recipients more than $3.2 million for their ideas.
The projects will provide new tools and approaches to improve the way people and governments interact. They tackle a range of issues from making it easier to open a local business to creating a simulator that helps citizens visualize the impact of public policies on communities….
Each of the winning projects offers a solution to a real-world need. They include:
Civic Insight: Providing up-to-date information on vacant properties so that communities can find ways to make tangible improvements to local spaces;
OpenCounter: Making it easier for residents to register and create new businesses by building open source software that governments can use to simplify the process;
Open Gov for the Rest of Us: Providing residents in low-income neighborhoods in Chicago with the tools to access and demand better data around issues important to them, like housing and education;
Outline.com: Launching a public policy simulator that helps people visualize the impact that public policies like health care reform and school budget changes might have on local economies and communities;
Oyez Project: Making state and appellate court documents freely available and useful to journalists, scholars and the public, by providing straightforward summaries of decisions, free audio recordings and more;
Procur.io: Making government contract bidding more transparent by simplifying the way smaller companies bid on government work;
GitMachines: Supporting government innovation by creating tools and servers that meet government regulations, so that developers can easily build and adopt new technology;
Plan in a Box: Making it easier to discover information about local planning projects, by creating a tool that governments and contractors can use to easily create websites with updates that also allow public input into the process.
Now in its sixth year, the Knight News Challenge accelerates media innovation by funding breakthrough ideas in news and information. Winners receive a share of $5 million in funding and support from Knight’s network of influential peers and advisors to help advance their ideas. Past News Challenge winners have created a lasting impact. They include: DocumentCloud, which analyzes and annotates public documents – turning them into data; Tools for OpenStreetMap, which makes it easier to contribute to the editable map of the world; and Safecast, which helps people measure air quality and became the leading provider of pollution data following the 2011 earthquake and tsunami in Japan.
For more, visit newschallenge.org and follow #newschallenge on Twitter.
Martin Tisné, Director of Policy at Omidyar Network, in The Telegraph: “Trust in government has rarely been at a lower ebb. Citizens in developed and developing countries alike feel increasingly disconnected from the political process and their political leaders. They complain of having too little influence over decisions, too little access to government information and too little control over their own data.
In such an environment, suspicion and anger can erupt as we have seen across the world, most recently in Istanbul’s Taksim square.
At the same time, governments are operating in very challenging circumstances. They have to meet rising expectations from their citizens with, thanks to the impact of the global financial crisis, often severely reduced revenues. They also face a whole range of pressures which will make bridging this gap ever more difficult. There has never been a greater need for open and honest dialogue.
There is no single answer to these concerns. But it is clear that opening up government data must be a major element of the answer. Open data has enormous potential to drive economic growth and spread prosperity. It improves accountability, strengthens governance, builds trust and drives innovation in both the private sector and the delivery of key public services.
There are already many examples from around the world that these benefits are already being delivered. In the UK, Mastodon C
, a start-up incubated by the Open Data Institute
, used open data on prescriptions by GPs to show that the NHS could have saved over £200 million by prescribing generic drugs instead of their more expensive patented equivalents.
In India, the technology platform I Paid A Bribe enables citizens to publicly log whenever they have been shaken down for a bribe. In Mexico, Compara Tu Escuela (Check Your School) empowers parents by providing them directly with information on school performance.
We all benefit as citizens and consumers, as economies and societies, if we get this right. It is why the expected decision by the G8 countries to adopt an Open Data Charter at the G8 summit in Lough Erne is so important.”
Philip Howard in The Atlantic: “Everyone in Turkey can tell a story about how they turned on the TV hoping for news about current events, but found game shows, beauty pageants, and nature documentaries. Even Erdogan’s devotees know that the state-run news programs are grindingly uncritical. The pall of media control even has an impact on foreign broadcasters like CNN, which aired a penguin documentary within Turkey while its international broadcasters covered the clashes. Even when the country’s newspapers and broadcasters began reporting on the crisis, they spun the story as being violent and local to Istanbul. Another friend, who attended the protests on Saturday, said “you see misinformation on Twitter. But social media has played a corrective role to faults in the other available media.” On the days he joined in, he was part of peaceful demonstrations, and he found the Twitter streams telling stories about how the protests were country-wide and mostly nonviolent.
These days, Turks find themselves caught in the crossfire between highly politicized media organizations, so it is not surprising that when people want news they trust their own networks. The country has a dedicated community of startups designing apps, building games and generating content for the country’s rapidly growing population of internet and mobile phone users. Half of the country’s 75 million people are under 30. Half of Turkish citizens are online, and they are Facebook’s seventh largest national audience. Government ministers and strategists do have Twitter accounts, but they still tend to treat social media as a broadcast tool, a way of pushing their perspectives out to followers. Erdogan has a twitter account with more than 2.5 million followers, but recently opined that “This thing called social media is a curse on societies”….
This isn’t just happening in Turkey: In moments of political and military crisis, people want to control their media and connect with family and friends. And ruling elites respond by investing in broadcast media and censoring and surveilling digital networks. So the battles between political elites who use broadcast media and the activists who use digital media are raging in other parts of the world, as well.”
Atlantic Cities: “Colab, a Brazilian mobile application designed to encourage better citizenship, is the winner of the 2013 AppMyCity! Prize for the year’s best urban app.
The app’s five founders, Bruno Aracaty, Gustavo Maia, Paulo Pandolfi, Josemando Sobral and Vitor Guedes, from Recife and São Paulo, claimed the $5,000 prize last week at the annual New Cities Summit in São Paulo. Colab competed against two other finalists, BuzzJourney, from Kfar-Saba, Israel, and PublicStuff, from New York City. All three finalists presented their project to the international audience at the New Cities Summit. The audience then voted to determine the winner.
Colab utilizes photos and geolocation to connect citizens to cities based on three pillars of interaction: reporting daily urban issues; elaborating on and proposing new projects and solutions; and evaluating public services….
In total, the New Cities Foundation received 98 submissions for the AppMyCity! Prize 2013. A panel of judges chose the finalists out of ten semi-finalists, based on ability to create widespread impact and helpful user interface”