The Economist: “But then comes the challenge of generating real insight into forecasting accuracy. How can one compare forecasting ability?
The only reliable method is to conduct a forecasting tournament in which independent judges ask all participants to make the same forecasts in the same timeframes. And forecasts must be expressed numerically, so there can be no hiding behind vague verbiage. Words like “may” or “possible” can mean anything from probabilities as low as 0.001% to as high as 60% or 70%. But 80% always and only means 80%.
In the late 1980s one of us (Philip Tetlock) launched such a tournament. It involved 284 economists, political scientists, intelligence analysts and journalists and collected almost 28,000 predictions. The results were startling. The average expert did only slightly better than random guessing. Even more disconcerting, experts with the most inflated views of their own batting averages tended to attract the most media attention. Their more self-effacing colleagues, the ones we should be heeding, often don’t get on to our radar screens.
That project proved to be a pilot for a far more ambitious tournament currently sponsored by the Intelligence Advanced Research Projects Activity (IARPA), part of the American intelligence world. Over 5,000 forecasters have made more than 1m forecasts on more than 250 questions, from euro-zone exits to the Syrian civil war. Results are pouring in and they are revealing. We can discover who has better batting averages, not take it on faith; discover which methods of training promote accuracy, not just track the latest gurus and fads; and discover methods of distilling the wisdom of the crowd.
The big surprise has been the support for the unabashedly elitist “super-forecaster” hypothesis. The top 2% of forecasters in Year 1 showed that there is more than luck at play. If it were just luck, the “supers” would regress to the mean: yesterday’s champs would be today’s chumps. But they actually got better. When we randomly assigned “supers” into elite teams, they blew the lid off IARPA’s performance goals. They beat the unweighted average (wisdom-of-overall-crowd) by 65%; beat the best algorithms of four competitor institutions by 35-60%; and beat two prediction markets by 20-35%.
Over to you
To avoid slipping back to business as usual—believing we know things that we don’t—more tournaments in more fields are needed, and more forecasters. So we invite you, our readers, to join the 2014-15 round of the IARPA tournament. Current questions include: Will America and the EU reach a trade deal? Will Turkey get a new constitution? Will talks on North Korea’s nuclear programme resume? To volunteer, go to the tournament’s website at www.goodjudgmentproject.com. We predict with 80% confidence that at least 70% of you will enjoy it—and we are 90% confident that at least 50% of you will beat our dart-throwing chimps.”
See also https://web.archive.org/web/2013/http://www.iarpa.gov/Programs/ia/ACE/ace.html
Open Data Index provides first major assessment of state of open government data
Press Release from the Open Knowledge Foundation: “In the week of a major international summit on government transparency in London, the Open Knowledge Foundation has published its 2013 Open Data Index, showing that governments are still not providing enough information in an accessible form to their citizens and businesses.
The UK and US top the 2013 Index, which is a result of community-based surveys in 70 countries. They are followed by Denmark, Norway and the Netherlands. Of the countries assessed, Cyprus, St Kitts & Nevis, the British Virgin Islands, Kenya and Burkina Faso ranked lowest. There are many countries where the governments are less open but that were not assessed because of lack of openness or a sufficiently engaged civil society. This includes 30 countries who are members of the Open Government Partnership.
The Index ranks countries based on the availability and accessibility of information in ten key areas, including government spending, election results, transport timetables, and pollution levels, and reveals that whilst some good progress is being made, much remains to be done.
Rufus Pollock, Founder and CEO of the Open Knowledge Foundation said:
Opening up government data drives democracy, accountability and innovation. It enables citizens to know and exercise their rights, and it brings benefits across society: from transport, to education and health. There has been a welcome increase in support for open data from governments in the last few years, but this Index reveals that too much valuable information is still unavailable.
The UK and US are leaders on open government data but even they have room for improvement: the US for example does not provide a single consolidated and open register of corporations, while the UK Electoral Commission lets down the UK’s good overall performance by not allowing open reuse of UK election data.
There is a very disappointing degree of openness of company registers across the board: only 5 out of the 20 leading countries have even basic information available via a truly open licence, and only 10 allow any form of bulk download. This information is critical for range of reasons – including tackling tax evasion and other forms of financial crime and corruption.
Less than half of the key datasets in the top 20 countries are available to re-use as open data, showing that even the leading countries do not fully understand the importance of citizens and businesses being able to legally and technically use, reuse and redistribute data. This enables them to build and share commercial and non-commercial services.
To see the full results: https://index.okfn.org. For graphs of the data: https://index.okfn.org/visualisations.”
Information Now: Open Access and the Public Good
Podcast from SMARTech (Georgia Tech): “Every year, the international academic and research community dedicates a week in October to discuss, debate, and learn more about Open Access. Open Access in the academic sense refers to the free, immediate, and online access to the results of scholarly research, primarily academic, peer-reviewed journal articles. In the United States, the movement in support of Open Access has, in the last decade, been growing dramatically. Because of this growing interest in Open Access, a group of academic librarians from the Georgia Tech library, Wendy Hagenmaier (Digital Collections Archivist), Fred Rascoe (Scholarly Communication Librarian), and Lizzy Rolando (Research Data Librarian), got together to talk to folks in the thick of it, to try and unravel some of the different concerns and benefits of Open Access. But we didn’t just want to talk about Open Access for journal articles – we wanted to examine more broadly what it means to be “open”, what is open information, and what relationship open information has to the public good. In this podcast, we talk with different people who have seen and experienced open information and open access in practice. In the first act, Dan Cohen from the DPLA speaks about efforts to expand public access to archival and library collections. In the second, we’ll hear an argument from Christine George about why things sometimes need to be closed, if we want them to be open in the future. Third, Kari Watkins speaks about specific example of when a government agency decided, against legitimate concerns, to make transit data open, and why it worked for them. Fourth, Peter Suber from Harvard University will give us the background on the Open Access movement, some myths that have been dispelled, and why it is important for academic researchers to take the leap to make their research openly accessible. And finally, we’ll hear from Michael Chang, a researcher who did take that leap and helped start an Open Access journal, and why he sees openness in research as his obligation.”
See also Personal Guide to Open Access
Bright Spots of open government to be recognised at global summit
Press Release of the UK Cabinet Office: “The 7 shortlisted initiatives vying for the Bright Spots award show how governments in Open Government Partnership countries are working with citizens to sharpen governance, harness new technologies to increase public participation and improve government responsiveness.
At the Open Government Partnership summit in London on 31 October 2013 and 1 November 2013, participants will be able to vote for one of the shortlisted projects. The winning project – the Bright Spot – will be announced in the summit’s final plenary session….
The shortlisted entries for the Bright Spots prize – which will be awarded at the London summit – are:
- Chile – ChileAtiende
The aim of ChileAtiende has been to simplify government to citizens by providing a one-stop shop for accessing public services. Today, ChileAtiende has more than 190 offices across the whole country, a national call centre and a digital platform, through which citizens can access multiple services and benefits without having to navigate multiple government offices.
- Estonia – People’s Assembly
The People’s Assembly is a deliberative democracy tool, designed to encourage input from citizens on the government’s legislative agenda. This web-based platform allows ordinary citizens to propose policy solutions to problems including fighting corruption. Within 3 weeks, 1,800 registered users posted nearly 6,000 ideas and comments. Parliament has since set a timetable for the most popular proposals to be introduced in the formal proceedings.
- Georgia – improvements to the Freedom of Information Act
Civil society organisations in Georgia have successfully used the government’s participation in OGP to advocate improvements to the country’s Freedom of Information legislation. Government agencies are now obliged to proactively publish information in a way that is accessible to anyone, and to establish an electronic request system for information.
- Indonesia – complaints portal
LAPOR! (meaning “to report” in Indonesian) is a social media channel where Indonesian citizens can submit complaints and enquiries about development programmes and public services. Comments are transferred directly to relevant ministries or government agencies, which can respond via the website. LAPOR! now has more than 225,350 registered users and receives an average of 1,435 inputs per day.
- Montenegro – Be Responsible app
“Be Responsible” is a mobile app that allows citizens to report local problems – from illegal waste dumps, misuse of official vehicles and irregular parking, to failure to comply with tax regulations and issues over access to healthcare and education.
- Philippines – citizen audits
The Citizen Participatory Audit (CPA) project is exploring ways in which citizens can be directly engaged in the audit process for government projects and contribute to ensuring greater efficiency and effectiveness in the use of public resources. 4 pilot audits are in progress, covering public works, welfare, environment and education projects.
- Romania – transparency in public sector recruitment
The PublicJob.ro website was set up to counter corruption and lack of transparency in civil service recruitment. PublicJob.ro takes recruitment data from public organisations and e-mails it to more than 20,000 subscribers in a weekly newsletter. As a result, it has become more difficult to manipulate the recruitment process.”
Where in the World are Young People Using the Internet?
Georgia Tech: “According to a common myth, today’s young people are all glued to the Internet. But in fact, only 30 percent of the world’s youth population between the ages of 15 and 24 years old has been active online for at least five years. In South Korea, 99.6 percent of young people are active, the highest percentage in the world. The least? The Asian island of Timor Leste with less than 1 percent.
Digital natives as a percentage of total population, 2012 (Courtesy: ITU)
Those are among the many findings in a study from the Georgia Institute of Technology and International Telecommunication Union (ITU). The study is the first attempt to measure, by country, the world’s “digital natives.” The term is typically used to categorize young people born around the time the personal computer was introduced and have spent their lives connected with technology.
Nearly 96 percent of American millennials are digital natives. That figure is behind Japan (99.5 percent) and several European countries, including Finland, Denmark and the Netherlands.
But the percentage that Georgia Tech Associate Professor Michael Best thinks is the most important is the number of digital natives as compared to a country’s total population….
The countries with the highest proportion of digital natives among their population are mostly rich nations, which have high levels of overall Internet penetration. Iceland is at the top of the list with 13.9 percent. The United States is sixth (13.1 percent). A big surprise is Malaysia, a middle-income country with one of the highest proportions of digital natives (ranked 4th at 13.4 percent). Malaysia has a strong history of investing in educational technology.
The countries with the smallest estimated proportion of digital natives are Timor-Leste, Myanmar and Sierra Leone. The bottom 10 consists entirely of African or Asian nations, many of which are suffering from conflict and/or have very low Internet availability.”
Open data for accountable governance: Is data literacy the key to citizen engagement?
Camilla Monckton at UNDP’s Voices of Eurasia blog: “How can technology connect citizens with governments, and how can we foster, harness, and sustain the citizen engagement that is so essential to anti-corruption efforts?
UNDP has worked on a number of projects that use technology to make it easier for citizens to report corruption to authorities:
- Serbia’s SMS corruption reporting in the health sector
- Montenegro’s ‘be responsible app’
- Kosovo’s online corruption reporting site kallxo.com
These projects are showing some promising results, and provide insights into how a more participatory, interactive government could develop.
At the heart of the projects is the ability to use citizen generated data to identify and report problems for governments to address….
Wanted: Citizen experts
As Kenneth Cukier, The Economist’s Data Editor, has discussed, data literacy will become the new computer literacy. Big data is still nascent and it is impossible to predict exactly how it will affect society as a whole. What we do know is that it is here to stay and data literacy will be integral to our lives.
It is essential that we understand how to interact with big data and the possibilities it holds.
Data literacy needs to be integrated into the education system. Educating non-experts to analyze data is critical to enabling broad participation in this new data age.
As technology advances, key government functions become automated, and government data sharing increases, newer ways for citizens to engage will multiply.
Technology changes rapidly, but the human mind and societal habits cannot. After years of closed government and bureaucratic inefficiency, adaptation of a new approach to governance will take time and education.
We need to bring up a generation that sees being involved in government decisions as normal, and that views participatory government as a right, not an ‘innovative’ service extended by governments.
What now?
In the meantime, while data literacy lies in the hands of a few, we must continue to connect those who have the technological skills with citizen experts seeking to change their communities for the better – as has been done in many a Social Innovation Camps recently (in Montenegro, Ukraine and Armenia at Mardamej and Mardamej Relaoded and across the region at Hurilab).
The social innovation camp and hackathon models are an increasingly debated topic (covered by Susannah Vila, David Eaves, Alex Howard and Clay Johnson).
On the whole, evaluations are leading to newer models that focus on greater integration of mentorship to increase sustainability – which I readily support. However, I do have one comment:
Social innovation camps are often criticized for a lack of sustainability – a claim based on the limited number of apps that go beyond the prototype phase. I find a certain sense of irony in this, for isn’t this what innovation is about: Opening oneself up to the risk of failure in the hope of striking something great?
In the words of Vinod Khosla:
“No failure means no risk, which means nothing new.”
As more data is released, the opportunity for new apps and new ways for citizen interaction will multiply and, who knows, someone might come along and transform government just as TripAdvisor transformed the travel industry.”
Facebook Is Being Redefined by Its Developing World Users
Tom Simonite in MIT Technology Review: “…as Facebook’s user base continues to expand, a growing proportion of its users think of it quite differently, as a luxury brand, badge of status, and or even a place to make a little extra money. That’s due to the rapid growth in the number of Facebook users signing on from developing countries, a trend underscored by news from the company today that more than 100 million people use a mobile app the company makes for feature phones…
Little research has been done on Facebook’s growth in developing countries (and a lot would be needed to capture even some of the diversity included under the blanket term “developing world”). Two small, recent studies of Kenyan Facebook users in poor areas by Susan Wyche of Michigan State University are among the first to be published, and they provide some interesting insights.
One of Wyche’s ethnographic studies took place in rural Internet cafes, where the researchers were told that “Facebook is a luxury,” only to be indulged if someone had money to spare (here’s a PDF of Wyche’s paper). When study participants thought about social networking, the challenges of low bandwidth and sometimes unreliable electricity supplies were foremost in their minds.
The barriers of cost and infrastructure associated with Facebook led people in another community Wyche and colleagues visited, a slum of Nairobi, to see the service as for more than just socializing. They used it—with mixed success—as a way to make a little money, look for jobs, market themselves, and seek remittances from friends and family overseas. (This reminded me of a recent report on people in Kuwait using Instagram to sell things and run retail businesses.)…
Should it want to, Facebook could even become a powerful tool for efforts to improve the lives of people in poor areas, where the site is gaining traction. The company has already dabbled with using social engineering to boost organ donations in the U.S. (see “Thank God for Facebook: When Platforms Proselytize”). There’s no shortage of similar experiments that could be run in places with more fundamental health problems, where Facebook’s status as a luxury could make it very influential.”
Why Governments Use Broadcast TV and Dissidents Use Twitter
Philip Howard in The Atlantic: “Everyone in Turkey can tell a story about how they turned on the TV hoping for news about current events, but found game shows, beauty pageants, and nature documentaries. Even Erdogan’s devotees know that the state-run news programs are grindingly uncritical. The pall of media control even has an impact on foreign broadcasters like CNN, which aired a penguin documentary within Turkey while its international broadcasters covered the clashes. Even when the country’s newspapers and broadcasters began reporting on the crisis, they spun the story as being violent and local to Istanbul. Another friend, who attended the protests on Saturday, said “you see misinformation on Twitter. But social media has played a corrective role to faults in the other available media.” On the days he joined in, he was part of peaceful demonstrations, and he found the Twitter streams telling stories about how the protests were country-wide and mostly nonviolent.
These days, Turks find themselves caught in the crossfire between highly politicized media organizations, so it is not surprising that when people want news they trust their own networks. The country has a dedicated community of startups designing apps, building games and generating content for the country’s rapidly growing population of internet and mobile phone users. Half of the country’s 75 million people are under 30. Half of Turkish citizens are online, and they are Facebook’s seventh largest national audience. Government ministers and strategists do have Twitter accounts, but they still tend to treat social media as a broadcast tool, a way of pushing their perspectives out to followers. Erdogan has a twitter account with more than 2.5 million followers, but recently opined that “This thing called social media is a curse on societies”….
This isn’t just happening in Turkey: In moments of political and military crisis, people want to control their media and connect with family and friends. And ruling elites respond by investing in broadcast media and censoring and surveilling digital networks. So the battles between political elites who use broadcast media and the activists who use digital media are raging in other parts of the world, as well.”
Colab: Winner of the 2013 AppMyCity! Prize
Atlantic Cities: “Colab, a Brazilian mobile application designed to encourage better citizenship, is the winner of the 2013 AppMyCity! Prize for the year’s best urban app.
The app’s five founders, Bruno Aracaty, Gustavo Maia, Paulo Pandolfi, Josemando Sobral and Vitor Guedes, from Recife and São Paulo, claimed the $5,000 prize last week at the annual New Cities Summit in São Paulo. Colab competed against two other finalists, BuzzJourney, from Kfar-Saba, Israel, and PublicStuff, from New York City. All three finalists presented their project to the international audience at the New Cities Summit. The audience then voted to determine the winner.
Colab utilizes photos and geolocation to connect citizens to cities based on three pillars of interaction: reporting daily urban issues; elaborating on and proposing new projects and solutions; and evaluating public services….
In total, the New Cities Foundation received 98 submissions for the AppMyCity! Prize 2013. A panel of judges chose the finalists out of ten semi-finalists, based on ability to create widespread impact and helpful user interface”
How to Clean Up Social News
David Talbot in MIT Technology Review: ” New platforms for fact-checking and reputation scoring aim to better channel social media’s power in the wake of a disaster…Researchers from the Masdar Institute of Technology and the Qatar Computer Research Institute plan to launch Verily, a platform that aims to verify social media information, in a beta version this summer. Verily aims to enlist people in collecting and analyzing evidence to confirm or debunk reports. As an incentive, it will award reputation points—or dings—to its contributors.
Verily will join services like Storyful that use various manual and technical means to fact-check viral information, and apps such as Swift River that, among other things, let people set up filters on social media to provide more weight to trusted users in the torrent of posts following major events…Reputation scoring has worked well for e-commerce sites like eBay and Amazon and could help to clean up social media reports in some situations.