AU: Govt finds one third of open data was "junk"


IT News: “The number of datasets available on the Government’s open data website has slimmed by more than half after the agency discovered one third of the datasets were junk.
Since its official launch in 2011 data.gov.au grew to hold 1200 datasets from government agencies for public consumption.
In July this year the Deaprtment of Finance migrated the portal to a new open source platform – the Open Knowledge Foundation CKAN platform – for greater ease of use and publishing ability.
Since July the number of datasets fell from 1200 to 500.
Australian Government CTO John Sheridan said in his blog late yesterday the agency had needed to review the 1200 datasets as a result of the CKAN migration, and discovered a significant amount of them were junk.
“We unfortunately found that a third of the “datasets” were just links to webpages or files that either didn’t exist anymore, or redirected somewhere not useful to genuine seekers of data,” Sheridan said.
“In the second instance, the original 1200 number included each individual file. On the new platform, a dataset may have multiple files. In one case we have a dataset with 200 individual files where before it was counted as 200 datasets.”
The number of datasets following the clean out now sits at 529. Around 123 government bodies contributed data to the portal.
Sheridan said the number was still too low.
“A lot of momentum has built around open data in Australia, including within governments around the country and we are pleased to report that a growing number of federal agencies are looking at how they can better publish data to be more efficient, improve policy development and analysis, deliver mobile services and support greater transparency and public innovation,” he said….
The Federal Government’s approach to open data has previously been criticised as “patchy” and slow, due in part to several shortcomings in the data.gov.au website as well as slow progress in agencies adopting an open approach by default.
The Australian Information Commissioner’s February report on open data in government outlined the manual uploading and updating of datasets, lack of automated entry for metadata and a lack of specific search functions within data.gov.au as obstacles affecting the efforts pushing a whole-of-government approach to open data.
The introduction of the new CKAN platform is expected to go some way to addressing the highlighted concerns.”

Lessons in the crowdsourced verification of news from Storyful and Reddit’s Syria forum


at GigaOm: “One of the most powerful trends in media over the past year is the crowdsourced verification of news, whether it’s the work of a blogger like Brown Moses or former NPR journalist Andy Carvin. Two other interesting efforts in this area are the “open newsroom” approach taken by Storyful — which specializes in verifying social-media reports for mainstream news entities — and a Reddit forum devoted to crowdsourcing news coverage of the civil war in Syria.
Storyful journalist Joe Galvin recently looked at some of the incidents that the company has helped either debunk or verify over the past year — including a fake tweet from the official account of the Associated Press about explosions at the White House (which sent the Dow Jones index plummeting before it was corrected), a claim from Russian authorities that a chemical attack in Syria had been pre-meditated, and a report from investigative journalist Seymour Hersh about the same attack that questioned whether the government had been involved….
Reddit, meanwhile, has been conducting some “open newsroom”-style experiments of its own around a number of news events, including the Syrian civil war. The site has come under fire in the past for some of those efforts — including the attempt to identify the bombers in the Boston bombings case, which went badly awry — but the Syrian thread in particular is a good example of how a smart aggregator can make sense of an ongoing news event. In a recent post at a site called Dissected News, one of the moderators behind the /r/SyrianCivilWar sub-Reddit — a 22-year-old law student named Christopher Kingdon (or “uptodatepronto” as he is known on the site) — wrote about his experiences with the forum, which is trying to be a broadly objective source for breaking news and information about the conflict….
Some of what the moderators do in the forum is similar to the kind of verification that Storyful or the BBC’s “user-generated content desk” do — checking photos and video for obvious signs of fakery and hoaxes. But Kingdon also describes how much effort his team of volunteers puts into ensuring that the sub-Reddit doesn’t degenerate into trolling or flame-wars. Strict rules are enforced “to prevent personal attacks, offensive and violent language and racism” and the moderators favor posts that “utilize sources, background information and a dash of common sense.”

Powering European public sector innovation – Towards a new architecture


Report of the expert group on public sector innovation: “The European Union faces an unprecedented crisis in economic growth, which has put public services under tremendous financial pressure. Many governments are also faced with long-term issues such as ageing societies, mounting social security and healthcare costs, high youth unemployment and a public service infrastructure that sometimes lags behind the needs of modern citizens and businesses. Under these conditions, innovation in public services is critical for the continued provision of such public services, in both quantity and quality. Public sector innovation can be defined as the process of generating new ideas, and implementing them to create value for society either through new or improved processes or services. The available evidence indicates that innovation in the public sector mostly happens randomly, rather than as a result of deliberate, systematic and strategic efforts. Innovation in the public sector, through strategic change, needs to become more ‘persistent’ and ‘cumulative’, in pursuit of a new and more collaborative governance model. There is a need for a new architecture for public sector innovation. Much can be done in individual Member States, regions and in local government to build capacity to innovate and to steer change processes. Innovation can emerge at all levels and innovation leadership can come from anyone. It is however the conviction of the expert group that the European institutions – including the Council of Ministers, the European Parliament, and the European Commission – can also play significant roles in fostering innovation both at European Union level and in individual Member States”

The Impact of Innovation Inducement Prizes


From the Compendium of Evidence on Innovation Policy/NESTA: “Innovation inducement prizes are one of the oldest types of innovation policy measure.  The popularity of innovation inducement prizes has gradually decreased during the early 20th century. However, innovation inducement prizes have regained some of their popularity since the 1990s with new prizes awarded by the US X Prize Foundation and with the current USA Administration’s efforts to use them in various government departments as an innovation policy instrument. Innovation Prizes are also becoming an important innovation policy instrument in the UK.  A recent report by McKinsey & Company (2009) estimates the value of prizes awarded to be between £600 million and £1.2million. Despite the growing popularity of innovation inducement prizes, the impact of this innovation policy measure is still not understood. This report brings together the existing evidence on the effects of innovation inducement prizes by drawing on a number of ex-ante and ex-post evaluations as well as limited academic literature. This report focuses on ex-ante innovation inducement prizes where the aim is to induce investment or attention to a specific goal or technology. This report does not discuss the impact of ex-post recognition prizes where the prize is given as a recognition after the intended outcome happens (e.g. Nobel Prize).
Innovation inducement prizes have a wide range of rationales and there is no agreed on dominant rationale in the literature. Traditionally, prizes have been seen as an innovation policy instrument that can overcome market failure by creating an incentive for the development of a particular technology or technology application. A second rationale is that the implementation demonstration projects in which not only creation of a specific technology is intended but also demonstration of the feasible application of this technology is targeted. A third rationale is related to the creation of a technology that will later be put in the public domain to attract subsequent research. Prizes are also increasingly organised for community and leadership building. As prizes probably allow more flexibility than most of the other innovation policy instruments, there is a large number of different prize characteristics and thus a vast number of prize typologies based on these characteristics.
Evidence on the effectiveness of prizes is scarce. There are only a few evaluations or academic works that deal with the creation of innovation output and even those which deal with the innovation output only rarely deals with the additionality. Only a very limited number of studies looked at if innovation inducement prizes led to more innovation itself or innovation outputs. As well as developing the particular technology that the innovation inducement prizes produce, they create prestige for both the prize sponsor and entrants. Prizes might also increase the public and sectoral awareness on specific technology issues. A related issue to the prestige gained from the prizes is the motivations of participants as a conditioning factor for innovation performance. Design issues are the main concern of the prizes literature. This reflects the importance of a careful design for the achievement of desired effects (and the limitation of undesired effects). There are a relatively large number of studies that investigated the influence of the design of prize objective on the innovation performance. A number of studies points out that sometimes prizes should be accompanied with or followed by other demand side initiatives to fulfil their objectives, mostly on the basis of ex-ante evaluations. Finally, prizes are also seen as a valuable opportunity for experimentation in innovation policy.
It is evident from the literature we analysed that the evidence on the impact of innovation inducement prizes is scarce. There is also a consensus that innovation inducement prizes are not a substitute for other innovation policy measures but are complementary under certain conditions. Prizes can be effective in creating innovation through more intense competition, engagement of wide variety of actors, distributing risks to many participants and by exploiting more flexible solutions through a less prescriptive nature of the definition of the problem in prizes. They can overcome some of the inherent barriers to other instruments, but if prizes are poorly designed, managed and awarded, they may be ineffective or even harmful.”

Ten thoughts for the future


The Economist: “CASSANDRA has decided to revisit her fellow forecasters Thomas Malnight and Tracey Keys to find out what their predictions are for 2014. Once again they have produced a collection of trends for the year ahead, in their “Global Trends Report”.
The possibilities of mind control seem alarming ( point 6) as do the  implications of growing income inequality (point 10). Cassandra also hopes that “unemployability” and “unemployerability”, as discussed in point 9, are contested next year (on both linguistic and social fronts).
Nevertheless, the forecasts make for intriguing reading and highlights appear below.
 1. From social everything to being smart socially
Social technologies are everywhere, but these vast repositories of digital “stuff” bury the exceptional among the unimportant. It’s time to get socially smart. Users are moving to niche networks to bring back the community feel and intelligence to social interactions. Businesses need to get smarter about extracting and delivering value from big data including challenging business models. For social networks, mobile is the great leveller. Competition for attention with other apps will intensify the battle to own key assets from identity to news sharing, demanding radical reinvention.
2. Information security: The genie is out of the bottle
Thought your information was safe? Think again. The information security genie is out of the bottle as cyber-surveillance and data mining by public and private organizations increases – and don’t forget criminal networks and whistleblowers. It will be increasingly hard to tell friend from foe in cyberspace as networks build artificial intelligence to decipher your emotions and smart cities track your every move. Big brother is here: Protecting identity, information and societies will be a priority for all.
3. Who needs shops anyway?
Retailers are facing a digitally driven perfect storm. Connectivity, rising consumer influence, time scarcity, mobile payments, and the internet of things, are changing where, when and how we shop – if smart machines have not already done the job. Add the sharing economy, driven by younger generations where experience and sustainable consumption are more important than ownership, and traditional retail models break down. The future of shops will be increasingly defined by experiential spaces offering personalized service, integrated online and offline value propositions, and pop-up stores to satisfy demands for immediacy and surprise.
4. Redistributing the industrial revolution
Complex, global value chains are being redistributed by new technologies, labour market shifts and connectivity. Small-scale manufacturing, including 3D and soon 4D printing, and shifting production economics are moving production closer to markets and enabling mass customization – not just by companies but by the tech-enabled maker movement which is going mainstream. Rising labour costs in developing markets, high unemployment in developed markets, global access to online talent and knowledge, plus advances in robotics mean reshoring of production to developed markets will increase. Mobility, flexibility and networks will define the future industrial landscape.
5. Hubonomics: The new face of globalization
As production and consumption become more distributed, hubs will characterize the next wave of “globalization.” They will specialize to support the needs of growing regional trade, emerging city states, on-line communities of choice, and the next generation of flexible workers and entrepreneurs. Underpinning these hubs will be global knowledge networks and new business and governance models based on hubonomics™, that leverage global assets and hub strengths to deliver local value.
6. Sci-Fi is here: Making the impossible possible
Cross-disciplinary approaches and visionary entrepreneurs are driving scientific breakthroughs that could change not just our lives and work but our bodies and intelligence. Labs worldwide are opening up the vast possibilities of mind control and artificial intelligence, shape-shifting materials and self-organizing nanobots, cyborgs and enhanced humans, space exploration, and high-speed, intelligent transportation. Expect great debate around the ethics, financing, and distribution of public and private benefits of these advances – and the challenge of translating breakthroughs into replicable benefits.
7. Growing pains: Transforming markets and generations
The BRICS are succumbing to Newton’s law of gravitation: Brazil’s lost it, India’s losing it, China’s paying the price for growth, Russia’s failing to make a superpower come-back, and South Africa’s economy is in disarray. In other developing markets currencies have tumbled, Arab Spring governments are still in turmoil and social unrest is increasing along with the number of failing states. But the BRICS & Beyond growth engine is far from dead. Rather it is experiencing growing pains which demand significant shifts in governance, financial systems, education and economic policies to catch up. The likely transformers will be younger generations who aspire to greater freedom and quality of life than their parents.
8. Panic versus denial: The resource gap grows, the global risks rise – but who is listening?
The complex nexus of food, water, energy and climate change presents huge global economic, environmental and societal challenges – heating up the battle to access new resources from the Arctic to fracking. Risks are growing, even as multilateral action stalls. It’s a crisis of morals, governance, and above all marketing and media, pitting crisis deniers against those who recognize the threats but are communicating panic versus reasoned solutions. Expect more debate and calls for responsible capitalism – those that are listening will be taking action at multiple levels in society and business.
9. Fighting unemployability and unemployerability
Companies are desperate for talented workers – yet unemployment rates remain high. Polarization towards higher and lower skill levels is squeezing mid-level jobs, even as employers complain that education systems are not preparing students for the jobs of the future. Fighting unemployability is driving new government-business partnerships worldwide, and will remain a critical issue given massive youth unemployment. Employers must also focus on organizational unemployerability – not being able to attract and retain desired talent – as new generations demand exciting and meaningful work where they can make an impact. If they can’t find it, they will quickly move on or swell the growing ranks of young entrepreneurs.
10. Surviving in a bipolar world: From expecting consistency to embracing ambiguity
Life is not fair, nor is it predictable.  Income inequality is growing. Intolerance and nationalism are rising but interdependence is the currency of a connected world. Pressure on leaders to deliver results today is intense but so too is the need for fundamental change to succeed in the long term. The contradictions of leadership and life are increasing faster than our ability to reconcile the often polarized perspectives and values each embodies. Increasingly, they are driving irrational acts of leadership (think the US debt ceiling), geopolitical, social and religious tensions, and individual acts of violence. Surviving in this world will demand stronger, responsible leadership comfortable with and capable of embracing ambiguity and uncertainty, as opposed to expecting consistency and predictability.”

Continued Progress: Engaging Citizen Solvers through Prizes


Blog post by Cristin Dorgelo: “Today OSTP released its second annual comprehensive report detailing the use of prizes and competitions by Federal agencies to spur innovation and solve Grand Challenges. Those efforts have expanded in the last two years under the America COMPETES Reauthorization Act of 2010, which granted all Federal agencies the authority to conduct prize competitions to spur innovation, solve tough problems, and advance their core missions.
This year’s report details the remarkable benefits the Federal Government reaped in Fiscal Year (FY) 2012 from more than 45 prize competitions across 10 agencies. To date, nearly 300 prize competitions have been implemented by 45 agencies through the website Challenge.gov.
Over the past four years, the Obama Administration has taken important steps to make prizes a standard tool in every agency’s toolbox. In his September 2009 Strategy for American Innovation, President Obama called on all Federal agencies to increase their use of prizes to address some of our Nation’s most pressing challenges. In March 2010, the Office of Management and Budget issued a policy framework to guide agencies in using prizes to mobilize American ingenuity and advance their respective core missions. Then, in September 2010, the Administration launched Challenge.gov, a one-stop shop where entrepreneurs and citizen solvers can find public-sector prize competitions.
The prize authority in COMPETES is a key piece of this effort. By giving agencies a clear legal path and expanded authority to deploy competitions and challenges, the legislation makes it dramatically easier for agencies to enlist this powerful approach to problem-solving and to pursue ambitious prizes with robust incentives…
To support these ongoing efforts, the General Services Administration  continues to train agencies about resources and vendors available to help them administer prize competitions. In addition, NASA’s Center of Excellence for Collaborative Innovation (CoECI) provides other agencies with a full suite of services for incentive prize pilots – from prize design, through implementation, to post-prize evaluation”

Reinventing Participation: Civic Agency and the Web Environment


New paper by Peter Dahlgren: “Participation is a key concept in the vocabulary of democracy, and can encompass a variety of dimensions. Moreover, it can be shaped by a range of different factors; my emphasis here is on the significance of the web environment in this regard. I first situate participation against the backdrop of democracy’s contemporary developments, including the onslaught of neolibealism. From there I offer a set of parameters that can help us grasp participation both conceptually and empirically: trajectory, visibility, voice , and sociality, and relate these to the affordances of the digital media. Thereafter I explore the cultural resources necessary for the facilitation of participation; for this I make use of a six-dimensional model of civic cultures. My discussion focuses on two of the dimensions, practices and identities; I again relate these to the web environment. I conclude with a dilemma that online democratic participation faces, namely what I call the isolation of the solo sphere, yet affirm that we are justified in maintaining a guarded optimism about the future of participation.”

Buenos Aires, A Pocket of Civic Innovation in Argentina


Rebecca Chao in TechPresident: “…In only a few years, the government, civil society and media in Buenos Aires have actively embraced open data. The Buenos Aires city government has been publishing data under a creative commons license and encouraging civic innovation through hackathons. NGOs have launched a number of tech-driven tools and Argentina’s second largest newspaper, La Nación, has published several hard-hitting data journalism projects. The result is a fledgling but flourishing open data culture in Buenos Aires, in a country that has not yet adopted a freedom of information law.

A Wikipedia for Open Government Data

In late August of this year, the Buenos Aires government declared a creative commons license for all of its digital content, which allows it be used for free, like Wikipedia content, with proper attribution. This applies to their new open data catalog that allows users to visualize the data, examine apps that have been created using the data and even includes a design lab for posting app ideas. Launched only in March, the government has already published fairly substantial data sets, including the salaries of city officials. The website also embodies the principals of openness in its design; it is built with open-source software and its code is available for reuse via GitHub.
“We were the first city in Argentina doing open government,” Rudi Borrmann tells techPresident over Skype. Borrmann is the Director of Buenos Aires’ Open Government Initiative. Previously, he was the social media editor at the city’s New Media Office but he also worked for many years in digital media…
While the civil society and media sectors have forged ahead in using open data, Borrmann tells techPresident that up in the ivory tower, openness to open data has been lagging. “Only technical schools are starting to create areas focused on working on open data,” he says.
In an interview with NYU’s govlab, Borrmann explained the significance of academia in using and pushing for more open data. “They have the means, the resources, the methodology to analyze…because in government you don’t have that time to analyze,” he said.
Another issue with open data is getting other branches of the government to modernize. Borrmann says that a lot of the Open Government’s work is done behind the scenes. “In general, you have very poor IT infrastructure all over Latin America” that interferes with the gathering and publishing of data, he says. “So in some cases it’s not about publishing or not publishing,” but about “having robust infrastructure for the information.”
It seems that the behind the scenes work is bearing some fruit. Just last week, on Dec. 6, the team behind the Buenos Aires open data website launched an impressive, interactive timeline, based on a similar timelapse map developed by a 2013 Knight-Mozilla Fellow, Noah Veltman. Against faded black and white photos depicting the subway from different decades over the last century, colorful pops of the Subterráneo lines emerge alongside factoids that go all the way back to 1910.”

The Documented Life


Sherry Turkle in the New York Times: “I’ve been studying people and mobile technology for more than 15 years. Until recently, it was the sharing that seemed most important. People didn’t seem to feel like themselves unless they shared a thought or feeling, even before it was clear in their mind. The new sensibility played on the Cartesian with a twist: “I share, therefore I am.”

These days, we still want to share, but now our first focus is to have, to possess, a photograph of our experience.

I interview people about their selfies. It’s how they keep track of their lives….We interrupt conversations for documentation all the time.

A selfie, like any photograph, interrupts experience to mark the moment. In this, it shares something with all the other ways we break up our day, when we text during class, in meetings, at the theater, at dinners with friends. And yes, at funerals, but also more regularly at church and synagogue services. We text when we are in bed with our partners and spouses. We watch our political representatives text during sessions.

Technology doesn’t just do things for us. It does thing to us, changing not just what we do but who we are. The selfie makes us accustomed to putting ourselves and those around us “on pause” in order to document our lives. It is an extension of how we have learned to put our conversations “on pause” when we send or receive a text, an image, an email, a call. When you get accustomed to a life of stops and starts, you get less accustomed to reflecting on where you are and what you are thinking.

We don’t experience interruptions as disruptions anymore….”

Philosophical Engineering: Toward a Philosophy of the Web


New book by Harry Halpin (Editor) and Alexandre Monnin (Editor) : “This is the first interdisciplinary exploration of the philosophical foundations of the Web, a new area of inquiry that has important implications across a range of domains.

  • Contains twelve essays that bridge the fields of philosophy, cognitive science, and phenomenology
  • Tackles questions such as the impact of Google on intelligence and epistemology, the philosophical status of digital objects, ethics on the Web, semantic and ontological changes caused by the Web, and the potential of the Web to serve as a genuine cognitive extension
  • Brings together insightful new scholarship from well-known analytic and continental philosophers, such as Andy Clark and Bernard Stiegler, as well as rising scholars in “digital native” philosophy and engineering
  • Includes an interview with Tim Berners-Lee, the inventor of the Web”…