Júlia Keserű at SunLight Foundation: “Strong evidence on the long-term impact of open data initiatives is incredibly scarce. The lack of compelling proof is partly due to the relative novelty of the open government field, but also to the inherent difficulties in measuring good governance and social change. We know that much of the impact of policy advocacy, for instance, occurs even before a new law or policy is introduced, and is thus incredibly difficult to evaluate. At the same time, it is also very hard to detect the causality between a direct change in the legal environment and the specific activities of a policy advocacy group. Attribution is equally challenging when it comes to assessing behavioral changes – who gets to take credit for increased political engagement and greater participation in democratic processes?
Open government projects tend to operate in an environment where the contribution of other stakeholders and initiatives is essential to achieving sustainable change, making it even more difficult to show the causality between a project’s activities and the impact it strives to achieve. Therefore, these initiatives cannot be described through simple “cause and effect” relationships, as they mostly achieve changes through their contribution to outcomes produced by a complex ecosystem of stakeholders — including journalists, think tanks, civil society organizations, public officials and many more — making it even more challenging to measure their direct impact.
We at the Sunlight Foundation wanted to tackle some of the methodological challenges of the field through building an evidence base that can empower further generalizations and advocacy efforts, as well as developing a methodological framework to unpack theories of change and to evaluate the impact of open data and digital transparency initiatives. A few weeks ago, we presented our research at the Cartagena Data Festival, and today we are happy to launch the first edition of our paper, which you can read below or on Scribd.
The outputs of this research include:
- A searchable repository of more than 100 examples on the outputs, outcomes and impacts of open data and digital technology projects;
- Three distinctive theories of change for open data and digital transparency initiatives from the Global South;
- A methodological framework to help develop more robust indicators of social and political change for the ecosystem of open data initiatives, by applying and revising the Outcome Mapping approach of IDRC to the field…(You can read the study at :The Social Impact of Open Data by juliakeseru)