Book by Alexandru C. Telea : “This book explores the study of processing and visually representing data sets. Data visualization is closely related to information graphics, information visualization, scientific visualization, and statistical graphics. This second edition presents a better treatment of the relationship between traditional scientific visualization and information visualization, a description of the emerging field of visual analytics, and updated techniques using the GPU and new generations of software tools and packages. This edition is also enhanced with exercises and downloadable code and data sets. See also Supplemental Material.
Social Collective Intelligence
New book edited by Daniele Miorandi, Vincenzo Maltese, Michael Rovatsos, Anton Nijholt, and James Stewart: “The book focuses on Social Collective Intelligence, a term used to denote a class of socio-technical systems that combine, in a coordinated way, the strengths of humans, machines and collectives in terms of competences, knowledge and problem solving capabilities with the communication, computing and storage capabilities of advanced ICT.
Social Collective Intelligence opens a number of challenges for researchers in both computer science and social sciences; at the same time it provides an innovative approach to solve challenges in diverse application domains, ranging from health to education and organization of work.
The book will provide a cohesive and holistic treatment of Social Collective Intelligence, including challenges emerging in various disciplines (computer science, sociology, ethics) and opportunities for innovating in various application areas.
By going through the book the reader will gauge insight and knowledge into the challenges and opportunities provided by this new, exciting, field of investigation. Benefits for scientists will be in terms of accessing a comprehensive treatment of the open research challenges in a multidisciplinary perspective. Benefits for practitioners and applied researchers will be in terms of access to novel approaches to tackle relevant problems in their field. Benefits for policy-makers and public bodies representatives will be in terms of understanding how technological advances can support them in supporting the progress of society and economy…”
Creating a national citizen engagement process for energy policy
Paper by Nick Pidgeon et al in the Proceedings of the National Academy of Sciences (PNAS): “This paper examines some of the science communication challenges involved when designing and conducting public deliberation processes on issues of national importance. We take as our illustrative case study a recent research project investigating public values and attitudes toward future energy system change for the United Kingdom. National-level issues such as this are often particularly difficult to engage the public with because of their inherent complexity, derived from multiple interconnected elements and policy frames, extended scales of analysis, and different manifestations of uncertainty. With reference to the energy system project, we discuss ways of meeting a series of science communication challenges arising when engaging the public with national topics, including the need to articulate systems thinking and problem scale, to provide balanced information and policy framings in ways that open up spaces for reflection and deliberation, and the need for varied methods of facilitation and data synthesis that permit access to participants’ broader values. Although resource intensive, national-level deliberation is possible and can produce useful insights both for participants and for science policy.”
Fighting Inequality in the New Gilded Age
Book Review by K. Sabeel Rahman in the Boston Review:
“In the years since the financial crisis, the realities of rapid economic recovery for some and stagnant wages for most has made increasingly clear that we live in a new Gilded Age: one marked by growing income inequality, decreasing social mobility, and concentrated corporate power. At the same time, we face an increasingly dysfunctional political system, apparently incapable of addressing these fundamental economic challenges.
This is not the first time the country has been caught in this confluence of economic inequality and political dysfunction. The first Gilded Age, in the late nineteenth century, experienced a similar moment of economic upheaval, instability, inequality, rising corporate power, and unresponsive government. These challenges triggered some of the most powerful reform movements in American history: the labor and antitrust movements, the Populist movement of agrarian reformers, and the Progressive movement of urban social and economic reformers. These reformers were not perfect—their record on racial and ethnic inequality is especially glaring—but they were enormously successful in creating new institutions and ideas that reshaped our economy and our politics. In particular, many of them were convinced that to address economic inequality, they had to first democratize politics, creating more robust forms of accountability and popular sovereignty against the influence of economic and political elites….
With his new book, White Collar Government: The Hidden Role of Class in Economic Policy-Making (2013), Nicholas Carnes argues that there is a third, even more important source of elite political influence: the dominance of upper class individuals in the composition of legislatures themselves. Despite the considerable external pressures of donors, constituent preferences, parties, and interest groups, legislators still possess significant discretion, and as a result their personal views about economic policy matter. Legislators of different class backgrounds, Carnes demonstrates, have distinct views on everything from labor to welfare programs and anti-poverty policies, to the very idea of government itself. On unemployment, labor rights, tax policy, and corporate protections, many of the central economic policy issues of our time involve a cleavage between wealthy and working class interests. The underrepresentation of the working class results in an underrepresentation of working class interests, exacerbating income inequality. “Whether our political system listens to one voice or another depends not just on who’s doing the talking or how loud they are,” writes Carnes; “it also depends on who’s doing the listening.”….
In The Promise of Participation: Experiments in Participatory Governance in Honduras and Guatemala (2013), Daniel Altschuler and Javier Corrales focus similar questions to those animating Carnes’ account: What institutional contexts enable ordinary citizens—especially poorer ones—to expand their representation in decision-making? What expands their knowledge of issues, their political networks, and their willingness to participate more broadly to advocate for their interests? To gain traction on this question, they undertook the first large-scale study of participatory governance, examining the nation-wide community-managed schools program in Honduras and Guatemala. These programs operated in areas that conventionally might be considered inhospitable to participatory governance: poor, rural districts. These programs engaged parents by giving them management and administrative duties in the daily activities of the school. In both countries, the programs were established to both address pervasive disparities in educational attainment, and to improve the accountability of government officials in delivering basic services to the poor….
In Making Democracy Fun: How Game Design Can Empower Citizens and Transform Politics, Lerner takes a practitioners’ look at participatory governance. Lerner is the Executive Director of the Participatory Budgeting Project, a non-profit dedicated to adapting participatory budgeting systems and implementing them in cities such as New York, Chicago, and Boston. Where Altschuler and Corrales are primarily concerned with the macro-institutional contexts that make participatory governance systems work well, Lerner’s insights revolve around the micro-practices of how to make participation effective at the face-to-face level….
Our recent experience of economic inequality has fueled the rise of a new social science of economic inequality and oligarchy, most recently and famously captured in the debates over Thomas Piketty’s Capital in the Twenty-First Century. But we also need a constructive account of what a more responsive and representative democratic politics looks like, and how to achieve it. Reformers coming out of the Gilded Age of the late nineteenth century similarly located the roots of economic inequality in political inequality. The era of Standard Oil and J.P. Morgan (the man, before the firm), and of widening income inequality was also the era of dysfunctional machine politics and a conservative Supreme Court that stymied social reform. These challenges fueled reform movements that struggled to restore popular sovereignty and genuine democracy—proposing everything from antitrust restraints on corporate power, to the first campaign finance systems, to new procedures for popular elections of Senators, party primaries, and direct democratic referenda. It was during this period that state and federal governments experimented with antitrust laws, rate regulation, and labor regulation. Many of the economic ideas first developed out of this ferment came to fruition in the New Deal.
Today we see the echoes of this zeal in the debates around campaign finance reform and the problem of “too-big-to-fail” banks. But reviving genuine democratic equality to address economic inequality requires a broader view of potential democratizing reforms. Carnes reminds us that the identity of who governs matters as much for class and economic policy as for any other dimension of representation. But Altschuler, Corrales, and Lerner suggest as well the importance of looking outside legislatures. Governing involves more than writing statutes; it is solving disputes, administering social services, implementing directives at the local level. And these are spaces where the prospects for greater political power—especially on the part of economically marginalized groups—may even be greater than at national scale legislatures. The proliferation of open government efforts in the United States—from governmental transparencyto engaging citizens to report potholes—suggests a growing reform interest in creating alternative channels for participation and representation. But too often these efforts are more limited than their rhetoric, focusing more narrowly on making existing policies well known or efficient, rather than empowering participants to challenge and reshape them. These books underscore that genuine democratic reform requires actually empowering ordinary citizens to drive the business of governing.”
DrivenData
DrivenData Blog: “As we begin launching our first competitions, we thought it would be a good idea to lay out what exactly we’re trying to do and why….
At DrivenData, we want to bring cutting-edge practices in data science and crowdsourcing to some of the world’s biggest social challenges and the organizations taking them on. We host online challenges, usually lasting 2-3 months, where a global community of data scientists competes to come up with the best statistical model for difficult predictive problems that make a difference.
Just like every major corporation today, nonprofits and NGOs have more data than ever before. And just like those corporations, they are trying to figure out how to make the best use of their data. We work with mission-driven organizations to identify specific predictive questions that they care about answering and can use their data to tackle.
Then we host the online competitions, where experts from around the world vie to come up with the best solution. Some competitors are experienced data scientists in the private sector, analyzing corporate data by day, saving the world by night, and testing their mettle on complex questions of impact. Others are smart, sophisticated students and researchers looking to hone their skills on real-world datasets and real-world problems. Still more have extensive experience with social sector data and want to bring their expertise to bear on new, meaningful challenges – with immediate feedback on how well their solution performs.
Like any data competition platform, we want to harness the power of crowds combined with the increasing prevalence of large, relevant datasets. Unlike other data competition platforms, our primary goal is to create actual, measurable, lasting positive change in the world with our competitions. At the end of each challenge, we work with the sponsoring organization to integrate the winning solutions, giving them the tools to drive real improvements in their impact….
We are launching soon and we want you to join us!
If you want to get updates about our launch this fall with exciting, real competitions, please sign up for our mailing list here and follow us on Twitter: @drivendataorg.
If you are a data scientist, feel free to create an account and start playing with our first sandbox competitions.
If you are a nonprofit or public sector organization, and want to squeeze every drop of mission effectiveness out of your data, check out the info on our site and let us know! “
Smartphone Movements Could Reveal Empty Parking Spots
Caleb Garling at MIT Technology Review: “Researchers have come up with a novel way to find parking spots with your smartphone. It promises to be much easier than driving around looking for an empty space, and doesn’t require the installation of pricey sensors or other methods for tracking available spots.
At the State University of New York at Buffalo, researchers built an app called PocketParker that does what they’re calling “pocketsourcing”—essentially, turning smartphones into passive sensors that track the location and movements of other users who’ve installed the app. A remote computer crunches the aggregate user actions and determines the likelihood that a lot has an open space. A paper about PocketParker will be presented at the ubiquitous computing conference UbiComp in Seattle next week.
While some parking lots employ sensors to gather information about capacity, PocketParker works without any such infrastructure. It pulls parking lot data from OpenStreetMap and calculates the number of spaces in a given lot based on its dimensions. During a study, researchers found that they could predict the number of spaces to within 6 percent of the actual number.
The app uses the smartphone’s accelerometer to determine where a user is and gauges whether he’s looking for a parking spot based on his movements. If a user drives slowly through a parking lot without stopping, that signals that the lot is full. If a user displays movements typical of walking and then suddenly speeds up and leaves the lot, that signifies that he likely just got into his car and drove away. The app calculates this in the background. “There should be no interaction required,” says SUNY Buffalo computer science professor and paper coauthor Geoffrey Challen….”
The Stasi, casinos and the Big Data rush
Book Review by Hannah Kuchler of “What Stays in Vegas” (by Adam Tanner) in the Financial Times: “Books with sexy titles and decidedly unsexy topics – like, say, data – have a tendency to disappoint. But What Stays in Vegas is an engrossing, story-packed takedown of the data industry.
It begins, far from America’s gambling capital, in communist East Germany. The author, Adam Tanner, now a fellow at Harvard’s Institute for Quantitative Social Science, was in the late 1980s a travel writer taking notes on Dresden. What he did not realise was that the Stasi was busy taking notes on him – 50 pages in all – which he found when the files were opened after reunification. The secret police knew where he had stopped to consult a map, to whom he asked questions and when he looked in on a hotel.
Today, Tanner explains: “Thanks to meticulous data gathering from both public documents and commercial records, companies . . . know far more about typical consumers than the feared East German secret police recorded about me.”
Shining a light on how businesses outside the tech sector have become data addicts, Tanner focuses on Las Vegas casinos, which spotted the value in data decades ago. He was given access to Caesar’s Entertainment, one of the world’s largest casino operators. When chief executive Gary Loveman joined in the late 1990s, the former Harvard Business School professor bet the company’s future on harvesting personal data from its loyalty scheme. Rather than wooing the “whales” who spent the most, the company would use the data to decide which freebies were worth giving away to lure in mid-spenders who came back often – a strategy credited with helping the business grow.
The real revelations come when Tanner examines the data brokers’ “Cheez Whiz”. Like the maker of a popular processed dairy spread, he argues, data brokers blend ingredients from a range of sources, such as public records, marketing lists and commercial records, to create a detailed picture of your identity – and you will never quite be able to pin down the origin of any component…
The Big Data rush has gone into overdrive since the global economic crisis as marketers from different industries have sought new methods to grab the limited consumer spending available. Tanner argues that while users have in theory given permission for much of this information to be made public in bits and pieces, increasingly industrial-scale aggregation often feels like an invasion of privacy.
Privacy policies are so long and obtuse (one study Tanner quotes found that it would take a person more than a month, working full-time, to read all the privacy statements they come across in a year), people are unwittingly littering their data all over the internet. Anyway, marketers can intuit what we are like from the people we are connected to online. And as the data brokers’ lists are usually private, there is no way to check the compilers have got their facts right…”
Citizen Science: The Law and Ethics of Public Access to Medical Big Data
New Paper by Sharona Hoffman: “Patient-related medical information is becoming increasingly available on the Internet, spurred by government open data policies and private sector data sharing initiatives. Websites such as HealthData.gov, GenBank, and PatientsLikeMe allow members of the public to access a wealth of health information. As the medical information terrain quickly changes, the legal system must not lag behind. This Article provides a base on which to build a coherent data policy. It canvasses emergent data troves and wrestles with their legal and ethical ramifications.
Publicly accessible medical data have the potential to yield numerous benefits, including scientific discoveries, cost savings, the development of patient support tools, healthcare quality improvement, greater government transparency, public education, and positive changes in healthcare policy. At the same time, the availability of electronic personal health information that can be mined by any Internet user raises concerns related to privacy, discrimination, erroneous research findings, and litigation. This Article analyzes the benefits and risks of health data sharing and proposes balanced legislative, regulatory, and policy modifications to guide data disclosure and use.”
Agency Liability Stemming from Citizen-Generated Data
Paper by Bailey Smith for The Wilson Center’s Science and Technology Innovation Program: “New ways to gather data are on the rise. One of these ways is through citizen science. According to a new paper by Bailey Smith, JD, federal agencies can feel confident about using citizen science for a few reasons. First, the legal system provides significant protection from liability through the Federal Torts Claim Act (FTCA) and Administrative Procedures Act (APA). Second, training and technological innovation has made it easier for the non-scientist to collect high quality data.”
What Is Big Data?
datascience@berkeley Blog: ““Big Data.” It seems like the phrase is everywhere. The term was added to the Oxford English Dictionary in 2013 , appeared in Merriam-Webster’s Collegiate Dictionary by 2014 , and Gartner’s just-released 2014 Hype Cycle shows “Big Data” passing the “Peak of Inflated Expectations” and on its way down into the “Trough of Disillusionment.” Big Data is all the rage. But what does it actually mean?
A commonly repeated definition cites the three Vs: volume, velocity, and variety. But others argue that it’s not the size of data that counts, but the tools being used, or the insights that can be drawn from a dataset.
To settle the question once and for all, we asked 40+ thought leaders in publishing, fashion, food, automobiles, medicine, marketing and every industry in between how exactly they would define the phrase “Big Data.” Their answers might surprise you! Take a look below to find out what big data is:
- John Akred, Founder and CTO, Silicon Valley Data Science
- Philip Ashlock, Chief Architect of Data.gov
- Jon Bruner, Editor-at-Large, O’Reilly Media
- Reid Bryant, Data Scientist, Brooks Bell
- Mike Cavaretta, Data Scientist and Manager, Ford Motor Company
- Drew Conway, Head of Data, Project Florida
- Rohan Deuskar, CEO and Co-Founder, Stylitics
- Amy Escobar, Data Scientist, 2U
- Josh Ferguson, Chief Technology Officer, Mode Analytics
- John Foreman, Chief Data Scientist, MailChimp
- …
FULL LIST at datascience@berkeley Blog”