Big data algorithms can discriminate, and it’s not clear what to do about it

Curated on Posted on by Stefaan Verhulst

 at the Conversation: “This program had absolutely nothing to do with race…but multi-variable equations.”

That’s what Brett Goldstein, a former policeman for the Chicago Police Department (CPD) and current Urban Science Fellow at the University of Chicago’s School for Public Policy, said about a predictive policing algorithm he deployed at the CPD in 2010. His algorithm tells police where to look for criminals based on where people have been arrested previously. It’s a “heat map” of Chicago, and the CPD claims it helps them allocate resources more effectively.

Chicago police also recently collaborated with Miles Wernick, a professor of electrical engineering at Illinois Institute of Technology, to algorithmically generate a “heat list” of 400 individuals it claims have thehighest chance of committing a violent crime. In response to criticism, Wernick said the algorithm does not use “any racial, neighborhood, or other such information” and that the approach is “unbiased” and “quantitative.” By deferring decisions to poorly understood algorithms, industry professionals effectively shed accountability for any negative effects of their code.

But do these algorithms discriminate, treating low-income and black neighborhoods and their inhabitants unfairly? It’s the kind of question many researchers are starting to ask as more and more industries use algorithms to make decisions. It’s true that an algorithm itself is quantitative – it boils down to a sequence of arithmetic steps for solving a problem. The danger is that these algorithms, which are trained on data produced by people, may reflect the biases in that data, perpetuating structural racism and negative biases about minority groups.

There are a lot of challenges to figuring out whether an algorithm embodies bias. First and foremost, many practitioners and “computer experts” still don’t publicly admit that algorithms can easily discriminate.More and more evidence supports that not only is this possible, but it’s happening already. The law is unclear on the legality of biased algorithms, and even algorithms researchers don’t precisely understand what it means for an algorithm to discriminate….

While researchers clearly understand the theoretical dangers of algorithmic discrimination, it’s difficult to cleanly measure the scope of the issue in practice. No company or public institution is willing to publicize its data and algorithms for fear of being labeled racist or sexist, or maybe worse, having a great algorithm stolen by a competitor.

Even when the Chicago Police Department was hit with a Freedom of Information Act request, they did not release their algorithms or heat list, claiming a credible threat to police officers and the people on the list. This makes it difficult for researchers to identify problems and potentially provide solutions.

Legal hurdles

Existing discrimination law in the United States isn’t helping. At best, it’s unclear on how it applies to algorithms; at worst, it’s a mess. Solon Barocas, a postdoc at Princeton, and Andrew Selbst, a law clerk for the Third Circuit US Court of Appeals, argued together that US hiring law fails to address claims about discriminatory algorithms in hiring.

The crux of the argument is called the “business necessity” defense, in which the employer argues that a practice that has a discriminatory effect is justified by being directly related to job performance….(More)”

Push, Pull, and Spill: A Transdisciplinary Case Study in Municipal Open Government

Curated on Posted on by Stefaan Verhulst

New paper by Jan Whittington et al: “Cities hold considerable information, including details about the daily lives of residents and employees, maps of critical infrastructure, and records of the officials’ internal deliberations. Cities are beginning to realize that this data has economic and other value: If done wisely, the responsible release of city information can also release greater efficiency and innovation in the public and private sector. New services are cropping up that leverage open city data to great effect.

Meanwhile, activist groups and individual residents are placing increasing pressure on state and local government to be more transparent and accountable, even as others sound an alarm over the privacy issues that inevitably attend greater data promiscuity. This takes the form of political pressure to release more information, as well as increased requests for information under the many public records acts across the country.

The result of these forces is that cities are beginning to open their data as never before. It turns out there is surprisingly little research to date into the important and growing area of municipal open data. This article is among the first sustained, cross-disciplinary assessments of an open municipal government system. We are a team of researchers in law, computer science, information science, and urban studies. We have worked hand-in-hand with the City of Seattle, Washington for the better part of a year to understand its current procedures from each disciplinary perspective. Based on this empirical work, we generate a set of recommendations to help the city manage risk latent in opening its data….(More)”

The Trouble With Disclosure: It Doesn’t Work

Curated on Posted on by Stefaan Verhulst

Jesse Eisinger at ProPublica: “Louis Brandeis was wrong. The lawyer and Supreme Court justice famously declared that sunlight is the best disinfectant, and we have unquestioningly embraced that advice ever since.

 Over the last century, disclosure and transparency have become our regulatory crutch, the answer to every vexing problem. We require corporations and government to release reams of information on food, medicine, household products, consumer financial tools, campaign finance and crime statistics. We have a booming “report card” industry for a range of services, including hospitals, public schools and restaurants.

All this sunlight is blinding. As new scholarship is demonstrating, the value of all this information is unproved. Paradoxically, disclosure can be useless — and sometimes actually harmful or counterproductive.

“We are doing disclosure as a regulatory move all over the board,” says Adam J. Levitin, a law professor at Georgetown, “The funny thing is, we are doing this despite very little evidence of its efficacy.”

Let’s start with something everyone knows about — the “terms of service” agreements for the likes of iTunes. Like everybody else, I click the “I agree” box, feeling a flash of resentment. I’m certain that in Paragraph 184 is a clause signing away my firstborn to a life of indentured servitude to Timothy D. Cook as his chief caviar spoon keeper.

Our legal theoreticians have determined these opaque monstrosities work because someone, somewhere reads the fine print in these contracts and keeps corporations honest. It turns out what we laymen intuit is true: No one reads them, according to research by a New York University law professor, Florencia Marotta-Wurgler.

In real life, there is no critical mass of readers policing the agreements. And if there were an eagle-eyed crew of legal experts combing through these agreements, what recourse would they have? Most people don’t even know that the Supreme Court has gutted their rights to sue in court, and they instead have to go into arbitration, which usually favors corporations.

The disclosure bonanza is easy to explain. Nobody is against it. It’s politically expedient. Companies prefer such rules, especially in lieu of actual regulations that would curtail bad products or behavior. The opacity lobby — the remora fish class of lawyers, lobbyists and consultants in New York and Washington — knows that disclosure requirements are no bar to dodgy practices. You just have to explain what you’re doing in sufficiently incomprehensible language, a task that earns those lawyers a hefty fee.

Of course, some disclosure works. Professor Levitin cites two examples. The first is an olfactory disclosure. Methane doesn’t have any scent, but a foul smell is added to alert people to a gas leak. The second is ATM. fees. A study in Australia showed that once fees were disclosed, people avoided the high-fee machines and took out more when they had to go to them.

But to Omri Ben-Shahar, co-author of a recent book, ” More Than You Wanted To Know: The Failure of Mandated Disclosure,” these are cherry-picked examples in a world awash in useless disclosures. Of course, information is valuable. But disclosure as a regulatory mechanism doesn’t work nearly well enough, he argues….(More)

Algorithms and Bias

Curated on Posted on by Stefaan Verhulst

Q. and A. With Cynthia Dwork in the New York Times: “Algorithms have become one of the most powerful arbiters in our lives. They make decisions about the news we read, the jobs we get, the people we meet, the schools we attend and the ads we see.

Yet there is growing evidence that algorithms and other types of software can discriminate. The people who write them incorporate their biases, and algorithms often learn from human behavior, so they reflect the biases we hold. For instance, research has shown that ad-targeting algorithms have shown ads for high-paying jobs to men but not women, and ads for high-interest loans to people in low-income neighborhoods.

Cynthia Dwork, a computer scientist at Microsoft Research in Silicon Valley, is one of the leading thinkers on these issues. In an Upshot interview, which has been edited, she discussed how algorithms learn to discriminate, who’s responsible when they do, and the trade-offs between fairness and privacy.

Q: Some people have argued that algorithms eliminate discriminationbecause they make decisions based on data, free of human bias. Others say algorithms reflect and perpetuate human biases. What do you think?

A: Algorithms do not automatically eliminate bias. Suppose a university, with admission and rejection records dating back for decades and faced with growing numbers of applicants, decides to use a machine learning algorithm that, using the historical records, identifies candidates who are more likely to be admitted. Historical biases in the training data will be learned by the algorithm, and past discrimination will lead to future discrimination.

Q: Are there examples of that happening?

A: A famous example of a system that has wrestled with bias is the resident matching program that matches graduating medical students with residency programs at hospitals. The matching could be slanted to maximize the happiness of the residency programs, or to maximize the happiness of the medical students. Prior to 1997, the match was mostly about the happiness of the programs.

This changed in 1997 in response to “a crisis of confidence concerning whether the matching algorithm was unreasonably favorable to employers at the expense of applicants, and whether applicants could ‘game the system,’ ” according to a paper by Alvin Roth and Elliott Peranson published in The American Economic Review.

Q: You have studied both privacy and algorithm design, and co-wrote a paper, “Fairness Through Awareness,” that came to some surprising conclusions about discriminatory algorithms and people’s privacy. Could you summarize those?

A: “Fairness Through Awareness” makes the observation that sometimes, in order to be fair, it is important to make use of sensitive information while carrying out the classification task. This may be a little counterintuitive: The instinct might be to hide information that could be the basis of discrimination….

Q: The law protects certain groups from discrimination. Is it possible to teach an algorithm to do the same?

A: This is a relatively new problem area in computer science, and there are grounds for optimism — for example, resources from the Fairness, Accountability and Transparency in Machine Learning workshop, which considers the role that machines play in consequential decisions in areas like employment, health care and policing. This is an exciting and valuable area for research. …(More)”

Crowdfunding sites aim to make the law accessible to all

Curated on Posted on by Stefaan Verhulst

Jonathan Ford at the Financial Times: “Using the internet to harness the financial power of crowds is hardly novel. Almost since the first electronic impulse pinged its way across the world wide web, entrepreneurs have been dreaming up sites to facilitate everything from charitable donation to hard-nosed investment.

Peer-to-peer lending is now almost part of the mainstream. JustGiving, the charitable portal, has been going since 2000. But employing the web to raise money for legal actions remains a less well ploughed piece of virtual terrain.

At first glance, you might wonder why this is. There is already a booming offline trade in the commercial funding of litigation, especially in America and Britain, whether through lawyers’ no-win, no-fee arrangements or third party investment. And, indeed, a few pioneering crowdfunding vehicles have recently emerged in the US. One such is Invest4Justice, a site that boldly touts returns of “500 per cent plus in a few months”.

Whether these eye-catching figures are ultimately deliverable is — as lawyers like to say — moot. But there are risks in seeking to share the fruits of a third party’s action that can make it perilous for the crowdfunding investor. One is that when actions fail, those same backers might have to pay not only their own, but the successful party’s, costs….

But not all crowdfunding ventures seek to reward participants in the currency of cold financial return. Crowdjustice, Britain’s first legal crowdfunding website, seeks to scratch quite a different itch in the psyches of its participants….Among the causes it has taken up are a criminal appeal and a planning dispute in Lancashire involving a landfill site. The only real requirement for consideration is that the legal David confronting the corporate or governmental Goliath must have already engaged a lawyer to take on their case….This certainly means the risk of being dragged into proceedings is far lower. But it also raises a question: why would the public want to donate money to lawyers in the first place?

Ms Salasky thinks it ranges from a sense of justice to enlightened self-interest. “Donors can be people who take human rights seriously, but they could also be those who worry that something which is happening to someone else could also happen to them,” she says. It is one reason why perhaps the most potent application is seen to be in the fields of environmental and planning law. …(More)”

 

Beyond the Common Rule: Ethical Structures for Data Research in Non-Academic Settings

Curated on Posted on by Stefaan Verhulst

Future of Privacy Forum: “In the wake of last year’s news about the Facebook “emotional contagion” study and subsequent public debate about the role of A/B Testing and ethical concerns around the use of Big Data, FPF Senior Fellow Omer Tene participated in a December symposum on corporate consumer research hosted by Silicon Flatirons. This past month, the Colorado Technology Law Journal published a series of papers that emerged out of the symposium, including “Beyond the Common Rule: Ethical Structures for Data Research in Non-Academic Settings.”

“Beyond the Common Rule,” by Jules Polonetsky, Omer Tene, and Joseph Jerome, continues the Future of Privacy Forum’s effort to build on the notion of consumer subject review boards first advocated by Ryan Calo at FPF’s 2013 Big Data symposium. It explores how researchers, increasingly in corporate settings, are analyzing data and testing theories using often sensitive personal information. Many of these new uses of PII are simply natural extensions of current practices, and are either within the expectations of individuals or the bounds of the FIPPs. Yet many of these projects could involve surprising applications or uses of data, exceeding user expectations, and offering notice and obtaining consent could may not be feasible.

This article expands on ideas and suggestions put forward around the recent discussion draft of the White House Consumer Privacy Bill of Rights, which espouses “Privacy Review Boards” as a safety value for noncontextual data uses. It explores how existing institutional review boards within the academy and for human testing research could offer lessons for guiding principles, providing accountability and enhancing consumer trust, and offers suggestions for how companies — and researchers — can pursue both knowledge and data innovation responsibly and ethically….(More)”

Digital government evolution: From transformation to contextualization

Curated on Posted on by Stefaan Verhulst

Paper by Tomasz Janowski in the Government Information Quarterly: “The Digital Government landscape is continuously changing to reflect how governments are trying to find innovative digital solutions to social, economic, political and other pressures, and how they transform themselves in the process. Understanding and predicting such changes is important for policymakers, government executives, researchers and all those who prepare, make, implement or evaluate Digital Government decisions. This article argues that the concept of Digital Government evolves toward more complexity and greater contextualization and specialization, similar to evolution-like processes that lead to changes in cultures and societies. To this end, the article presents a four-stage Digital Government Evolution Model comprising Digitization (Technology in Government), Transformation (Electronic Government), Engagement (Electronic Governance) and Contextualization (Policy-Driven Electronic Governance) stages; provides some evidence in support of this model drawing upon the study of the Digital Government literature published in Government Information Quarterly between 1992 and 2014; and presents a Digital Government Stage Analysis Framework to explain the evolution. As the article consolidates a representative body of the Digital Government literature, it could be also used for defining and integrating future research in the area….(More)”

We are data: the future of machine intelligence

Curated on Posted on by Stefaan Verhulst

Douglas Coupland in the Financial Times: “…But what if the rise of Artificial Intuition instead blossoms under the aegis of theology or political ideology? With politics we can see an interesting scenario developing in Europe, where Google is by far the dominant search engine. What is interesting there is that people are perfectly free to use Yahoo or Bing yet they choose to stick with Google and then they get worried about Google having too much power — which is an unusual relationship dynamic, like an old married couple. Maybe Google could be carved up into baby Googles? But no. How do you break apart a search engine? AT&T was broken into seven more or less regional entities in 1982 but you can’t really do that with a search engine. Germany gets gaming? France gets porn? Holland gets commerce? It’s not a pie that can be sliced.

The time to fix this data search inequity isn’t right now, either. The time to fix this problem was 20 years ago, and the only country that got it right was China, which now has its own search engine and social networking systems. But were the British or Spanish governments — or any other government — to say, “OK, we’re making our own proprietary national search engine”, that would somehow be far scarier than having a private company running things. (If you want paranoia, let your government control what you can and can’t access — which is what you basically have in China. Irony!)

The tendency in theocracies would almost invariably be one of intense censorship, extreme limitations of access, as well as machine intelligence endlessly scouring its system in search of apostasy and dissent. The Americans, on the other hand, are desperately trying to implement a two-tiered system to monetise information in the same way they’ve monetised medicine, agriculture, food and criminality. One almost gets misty-eyed looking at North Koreans who, if nothing else, have yet to have their neurons reconfigured, thus turning them into a nation of click junkies. But even if they did have an internet, it would have only one site to visit, and its name would be gloriousleader.nk.

. . .

To summarise. Everyone, basically, wants access to and control over what you will become, both as a physical and metadata entity. We are also on our way to a world of concrete walls surrounding any number of niche beliefs. On our journey, we get to watch machine intelligence become profoundly more intelligent while, as a society, we get to watch one labour category after another be systematically burped out of the labour pool. (Doug’s Law: An app is only successful if it puts a lot of people out of work.)…(More)”

How Collaboration and Crowdsourcing are Changing Legal Research

Curated on Posted on by Stefaan Verhulst

Susan Martin at Legal Current/ThomsonReuters: “Bob Ambrogi, lawyer, consultant and blogger at Law Sites, spoke at a well-attended session this morning at the American Association of Law Libraries (AALL) Annual Meeting. Titled “Playing Well With Others: How Collaboration and Crowdsourcing are Changing Legal Research,” Ambrogi’s presentation began with a light-hearted scolding of lawyers and legal professionals who simply “aren’t very good at sharing.”

“Crowdsourcing requires sharing and lawyers tend to be very possessive, so that makes it difficult,” said Ambrogi….

Why would a legal researcher want to do this? To establish credibility, according to Ambrogi. “Blogging is another way of doing this. It’s a good example of pulling together all the commentary out there so it lives in one place,” he said. “The more we can tap into the collective knowledge out there and use professionals to share their own legal materials in one central space…that’s a real benefit.”

Ambrogi then shared some examples of crowdsourcing gone wrong, where sites were built and abandoned or simply not updated enough to be effective. Examples include Spindle Law, Jurify and Standardforms.org.

He then went on to showcase three examples of great crowdsourced sites:

So how can lawyers learn to play well with others? Ambrogi offered the following tips, in closing:

  1. Make it easy to contribute
  2. Make it rewarding to contribute
  3. Make the content useful to others
  4. Success will breed success. (More)”

The Art of Changing a City

Curated on Posted on by Stefaan Verhulst

Antanas Mockus in the New York Times: “Between 1995 and 2003, I served two terms as mayor of Bogotá. Like most cities in the world, Colombia’s capital had a great many problems that needed fixing and few people believed they could be fixed.

As a professor of philosophy, I had little patience with conventional wisdom. When I was threatened by the leftist guerrilla group known as FARC, as hundreds of Colombian mayors were, I decided to wear a bulletproof vest. But mine had a hole cut in the shape of a heart over my chest. I wore that symbol of confidence, or defiance, for nine months.

Here’s what I learned: People respond to humor and playfulness from politicians. It’s the most powerful tool for change we have.

Bogotá’s traffic was chaotic and dangerous when I came to office. We decided the city needed a radical new approach to traffic safety. Among various strategies, we printed and distributed hundreds of thousands of “citizens’ cards,” which had a thumbs-up image on one side to flash at courteous drivers, and a thumbs-down on the other to express disapproval. Within a decade, traffic fatalities fell by more than half.

Another initiative in a small area of the city was to replace corrupt traffic police officers with mime artists. The idea was that instead of cops handing out tickets and pocketing fines, these performers would “police” drivers’ behavior by communicating with mime — for instance, pretending to be hurt or offended when a vehicle ignored the pedestrian right of way in a crosswalk. Could this system, which boiled down to publicly signaled approval or disapproval, really work?

We had plenty of skeptics. At a news conference, a journalist asked, “Can the mimes serve traffic fines?” That is legally impermissible, I answered. “Then it won’t work,” he declared.

But change is possible. People began to obey traffic signals and, for the first time, they respected crosswalks. Within months, I was able to dissolve the old, corrupt transit police force of about 1,800 officers, arranging with the national police service to replace them.

….

This illustrates another lesson we learned. It helps to develop short, pleasing experiences for people that generate stories of delightful surprise, moments of mutual admiration among citizens and the welcome challenge of understanding something new. But then you need to consolidate those stories with good statistical results obtained through cold, rational measurement. That creates a virtuous cycle, so that congenial new experiences lead to statistically documented improvements, and the documentation raises expectations for more welcome change.

The art of politics is a curious business. It combines, as no other profession or occupation does, rigorous reasoning, sincere emotions and extroverted body language, with what are sometimes painfully cold, slow and planned strategic interactions. It is about leading, but not directing: What people love most is when you write on the blackboard a risky first half of a sentence and then recognize their freedom to write the other half.

My main theoretical and practical concern has been how to use the force of social and moral regulation to obtain the rule of law. This entailed a fundamental respect for human lives, expressed in the dictum “Life is sacred.” My purpose was to create a cosmopolitan culture of citizenship in which expressions like “crimes against humanity” would find a precise operational meaning….(More)”