Assessing employer intent when AI hiring tools are biased

Curated on Posted on by Stefaan Verhulst

Report by Caitlin Chin at Brookings: “When it comes to gender stereotypes in occupational roles, artificial intelligence (AI) has the potential to either mitigate historical bias or heighten it. In the case of the Word2vec model, AI appears to do both.

Word2vec is a publicly available algorithmic model built on millions of words scraped from online Google News articles, which computer scientists commonly use to analyze word associations. In 2016, Microsoft and Boston University researchers revealed that the model picked up gender stereotypes existing in online news sources—and furthermore, that these biased word associations were overwhelmingly job related. Upon discovering this problem, the researchers neutralized the biased word correlations in their specific algorithm, writing that “in a small way debiased word embeddings can hopefully contribute to reducing gender bias in society.”

Their study draws attention to a broader issue with artificial intelligence: Because algorithms often emulate the training datasets that they are built upon, biased input datasets could generate flawed outputs. Because many contemporary employers utilize predictive algorithms to scan resumes, direct targeted advertising, or even conduct face- or voice-recognition-based interviews, it is crucial to consider whether popular hiring tools might be susceptible to the same cultural biases that the researchers discovered in Word2vec.

In this paper, I discuss how hiring is a multi-layered and opaque process and how it will become more difficult to assess employer intent as recruitment processes move online. Because intent is a critical aspect of employment discrimination law, I ultimately suggest four ways upon which to include it in the discussion surrounding algorithmic bias….(More)”

This report from The Brookings Institution’s Artificial Intelligence and Emerging Technology (AIET) Initiative is part of “AI and Bias,” a series that explores ways to mitigate possible biases and create a pathway toward greater fairness in AI and emerging technologies.

Engaging citizens in determining the appropriate conditions and purposes for re-using Health Data

Curated on Posted on by Stefaan Verhulst

Beth Noveck at The GovLab: “…The term, big health data, refers to the ability to gather and analyze vast quantities of online information about health, wellness and lifestyle. It includes not only our medical records but data from apps that track what we buy, how often we exercise and how well we sleep, among many other things. It provides an ocean of information about how healthy or ill we are, and unsurprisingly, doctors, medical researchers, healthcare organizations, insurance companies and governments are keen to get access to it. Should they be allowed to?

It’s a huge question, and AARP is partnering with GovLab to learn what older Americans think about it. AARP is a non-profit organization — the largest in the nation and the world — dedicated to empowering Americans to choose how they live as they age. In 2018 it had more than 38 million members. It is a key voice in policymaking in the United States, because it represents the views of people aged over 50 in this country.

From today, AARP and the GovLab are using the Internet to capture what AARP members feel are the most urgent issues confronting them to try to discover what worries people most: the use of big health data or the failure to use it.

The answers are not simple. On the one hand, increasing the use and sharing of data could enable doctors to make better diagnoses and interventions to prevent disease and make us healthier. It could lead medical researchers to find cures faster, while the creation of health data businesses could strengthen the economy.

On the other hand, the collection, sharing, and use of big health data could reveal sensitive personal information over which we have little control. This data could be sold without our consent, and be used by entities for surveillance or discrimination, rather than to promote well-being….(More)”.

A World With a Billion Cameras Watching You Is Just Around the Corner

Curated on Posted on by Stefaan Verhulst

Liza Lin and Newley Purnell at the Wall Street Journal: “As governments and companies invest more in security networks, hundreds of millions more surveillance cameras will be watching the world in 2021, mostly in China, according to a new report.

The report, from industry researcher IHS Markit, to be released Thursday, said the number of cameras used for surveillance would climb above 1 billion by the end of 2021. That would represent an almost 30% increase from the 770 million cameras today. China would continue to account for a little over half the total.

Fast-growing, populous nations such as India, Brazil and Indonesia would also help drive growth in the sector, the report said. The number of surveillance cameras in the U.S. would grow to 85 million by 2021, from 70 million last year, as American schools, malls and offices seek to tighten security on their premises, IHS analyst Oliver Philippou said.

Mr. Philippou said government programs to implement widespread video surveillance to monitor the public would be the biggest catalyst for the growth in China. City surveillance also was driving demand elsewhere.

“It’s a public-safety issue,” Mr. Philippou said in an interview. “There is a big focus on crime and terrorism in recent years.”

The global security-camera industry has been energized by breakthroughs in image quality and artificial intelligence. These allow better and faster facial recognition and video analytics, which governments are using to do everything from managing traffic to predicting crimes.

China leads the world in the rollout of this kind of technology. It is home to the world’s largest camera makers, with its cameras on street corners, along busy roads and in residential neighborhoods….(More)”.

Public Entrepreneurship and Policy Engineering

Curated on Posted on by Stefaan Verhulst

Essay by Beth Noveck at Communications of the ACM: “Science and technology have progressed exponentially, making it possible for humans to live longer, healthier, more creative lives. The explosion of Internet and mobile phone technologies have increased trade, literacy, and mobility. At the same time, life expectancy for the poor has not increased and is declining.

As science fiction writer William Gibson famously quipped, the future is here, but unevenly distributed. With urgent problems from inequality to climate change, we must train more passionate and innovative people—what I call public entrepreneurs—to learn how to leverage new technology to tackle public problems. Public problems are those compelling and important challenges where neither the problem is well understood nor the solution agreed upon, yet we must devise and implement approaches, often from different disciplines, in an effort to improve people’s lives….(More)”.

Regulating Artificial Intelligence

Curated on Posted on by Stefaan Verhulst

Book by Thomas Wischmeyer and Timo Rademacher: “This book assesses the normative and practical challenges for artificial intelligence (AI) regulation, offers comprehensive information on the laws that currently shape or restrict the design or use of AI, and develops policy recommendations for those areas in which regulation is most urgently needed. By gathering contributions from scholars who are experts in their respective fields of legal research, it demonstrates that AI regulation is not a specialized sub-discipline, but affects the entire legal system and thus concerns all lawyers. 

Machine learning-based technology, which lies at the heart of what is commonly referred to as AI, is increasingly being employed to make policy and business decisions with broad social impacts, and therefore runs the risk of causing wide-scale damage. At the same time, AI technology is becoming more and more complex and difficult to understand, making it harder to determine whether or not it is being used in accordance with the law. In light of this situation, even tech enthusiasts are calling for stricter regulation of AI. Legislators, too, are stepping in and have begun to pass AI laws, including the prohibition of automated decision-making systems in Article 22 of the General Data Protection Regulation, the New York City AI transparency bill, and the 2017 amendments to the German Cartel Act and German Administrative Procedure Act. While the belief that something needs to be done is widely shared, there is far less clarity about what exactly can or should be done, or what effective regulation might look like. 

The book is divided into two major parts, the first of which focuses on features common to most AI systems, and explores how they relate to the legal framework for data-driven technologies, which already exists in the form of (national and supra-national) constitutional law, EU data protection and competition law, and anti-discrimination law. In the second part, the book examines in detail a number of relevant sectors in which AI is increasingly shaping decision-making processes, ranging from the notorious social media and the legal, financial and healthcare industries, to fields like law enforcement and tax law, in which we can observe how regulation by AI is becoming a reality….(More)”.

The Golden Age of Social Science

Curated on Posted on by Stefaan Verhulst

Essay by Anastasia Buyalskaya, Marcos Gallo and Colin Camerer: “In this short essay we argue that social science is entering a golden age, marked by explosive growth in new data and analytic methods, interdisciplinarity, and a recognition that both of those ingredients are necessary to solve hard problems. Two examples are given to illustrate these themes, which are behavioral economics and social networks. Numerous other specific study examples are then given. We also address the challenges that accompany the three positive trends, which include informatics, career incentives, and the search for unifying frameworks….(More)”.

Is There a Crisis of Truth?

Curated on Posted on by Stefaan Verhulst

Essay by Steven Shapin: “…It seems irresponsible or perverse to reject the idea that there is a Crisis of Truth. No time now for judicious reflection; what’s needed is a full-frontal attack on the Truth Deniers. But it’s good to be sure about the identity of the problem before setting out to solve it. Conceiving the problem as a Crisis of Truth, or even as a Crisis of Scientific Authority, is not, I think, the best starting point. There’s no reason for complacency, but there is reason to reassess which bits of our culture are in a critical state and, once they are securely identified, what therapies are in order.

Start with the idea of Truth. What could be more important, especially if the word is used — as it often is in academic writing — as a placeholder for Reality? But there’s a sort of luminous glow around the notion of Truth that prejudges and pre-processes the attitudes proper to entertain about it. The Truth goes marching on. God is Truth. The Truth shall set you free. Who, except the mad and the malevolent, could possibly be against Truth? It was, after all, Pontius Pilate who asked, “What is Truth?” — and then went off to wash his hands.

So here’s an only apparently pedantic hint about how to construe Truth and also about why our current problem might not be described as a Crisis of Truth. In modern common usage, Truth is a notably uncommon term. The natural home of Truth is not in the workaday vernacular but in weekend, even language-gone-on-holiday, scenes. The notion of Truth tends to crop up when statements about “what’s the case” are put under pressure, questioned, or picked out for celebration. Statements about “the case” can then become instances of the Truth, surrounded by an epistemic halo. Truth is invoked when we swear to tell it — “the whole Truth and nothing but” — in legal settings or in the filling-out of official forms when we’re cautioned against departing from it; or in those sorts of school and bureaucratic exams where we’re made to choose between True and False. Truth is brought into play when it’s suspected that something of importance has been willfully obscured — as when Al Gore famously responded to disbelief in climate change by insisting on “an inconvenient truth” or when we demand to be told the Truth about the safety of GMOs. [2]

Truth-talk appears in such special-purpose forums as valedictory statements where scientists say that their calling is a Search for Truth. And it’s worth considering the difference between saying that and saying they’re working to sequence a breast cancer gene or to predict when a specific Indonesian volcano is most likely to erupt. Truth stands to Matters-That-Are-the-Case roughly as incantations, proverbs, and aphorisms stand to ordinary speech. Truth attaches more to some formal intellectual practices than to others — to philosophy, religion, art, and, of course, science, even though in science there is apparent specificity. Compare those sciences that seem good fits with the notion of a Search for Truth to those that seem less good fits: theoretical physics versus seismology, academic brain science versus research on the best flavoring for a soft drink. And, of course, Truth echoes around philosophy classrooms and journals, where theories of what it is are advanced, defended, and endlessly disputed. Philosophers collectively know that Truth is very important, but they don’t collectively know what it is.

I’ve said that Truth figures in worries about the problems of knowledge we’re said to be afflicted with, where saying that we have a Crisis of Truth both intensifies the problem and gives it a moral charge. In May 2019, Angela Merkel gave the commencement speech at Harvard. Prettily noting the significance of Harvard’s motto, Veritas, the German Chancellor described the conditions for academic inquiry, which, she said, requires that “we do not describe lies as truth and truth as lies,” nor that “we accept abuses [Missstände] as normal.” The Harvard audience stood and cheered: they understood the coded political reference to Trump and evidently agreed that the opposite of Truth was a lie — not just a statement that didn’t match reality but an intentional deception. You can, however, think of Truth’s opposite as nonsense, error, or bullshit, but calling it a lie was to position Truth in a moral field. Merkel was not giving Harvard a lesson in philosophy but a lesson in global civic virtue….(More)”.

Algorithmic Regulation

Curated on Posted on by Stefaan Verhulst

Book edited by Karen Yeung and Martin Lodge: “As the power and sophistication of of ‘big data’ and predictive analytics has continued to expand, so too has policy and public concern about the use of algorithms in contemporary life. This is hardly surprising given our increasing reliance on algorithms in daily life, touching policy sectors from healthcare, transport, finance, consumer retail, manufacturing education, and employment through to public service provision and the operation of the criminal justice system. This has prompted concerns about the need and importance of holding algorithmic power to account, yet it is far from clear that existing legal and other oversight mechanisms are up to the task. This collection of essays, edited by two leading regulatory governance scholars, offers a critical exploration of ‘algorithmic regulation’, understood both as a means for co-ordinating and regulating social action and decision-making, as well as the need for institutional mechanisms through which the power of algorithms and algorithmic systems might themselves be regulated. It offers a unique perspective that is likely to become a significant reference point for the ever-growing debates about the power of algorithms in daily life in the worlds of research, policy and practice. The range of contributors are drawn from a broad range of disciplinary perspectives including law, public administration, applied philosophy, data science and artificial intelligence.

Taken together, they highlight the rise of algorithmic power, the potential benefits and risks associated with this power, the way in which Sheila Jasanoff’s long-standing claim that ‘technology is politics’ has been thrown into sharp relief by the speed and scale at which algorithmic systems are proliferating, and the urgent need for wider public debate and engagement of their underlying values and value trade-offs, the way in which they affect individual and collective decision-making and action, and effective and legitimate mechanisms by and through which algorithmic power is held to account….(More)”.

Artificial Intelligence and National Security

Curated on Posted on by Stefaan Verhulst

CRS Report: “Artificial intelligence (AI) is a rapidly growing field of technology with potentially significant implications for national security. As such, the U.S. Department of Defense (DOD) and other nations are developing AI applications for a range of military functions. AI research is underway in the fields of intelligence collection and analysis, logistics, cyber operations, information operations, command and control, and in a variety of semiautonomous and autonomous vehicles.

Already, AI has been incorporated into military operations in Iraq and Syria. Congressional action has the potential to shape the technology’s development further, with budgetary and legislative decisions influencing the growth of military applications as well as the pace of their adoption.

AI technologies present unique challenges for military integration, particularly because the bulk of AI development is happening in the commercial sector. Although AI is not unique in this regard, the defense acquisition process may need to be adapted for acquiring emerging technologies like AI. In addition, many commercial AI applications must undergo significant modification prior to being functional for the military.

A number of cultural issues also challenge AI acquisition, as some commercial AI companies are averse to partnering with DOD due to ethical concerns, and even within the department, there can be resistance to incorporating AI technology into existing weapons systems and processes.

Potential international rivals in the AI market are creating pressure for the United States to compete for innovative military AI applications. China is a leading competitor in this regard, releasing a plan in 2017 to capture the global lead in AI development by 2030. Currently, China is primarily focused on using AI to make faster and more well-informed decisions, as well as on developing a variety of autonomous military vehicles. Russia is also active in military AI development, with a primary focus on robotics.

Although AI has the potential to impart a number of advantages in the military context, it may also introduce distinct challenges. AI technology could, for example, facilitate autonomous operations, lead to more informed military decisionmaking, and increase the speed and scale of military action. However, it may also be unpredictable or vulnerable to unique forms of manipulation. As a result of these factors, analysts hold a broad range of opinions on how influential AI will be in future combat operations. While a small number of analysts believe that the technology will have minimal impact, most believe that AI will have at least an evolutionary—if not revolutionary—effect….(More)”.

Rosie the Robot: Social accountability one tweet at a time

Curated on Posted on by Stefaan Verhulst

Blogpost by Yasodara Cordova and Eduardo Vicente Goncalvese: “Every month in Brazil, the government team in charge of processing reimbursement expenses incurred by congresspeople receives more than 20,000 claims. This is a manually intensive process that is prone to error and susceptible to corruption. Under Brazilian law, this information is available to the public, making it possible to check the accuracy of this data with further scrutiny. But it’s hard to sift through so many transactions. Fortunately, Rosie, a robot built to analyze the expenses of the country’s congress members, is helping out.

Rosie was born from Operação Serenata de Amor, a flagship project we helped create with other civic hackers. We suspected that data provided by members of Congress, especially regarding work-related reimbursements, might not always be accurate. There were clear, straightforward reimbursement regulations, but we wondered how easily individuals could maneuver around them. 

Furthermore, we believed that transparency portals and the public data weren’t realizing their full potential for accountability. Citizens struggled to understand public sector jargon and make sense of the extensive volume of data. We thought data science could help make better sense of the open data  provided by the Brazilian government.

Using agile methods, specifically Domain Driven Design, a flexible and adaptive process framework for solving complex problems, our group started studying the regulations, and converting them into  software code. We did this by reverse-engineering the legal documents–understanding the reimbursement rules and brainstorming ways to circumvent them. Next, we thought about the traces this circumvention would leave in the databases and developed a way to identify these traces using the existing data. The public expenses database included the images of the receipts used to claim reimbursements and we could see evidence of expenses, such as alcohol, which weren’t allowed to be paid with public money. We named our creation, Rosie.

This method of researching the regulations to then translate them into software in an agile way is called Domain-Driven Design. Used for complex systems, this useful approach analyzes the data and the sector as an ecosystem, and then uses observations and rapid prototyping to generate and test an evolving model. This is how Rosie works. Rosie sifts through the reported data and flags specific expenses made by representatives as “suspicious.” An example could be purchases that indicate the Congress member was in two locations on the same day and time.

After finding a suspicious transaction, Rosie then automatically tweets the results to both citizens and congress members.  She invites citizens to corroborate or dismiss the suspicions, while also inviting congress members to justify themselves.

Rosie isn’t working alone. Beyond translating the law into computer code, the group also created new interfaces to help citizens check up on Rosie’s suspicions. The same information that was spread in different places in official government websites was put together in a more intuitive, indexed and machine-readable platform. This platform is called Jarbas – its name was inspired by the AI system that controls Tony Stark’s mansion in Iron Man, J.A.R.V.I.S. (which has origins in the human “Jarbas”) – and it is a website and API (application programming interface) that helps citizens more easily navigate and browse data from different sources. Together, Rosie and Jarbas helps citizens use and interpret the data to decide whether there was a misuse of public funds. So far, Rosie has tweeted 967 times. She is particularly good at detecting overpriced meals. According to an open research, made by the group, since her introduction, members of Congress have reduced spending on meals by about ten percent….(More)”.