How Smart Are the Robots Getting?

Curated on Posted on by Stefaan Verhulst

Cade Metz at The New York Times: “…These are not systems that anyone can properly evaluate with the Turing test — or any other simple method. Their end goal is not conversation.

Researchers at Google and DeepMind, which is owned by Google’s parent company, are developing tests meant to evaluate chatbots and systems like DALL-E, to judge what they do well, where they lack reason and common sense, and more. One test shows videos to artificial intelligence systems and asks them to explain what has happened. After watching someone tinker with an electric shaver, for instance, the A.I. must explain why the shaver did not turn on.

These tests feel like academic exercises — much like the Turing test. We need something that is more practical, that can really tell us what these systems do well and what they cannot, how they will replace human labor in the near term and how they will not.

We could also use a change in attitude. “We need a paradigm shift — where we no longer judge intelligence by comparing machines to human behavior,” said Oren Etzioni, professor emeritus at the University of Washington and founding chief executive of the Allen Institute for AI, a prominent lab in Seattle….

At the same time, there are many ways these bots are superior to you and me. They do not get tired. They do not let emotion cloud what they are trying to do. They can instantly draw on far larger amounts of information. And they can generate text, images and other media at speeds and volumes we humans never could.

Their skills will also improve considerably in the coming years.

Researchers can rapidly hone these systems by feeding them more and more data. The most advanced systems, like ChatGPT, require months of training, but over those months, they can develop skills they did not exhibit in the past.

“We have found a set of techniques that scale effortlessly,” said Raia Hadsell, senior director of research and robotics at DeepMind. “We have a simple, powerful approach that continues to get better and better.”

The exponential improvement we have seen in these chatbots over the past few years will not last forever. The gains may soon level out. But even then, multimodal systems will continue to improve — and master increasingly complex skills involving images, sounds and computer code. And computer scientists will combine these bots with systems that can do things they cannot. ChatGPT failed Turing’s chess test. But we knew in 1997 that a computer could beat the best humans at chess. Plug ChatGPT into a chess program, and the hole is filled.

In the months and years to come, these bots will help you find information on the internet. They will explain concepts in ways you can understand. If you like, they will even write your tweets, blog posts and term papers.

They will tabulate your monthly expenses in your spreadsheets. They will visit real estate websites and find houses in your price range. They will produce online avatars that look and sound like humans. They will make mini-movies, complete with music and dialogue…

Certainly, these bots will change the world. But the onus is on you to be wary of what these systems say and do, to edit what they give you, to approach everything you see online with skepticism. Researchers know how to give these systems a wide range of skills, but they do not yet know how to give them reason or common sense or a sense of truth.

That still lies with you…(More)”.

Why Europe must embrace participatory policymaking

Curated on Posted on by Stefaan Verhulst

Article by Alberto Alemanno, Claire Davenport, and Laura Batalla: “Today, Europe faces many threats – from economic uncertainty and war on its eastern borders to the rise of illiberal democracies and popular reactionary politicians.

As Europe recovers from the pandemic and grapples with economic and social unrest, it is at an inflection point; it can either create new spaces to build trust and a sense of shared purpose between citizens and governments, or it can continue to let its democratic institutions erode and distrust grow. 

The scale of such problems requires novel problem-solving and new perspectives, including those from civil society and citizens. Increased opportunities for citizens to engage with policymakers can lend legitimacy and accountability to traditionally ‘opaque’ policymaking processes. The future of the bloc hinges on its ability to not only sustain democratic institutions but to do so with buy-in from constituents.

Yet policymaking in the EU is often understood as a technocratic process that the public finds difficult, if not impossible, to navigate. The Spring 2022 Eurobarometer found that just 53% of respondents believed their voice counts in the EU. The issue is compounded by a lack of political literacy coupled with a dearth of channels for participation or co-creation. 

In parallel, there is a strong desire from citizens to make their voices heard. A January 2022 Special Eurobarometer on the Future of Europe found that 90% of respondents agreed that EU citizens’ voices should be taken more into account during decision-making. The Russian war in Ukraine has strengthened public support for the EU as a whole. According to the Spring 2022 Eurobarometer, 65% of Europeans view EU membership as a good thing. 

This is not to say that the EU has no existing models for citizen engagement. The European Citizens Initiative – a mechanism for petitioning the Commission to propose new laws – is one example of existing infrastructure. There is also an opportunity to build on the success of The Conference on the Future of Europe, a gathering held this past spring that gave citizens the opportunity to contribute policy recommendations and justifications alongside traditional EU policymakers…(More)”

The Autocrat in Your iPhone

Curated on Posted on by Stefaan Verhulst

Article by Ronald J. Deibert: “In the summer of 2020, a Rwandan plot to capture exiled opposition leader Paul Rusesabagina drew international headlines. Rusesabagina is best known as the human rights defender and U.S. Presidential Medal of Freedom recipient who sheltered more than 1,200 Hutus and Tutsis in a hotel during the 1994 Rwandan genocide. But in the decades after the genocide, he also became a prominent U.S.-based critic of Rwandan President Paul Kagame. In August 2020, during a layover in Dubai, Rusesabagina was lured under false pretenses into boarding a plane bound for Kigali, the Rwandan capital, where government authorities immediately arrested him for his affiliation with an opposition group. The following year, a Rwandan court sentenced him to 25 years in prison, drawing the condemnation of international human rights groups, the European Parliament, and the U.S. Congress. 

Less noted at the time, however, was that this brazen cross-border operation may also have employed highly sophisticated digital surveillance. After Rusesabagina’s sentencing, Amnesty International and the Citizen Lab at the University of Toronto, a digital security research group I founded and direct, discovered that smartphones belonging to several of Rusesabagina’s family members who also lived abroad had been hacked by an advanced spyware program called Pegasus. Produced by the Israel-based NSO Group, Pegasus gives an operator near-total access to a target’s personal data. Forensic analysis revealed that the phone belonging to Rusesabagina’s daughter Carine Kanimba had been infected by the spyware around the time her father was kidnapped and again when she was trying to secure his release and was meeting with high-level officials in Europe and the U.S. State Department, including the U.S. special envoy for hostage affairs. NSO Group does not publicly identify its government clients and the Rwandan government has denied using Pegasus, but strong circumstantial evidence points to the Kagame regime.

In fact, the incident is only one of dozens of cases in which Pegasus or other similar spyware technology has been found on the digital devices of prominent political opposition figures, journalists, and human rights activists in many countries. Providing the ability to clandestinely infiltrate even the most up-to-date smartphones—the latest “zero click” version of the spyware can penetrate a device without any action by the user—Pegasus has become the digital surveillance tool of choice for repressive regimes around the world. It has been used against government critics in the United Arab Emirates (UAE) and pro-democracy protesters in Thailand. It has been deployed by Mohammed bin Salman’s Saudi Arabia and Viktor Orban’s Hungary…(More)”.

Recentring the demos in the measurement of democracy

Curated on Posted on by Stefaan Verhulst

Article by Seema Shah: “Rethinking how we measure and evaluate democratic performance is vital to reversing a longstanding negative trend in global democracy. We must confront the past, including democracy’s counter-intuitively intrinsic inequality. This is key to revitalising institutions in a way that allows democratic practice to live up to its potential…

In the global democracy assessment space, teams like the one I lead at International IDEA compete to provide the most rigorous, far-reaching and understandable set of democracy measurements in the world. Alexander Hudson explains how critical these indicators are, providing important benchmarks for democratic growth and decline to policymakers, governments, international organisations, and journalists.

Yet in so many ways, the core of what these datasets measure and help assess are largely the same. This redundancy is no doubt at least partially a product of wealthy donors’ prioritisation of liberal democracy as an ideal. It is compounded by how the measures are calculated. As Adam Przeworksi recently stated, reliance on expert coders runs the risk of measuring little other than those experts’ biases.

But if that is the case, and quantitative measurements continue to be necessary for democracy assessment, shouldn’t we rethink exactly what we are measuring and how we are measuring it?..

Democracy assessment indices do not typically measure ordinary people’s evaluations of the state of democracy. Instead, other specialised ‘barometers’ often take on this task. See, for example, Afrobarometer, Eurobarometer, Asian Barometer, and Latinobarometro. Surveys of public perceptions on a range of issues also exist, including, but not limited to democracy. The problem is, however, that these do not systematically make it into overall democracy assessments or onto policymakers’ desks. This means that policymakers and others do not consistently prioritise or consider lived experiences as they make decisions about democracy and human rights-related funding and interventions…(More)”.

How games can make behavioural science better

Curated on Posted on by Stefaan Verhulst

Article by Bria Long et al: “When US cognitive scientist Joshua Hartshorne was investigating how people around the world learn English, he needed to get tens of thousands of people to take a language test. He designed ‘Which English?’, a grammar game that presented a series of tough word problems and then guessed where in the world the player learnt the language. Participants shared their results — whether accurate or not — on social media, creating a snowball effect for recruitment. The findings, based on data from almost 670,000 people, revealed that there is a ‘critical period’ for second-language learning that extends into adolescence.

This sort of ‘gamification’ is becoming a powerful research tool across fields that study humans, including psychology, neuroscience, economics and behavioural economics. By making research fun, the approach can help experiments to reach thousands or millions of participants. For instance, experiments embedded in a video game demonstrated that the layout of the city where a child lives shapes their future navigational ability. Data from a digital word search showed that people who are skilled at the game do not necessarily give better advice to those trying to learn it. And a dilemma game involving millions of people revealed that most individuals have reliable moral intuition.

Gamification can help to avoid the pitfalls of conventional laboratory-based experiments by allowing researchers to study diverse populations, to conduct more-sophisticated experiments and to observe human behaviour in naturalistic environments. It can improve statistical power and reproducibility, making research more robust. Technical advances are making gamification cheaper and more straightforward, and the COVID-19 pandemic has forced many labs to move their human experiments online. But despite these changes, most have not yet embraced the opportunities gamification affords.

To reach the full potential of this approach, researchers must dispel misconceptions, develop new gamification technologies, improve access to existing ones and apply the methods to productive research questions. We are researchers in psychology, linguistics, developmental science, data science and music who have run our own gamified experiments. We think it’s time for science to get serious about games…(More)”.

Building Trust with the Algorithms in Our Lives

Curated on Posted on by Stefaan Verhulst

Essay by Dylan Walsh: “Algorithms are omnipresent in our increasingly digital lives. They offer us new music and friends. They recommend books and clothing. They deliver information about the world. They help us find romantic partners one day, efficient commutes the next, cancer diagnoses the third.

And yet most people display an aversion to algorithms. They don’t fully trust the recommendations made by computer programs. When asked, they prefer human predictions to those put forward by algorithms.

“But given the growing prevalence of algorithms, it seems important we learn to trust and appreciate them,” says Taly Reich, associate professor at Yale SOM. “Is there an intervention that would help reduce this aversion?”

New research conducted by Reich and two colleagues, Alex Kaju of HEC Montreal and Sam Maglio of the University of Toronto, finds that clearly demonstrating an algorithm’s ability to learn from past mistakes increases the trust that people place in the algorithm. It also inclines people to prefer the predictions made by algorithms over those made by humans.

In arriving at this result, Reich drew on her foundational work on the value of mistakes. In a series of prior papers, Reich has established how mistakes, in the right context, can create benefits; people who make mistakes can come across as more knowledgeable and credible than people who don’t. Applying this insight to predictive models, Reich and her colleagues investigated whether framing algorithms as capable of learning from their mistakes enhanced trust in the recommendations that algorithms make.

In one of several experiments, for instance, participants were asked whether a trained psychologist or an algorithm would be better at evaluating somebody’s personality. Under one condition, no further information was provided. In another condition, identical performance data for both the psychologist and the algorithm explicitly demonstrated improvement over time. In the first three months, each one was correct 60% of the time, incorrect 40% of the time; by six months, they were correct 70% of the time; and over the course of the first year the rate moved up to 80% correct.

Absent information about the capacity to learn, participants chose a psychologist over an algorithm 75% of the time. But when shown how the algorithm improved over time, they chose it 66% of the time—more often than the human. Participants overcame any potential algorithm aversion and instead expressed what Reich and her colleagues term “algorithm appreciation,” or even “algorithm investment,” by choosing it at a higher rate than the human. These results held across several different cases, from selecting the best artwork to finding a well-matched romantic partner. In every instance, when the algorithm exhibited learning over time, it was trusted more often than its human counterpart…(More)”

Here’s how the agricultural sector can solve its data problem

Curated on Posted on by Stefaan Verhulst

Article by Satyanarayana Jeedigunta and Arushi Goel: “Food and nutrition security, skewed distribution of farmer incomes, natural disasters and climate change are severely impacting the sustainability of agricultural systems across the globe. Policy reforms are needed to correct these distortions, but innovative emerging technologies like artificial intelligence, machine learning, distributed ledger technologies, sensors and drones, can make a significant difference.

Emerging technologies need data, and it must be the right data, for the right purpose at the right time. This is how it can deliver maximum impact. Agricultural value chains comprise a complex system of stakeholders and activities. The enormity of the size and complexity of agricultural data, coupled with its fragmented nature, pose significant challenges to unlocking its potential economic value, estimated at $65 billion in India alone….

As such, there is a need to promote standards-based interoperability, which enables multiple digital systems to exchange agricultural data in an automated manner with limited human intervention. The ease and speed of such an exchange of data, across domains and technologies, would spur the development of innovative solutions and lead to evidence-driven, prediction-based decision-making on the farm and in the market.

Most agricultural data is dynamic

Most current efforts to develop standards of agriculture data are isolated and localized. The AGROVOC initiative of the United Nations’ Food and Agriculture Organization addresses a part of the data problem by creating an exhaustive vocabulary of agricultural terms. There is also a need to develop an open data format for the automated interchange of agriculture data. A coordinated initiative of the industry is an attractive approach to develop such a format…(More)”.

Database States

Curated on Posted on by Stefaan Verhulst

Essay by Sanjana Varghese: “In early 2007, a package sent from the north of England to the National Audit Office (NAO) in London went missing. In it were two discs containing the personal records of twenty-five million people—including their addresses, birthdays, and national insurance numbers, which are required to work in the UK—that the NAO intended to use for an “independent survey” of the child benefits database to check for supposed fraud. Instead, that information was never recovered, a national scandal ensued, and the junior official who mailed the package was fired.

The UK, as it turns out, is not particularly adept at securing its data. In 2009, a group of British academics released a report calling the UK a “database state,” citing the existence of forty-six leaky databases that were poorly constructed and badly maintained. Databases that they examined ranged from one on childhood obesity rates (which recorded the height and weight measurements of every school pupil in the UK between the ages of five and eleven) to IDENT1, a police database containing the fingerprints of all known offenders. “In too many cases,” the researchers wrote, “the public are neither served nor protected by the increasingly complex and intrusive holdings of personal information, invading every aspect of our lives.”

In the years since, databases in the UK—and elsewhere—have only proliferated; increasingly manufactured and maintained by a nexus of private actors and state agencies, they are generated by and produce more and more information streams that inevitably have a material effect on the populations they’re used by and against. More than just a neutral method of storing information, databases shape and reshape the world around us; they aid and abet the state and private industry in matters of surveillance, police violence, environmental destruction, border enforcement, and more…(More)”.

Government must earn public trust that AI is being used safely and responsibly

Curated on Posted on by Stefaan Verhulst

Article by Sue Bateman and Felicity Burch: “Algorithms have the potential to improve so much of what we do in the public sector, from the delivery of frontline public services to informing policy development across every sector. From first responders to first permanent secretaries, artificial intelligence has the potential to enable individuals to make better and more informed decisions.

In order to realise that potential over the long term, however, it is vital that we earn the public’s trust that AI is being used in a way that is safe and responsible.

One way to build that trust is transparency. That is why today, we’re delighted to announce the launch of the Algorithmic Transparency Recording Standard (the Standard), a world-leading, simple and clear format to help public sector organisations to record the algorithmic tools they use. The Standard has been endorsed by the Data Standards Authority, which recommends the standards, guidance and other resources government departments should follow when working on data projects.

Enabling transparent public sector use of algorithms and AI is vital for a number of reasons. 

Firstly, transparency can support innovation in organisations, whether that is helping senior leaders to engage with how their teams are using AI, sharing best practice across organisations or even just doing both of those things better or more consistently than done previously. The Information Commissioner’s Office took part in the piloting of the Standard and they have noted how it “encourages different parts of an organisation to work together and consider ethical aspects from a range of perspective”, as well as how it “helps different teams… within an organisation – who may not typically work together – learn about each other’s work”.

Secondly, transparency can help to improve engagement with the public, and reduce the risk of people opting out of services – where that is an option. If a significant proportion of the public opt out, this can mean that the information the algorithms use is not representative of the wider public and risks perpetuating bias. Transparency can also facilitate greater accountability: enabling citizens to understand or, if necessary, challenge a decision.

Finally, transparency is a gateway to enabling other goals in data ethics that increase justified public trust in algorithms and AI. 

For example, the team at The National Archives described the benefit of using the Standard as a “checklist of things to think about” when procuring algorithmic systems, and the Thames Valley Police team who piloted the Standard emphasised how transparency could “prompt the development of more understandable models”…(More)”.

Kid-edited journal pushes scientists for clear writing on complex topics

Curated on Posted on by Stefaan Verhulst

Article by Mark Johnson: “The reviewer was not impressed with the paper written by Israeli brain researcher Idan Segev and a colleague from Switzerland.

“Professor Idan,” she wrote to Segev. “I didn’t understand anything that you said.”

Segev and co-author Felix Schürmann revised their paper on the Human Brain project, a massive effort seeking to channel all that we know about the mind into a vast computer model. But once again the reviewer sent it back. Still not clear enough. It took a third version to satisfy the reviewer.

“Okay,” said the reviewer, an 11-year-old girl from New York named Abby. “Now I understand.”

Such is the stringent editing process at the online science journal Frontiers for Young Minds, where top scientists, some of them Nobel Prize winners, submit papers on gene-editing, gravitational waves and other topics — to demanding reviewers ages 8 through 15.

Launched in 2013, the Lausanne, Switzerland-based publication is coming of age at a moment when skeptical members of the public look to scientists for clear guidance on the coronavirus and on potentially catastrophic climate change, among other issues. At Frontiers for Young Minds, the goal is not just to publish science papers but also to make them accessible to young readers like the reviewers. In doing so, it takes direct aim at a long-standing problem in science — poor communication between professionals and the public.

“Scientists tend to default to their own jargon and don’t think carefully about whether this is a word that the public actually knows,” said Jon Lorsch, director of the National Institute of General Medical Sciences. “Sometimes to actually explain something you need a sentence as opposed to the one word scientists are using.”

Dense language sends a message “that science is for scientists; that you have to be an ‘intellectual’ to read and understand scientific literature; and that science is not relevant or important for everyday life,” according to a paper published last year in Advances in Physiology Education.

Frontiers for Young Minds, which has drawn nearly 30 million online page views in its nine years, offers a different message on its homepage: “Science for kids, edited by kids.”..(More)”.