Coronavirus: country comparisons are pointless unless we account for these biases in testing

Curated on Posted on by Stefaan Verhulst

Norman Fenton, Magda Osman, Martin Neil, and Scott McLachlan at The Conversation: “Suppose we wanted to estimate how many car owners there are in the UK and how many of those own a Ford Fiesta, but we only have data on those people who visited Ford car showrooms in the last year. If 10% of the showroom visitors owned a Fiesta, then, because of the bias in the sample, this would certainly overestimate the proportion of Ford Fiesta owners in the country.

Estimating death rates for people with COVID-19 is currently undertaken largely along the same lines. In the UK, for example, almost all testing of COVID-19 is performed on people already hospitalised with COVID-19 symptoms. At the time of writing, there are 29,474 confirmed COVID-19 cases (analogous to car owners visiting a showroom) of whom 2,352 have died (Ford Fiesta owners who visited a showroom). But it misses out all the people with mild or no symptoms.

Concluding that the death rate from COVID-19 is on average 8% (2,352 out of 29,474) ignores the many people with COVID-19 who are not hospitalised and have not died (analogous to car owners who did not visit a Ford showroom and who do not own a Ford Fiesta). It is therefore equivalent to making the mistake of concluding that 10% of all car owners own a Fiesta.

There are many prominent examples of this sort of conclusion. The Oxford COVID-19 Evidence Service have undertaken a thorough statistical analysis. They acknowledge potential selection bias, and add confidence intervals showing how big the error may be for the (potentially highly misleading) proportion of deaths among confirmed COVID-19 patients.

They note various factors that can result in wide national differences – for example the UK’s 8% (mean) “death rate” is very high compared to Germany’s 0.74%. These factors include different demographics, for example the number of elderly in a population, as well as how deaths are reported. For example, in some countries everybody who dies after having been diagnosed with COVID-19 is recorded as a COVID-19 death, even if the disease was not the actual cause, while other people may die from the virus without actually having been diagnosed with COVID-19.

However, the models fail to incorporate explicit causal explanations in their modelling that might enable us to make more meaningful inferences from the available data, including data on virus testing.

What a causal model would look like. Author provided

We have developed an initial prototype “causal model” whose structure is shown in the figure above. The links between the named variables in a model like this show how they are dependent on each other. These links, along with other unknown variables, are captured as probabilities. As data are entered for specific, known variables, all of the unknown variable probabilities are updated using a method called Bayesian inference. The model shows that the COVID-19 death rate is as much a function of sampling methods, testing and reporting, as it is determined by the underlying rate of infection in a vulnerable population….(More)”

A guide to healthy skepticism of artificial intelligence and coronavirus

Curated on Posted on by Stefaan Verhulst

Alex Engler at Brookings: “The COVID-19 outbreak has spurred considerable news coverage about the ways artificial intelligence (AI) can combat the pandemic’s spread. Unfortunately, much of it has failed to be appropriately skeptical about the claims of AI’s value. Like many tools, AI has a role to play, but its effect on the outbreak is probably small. While this may change in the future, technologies like data reporting, telemedicine, and conventional diagnostic tools are currently far more impactful than AI.

Still, various news articles have dramatized the role AI is playing in the pandemic by overstating what tasks it can perform, inflating its effectiveness and scale, neglecting the level of human involvement, and being careless in consideration of related risks. In fact, the COVID-19 AI-hype has been diverse enough to cover the greatest hits of exaggerated claims around AI. And so, framed around examples from the COVID-19 outbreak, here are eight considerations for a skeptic’s approach to AI claims….(More)”.

The Rules of Contagion: Why Things Spread–And Why They Stop

Curated on Posted on by Stefaan Verhulst

Book by Adam Kucharski: “From ideas and infections to financial crises and “fake news,” why the science of outbreaks is the science of modern life.


These days, whenever anything spreads, whether it’s a YouTube fad or a political rumor, we say it went viral. But how does virality actually work? In The Rules of Contagion, epidemiologist Adam Kucharski explores topics including gun violence, online manipulation, and, of course, outbreaks of disease to show how much we get wrong about contagion, and how astonishing the real science is.
Why did the president retweet a Mussolini quote as his own? Why do financial bubbles take off so quickly? Why are disinformation campaigns so effective? And what makes the emergence of new illnesses–such as MERS, SARS, or the coronavirus disease COVID-19–so challenging? By uncovering the crucial factors driving outbreaks, we can see how things really spread — and what we can do about it….(More)”.

Mobile phone data and COVID-19: Missing an opportunity?

Curated on Posted on by Stefaan Verhulst

Paper by Nuria Oliver, et al: “This paper describes how mobile phone data can guide government and public health authorities in determining the best course of action to control the COVID-19 pandemic and in assessing the effectiveness of control measures such as physical distancing. It identifies key gaps and reasons why this kind of data is only scarcely used, although their value in similar epidemics has proven in a number of use cases. It presents ways to overcome these gaps and key recommendations for urgent action, most notably the establishment of mixed expert groups on national and regional level, and the inclusion and support of governments and public authorities early on. It is authored by a group of experienced data scientists, epidemiologists, demographers and representatives of mobile network operators who jointly put their work at the service of the global effort to combat the COVID-19 pandemic….(More)”.

Collective Intelligence at EU Level – Social and Democratic Dimensions

Curated on Posted on by Stefaan Verhulst

Paper by Nora Milotay and Gianluca Sgueo: “Humans are among the many living species capable of collaborative and imaginative thinking. While it is widely agreed among scholars that this capacity has contributed to making humans the dominant species, other crucial questions remain open to debate. Is it possible to encourage large groups of people to engage in collective thinking? Is it possible to coordinate citizens to find solutions to address global challenges? Some scholars claim that large groups of independent, motivated, and well-informed people can, collectively, make better decisions than isolated individuals can – what is known as ‘collective intelligence.’

The social dimension of collective intelligence mainly relates to social aspects of the economy and of innovation. It shows that a holistic approach to innovation – one that includes not only technological but also social aspects – can greatly contribute to the EU’s goal of promoting a just transition for everyone to a sustainable and green economy in the digital age. The EU has been taking concrete action to promote social innovation by supporting the development of its theory and practice. Mainly through funding programmes, it helps to seek new types of partners and build new capacity – and thus shape the future of local and national innovations aimed at societal needs.

The democratic dimension suggests that the power of the collective can be leveraged so as to improve public decision-making systems. Supported by technology, policy-makers can harness the ‘civic surplus’ of citizens – thus providing smarter solutions to regulatory challenges. This is particularly relevant at EU level in view of the planned Conference on the Future of Europe, aimed at engaging communities at large and making EU decision-making more inclusive and participatory.

The current coronavirus crisis is likely to change society and our economy in ways as yet too early to predict, but recovery after the crisis will require new ways of thinking and acting to overcome common challenges, and thus making use of our collective intelligence should be more urgent than ever. In the longer term, in order to mobilise collective intelligence across the EU and to fully exploit its innovative potential, the EU needs to strengthen its education policies and promote a shared understanding of a holistic approach to innovation and of collective intelligence – and thus become a ‘global brain,’ with a solid institutional set-up at the centre of a subsidised experimentation process that meets the challenges imposed by modernday transformations…(More)”.

A Closer Look at Location Data: Privacy and Pandemics

Curated on Posted on by Stefaan Verhulst

Assessment by Stacey Gray: “In light of COVID-19, there is heightened global interest in harnessing location data held by major tech companies to track individuals affected by the virus, better understand the effectiveness of social distancing, or send alerts to individuals who might be affected based on their previous proximity to known cases. Governments around the world are considering whether and how to use mobile location data to help contain the virus: Israel’s government passed emergency regulations to address the crisis using cell phone location data; the European Commission requested that mobile carriers provide anonymized and aggregate mobile location data; and South Korea has created a publicly available map of location data from individuals who have tested positive. 

Public health agencies and epidemiologists have long been interested in analyzing device location data to track diseases. In general, the movement of devices effectively mirrors movement of people (with some exceptions discussed below). However, its use comes with a range of ethical and privacy concerns. 

In order to help policymakers address these concerns, we provide below a brief explainer guide of the basics: (1) what is location data, (2) who holds it, and (3) how is it collected? Finally we discuss some preliminary ethical and privacy considerations for processing location data. Researchers and agencies should consider: how and in what context location data was collected; the fact and reasoning behind location data being classified as legally “sensitive” in most jurisdictions; challenges to effective “anonymization”; representativeness of the location dataset (taking into account potential bias and lack of inclusion of low-income and elderly subpopulations who do not own phones); and the unique importance of purpose limitation, or not re-using location data for other civil or law enforcement purposes after the pandemic is over….(More)”.

Human migration: the big data perspective

Curated on Posted on by Stefaan Verhulst

Alina Sîrbu et al at the International Journal of Data Science and Analytics: “How can big data help to understand the migration phenomenon? In this paper, we try to answer this question through an analysis of various phases of migration, comparing traditional and novel data sources and models at each phase. We concentrate on three phases of migration, at each phase describing the state of the art and recent developments and ideas. The first phase includes the journey, and we study migration flows and stocks, providing examples where big data can have an impact. The second phase discusses the stay, i.e. migrant integration in the destination country. We explore various data sets and models that can be used to quantify and understand migrant integration, with the final aim of providing the basis for the construction of a novel multi-level integration index. The last phase is related to the effects of migration on the source countries and the return of migrants….(More)”.

A controlled trial for reproducibility

Curated on Posted on by Stefaan Verhulst

Marc P. Raphael, Paul E. Sheehan & Gary J. Vora at Nature: “In 2016, the US Defense Advanced Research Projects Agency (DARPA) told eight research groups that their proposals had made it through the review gauntlet and would soon get a few million dollars from its Biological Technologies Office (BTO). Along with congratulations, the teams received a reminder that their award came with an unusual requirement — an independent shadow team of scientists tasked with reproducing their results.

Thus began an intense, multi-year controlled trial in reproducibility. Each shadow team consists of three to five researchers, who visit the ‘performer’ team’s laboratory and often host visits themselves. Between 3% and 8% of the programme’s total funds go to this independent validation and verification (IV&V) work. But DARPA has the flexibility and resources for such herculean efforts to assess essential techniques. In one unusual instance, an IV&V laboratory needed a sophisticated US$200,000 microscopy and microfluidic set-up to make an accurate assessment.

These costs are high, but we think they are an essential investment to avoid wasting taxpayers’ money and to advance fundamental research towards beneficial applications. Here, we outline what we’ve learnt from implementing this programme, and how it could be applied more broadly….(More)”.

COVID-19 is creating a democratic deficit – here’s how to reduce it

Curated on Posted on by Stefaan Verhulst

Article by Matt Ryan: “As parliaments around the country move to scale down operations and defer sittings as part of containing COVID-19 people are beginning to ring the accountability alarm bells….

The good news is that we can learn from those parliaments and politicians around the world who have already been trialling new ways of working that go beyond traditional sittings. Leveraging simple and widely available technologies, they are involving more people with more diverse backgrounds in their processes with less reliance on those people being physically present.

Select Committees in the UK Parliament, for example, have used online “evidence checks” to scrutinise the basis for policy. These one-month exercises use targeted outreach and social media strategies to invite comments from knowledgeable stakeholders and members of the public about the rigour of evidence on which a government department’s policy is based. Evidence for departmental policy is summarised in a two-page document and comments publicly displayed in a web forum that resembles a readers’ comments section in an online news article.

In Taiwan, a participatory governance process pioneered by civic rights activists at the behest of a government minister combines large-scale online participation with smaller in-person gatherings to build a “rough consensus” on legislative proposals related to the digital economy before they are introduced. Known as vTaiwan, the process has led to 26 pieces of national legislation dealing with issues such as Uber, telemedicine and online alcohol sales, and has involved 200,000 people.

The government of Mexico City has raised the stakes even higher, involving more than 400,000 people in a process to draft a new constitution. It included a novel partnership between Change.org and the city mayor that enabled residents to create petition-backed proposals which, once they reached a certain threshold of support, bound the mayor to include them in the draft he submitted to a special constitutional assembly.

Processes like these can also offer relief for politicians and parliamentary officials managing the strain of examining an ever-increasing number of issues of greater complexity with limited personnel and budget. Evidence checks provide access to a wider pool of experts who can bolster existing research capacity. vTaiwan helps to find workable ways forward in industries being rapidly transformed by digital technologies. By “crowdsourcing” the city’s constitution, Mexico City’s mayor retained the trust of residents while undertaking reform at a grand scale….(More)”.

Ask a Scientist

Curated on Posted on by Stefaan Verhulst

NYU Press Release: “Unreliable tips on how to protect oneself from the novel coronavirus and fake news about the COVID-19 pandemic are spreading as quickly as the virus itself.

The Governance Lab (The GovLab) at the New York University Tandon School of Engineering has collaborated with the Federation of American Scientists (FAS) and the State of New Jersey Office of Innovation to launch a free, interactive tool aimed at cutting through the noise and presenting clear, scientist-led, and evidence-based information and advice to the public.

Available in English and Spanish, “Ask a Scientist,” allows users to find answers to a wide range of commonly asked questions about the virus, the severity of the outbreak, best methods of prevention, and steps to take in the event you fall ill. All posted content is obtained from the World Health Organization, the Centers for Disease Control and Prevention, and other rigorously verified sources.

screenshot of website that allows users to type in questions about COVID-19

“Ask a Scientist” features a free, interactive tool allowing users to submit questions to a team of FAS researchers and a crowdsourced network of vetted science experts. In English and Spanish, the site also includes top articles and the latest information, and answers to a wide range of commonly asked questions about the COVID-19 epidemic, the severity of the outbreak, best methods of prevention, and steps to take in the event you fall ill.

If users do not find an answer to their specific questions, they have the option of submitting them to a team of FAS researchers and a crowdsourced network of vetted science experts led by the National Science Policy Network. Users can expect an answer within an hour, although that timeframe is expected to shorten as the network increases in size. Every answer is reviewed to ensure accuracy and timeliness, then added to the knowledge base for the benefit of others….(More)”.