Many researchers say they’ll share data — but don’t

Curated on Posted on by Stefaan Verhulst

Article by Clare Watson: “Most biomedical and health researchers who declare their willingness to share the data behind journal articles do not respond to access requests or hand over the data when asked, a study reports1.

Livia Puljak, who studies evidence-based medicine at the Catholic University of Croatia in Zagreb, and her colleagues analysed 3,556 biomedical and health science articles published in a month by 282 BMC journals. (BMC is part of Springer Nature, the publisher of Nature; Nature’s news team is editorially independent of its publisher.)

The team identified 381 articles with links to data stored in online repositories and another 1,792 papers for which the authors indicated in statements that their data sets would be available on reasonable request. The remaining studies stated that their data were in the published manuscript and its supplements, or generated no data, so sharing did not apply.

But of the 1,792 manuscripts for which the authors stated they were willing to share their data, more than 90% of corresponding authors either declined or did not respond to requests for raw data (see ‘Data-sharing behaviour’). Only 14%, or 254, of the contacted authors responded to e-mail requests for data, and a mere 6.7%, or 120 authors, actually handed over the data in a usable format. The study was published in the Journal of Clinical Epidemiology on 29 May.

DATA-SHARING BEHAVIOUR. Graphic showing percentage of authors that were willing to share data.
Source: Livia Puljak et al

Puljak was “flabbergasted” that so few researchers actually shared their data. “There is a gap between what people say and what people do,” she says. “Only when we ask for the data can we see their attitude towards data sharing.”

“It’s quite dismaying that [researchers] are not coming forward with the data,” says Rebecca Li, who is executive director of non-profit global data-sharing platform Vivli and is based in Cambridge, Massachusetts…(More)”.

Societal Readiness Thinking Tool

Curated on Posted on by Stefaan Verhulst

About: “…The thinking tool offers practical guidance for researchers who wish to mature the societal readiness of their work. The primary goal is to help researchers align their project activities with societal needs and expectations. The thinking tool asks reflective questions to stimulate thinking about how to integrate ideas about responsible research and innovation  into research practice, at different stages in the project life. We have designed the tool so that it is useful for researchers engaged in new as well as ongoing projects. Some of the reflective questions used in the tool are adapted from other RRI projects. References for these projects and a detailed account of the tool’s underlying methodology is available  here.   If your project involves several researchers, we recommend that the full team is involved in using the Societal Readiness Thinking Tool together, and that you reserve sufficient time for discussions along the way. Ideally, the team would use the tool from the from the earliest phases of the project and return at later stages thougout the project life. You can learn more about the tool’s RRI terminology  here…(More)”.

Six Prescriptions for Applied Behavioral Science as It Comes of Age

Curated on Posted on by Stefaan Verhulst

Article by Dilip Soman and Nina Mažar: “…But it has now been over 14 years since the publication of Nudge and more than 10 years since the first behavioral unit in government started functioning. While we have made a lot of progress as a field, we believe that the applied science is at a critical juncture. Our efforts at this stage will determine whether the field matures in a systematic and stable manner, or grows wildly and erratically. Unless we take stock of the science, the practice, and the mechanisms that we can put into place to align the two, we will run the danger of the promise of behavioral science being an illusion for many—not because the science itself was faulty, but because we did not successfully develop a science for using the science.  

We offer six prescriptions for how the field of applied behavioral science can better align itself so that it grows in a systematic and not in a wild manner. 

1. Offer a balanced and nuanced view of the promise of behavioral science 

We believe that it is incumbent on leaders in both the academic and applied space to offer a balanced view of the promise of behavioral science. While we understand that the nature of the book publication process or of public lectures tends to skew on additives to highlight success, we also believe that it is perhaps more of a contribution for the field to highlight limitations and nuances. Rather than narratives along the lines of “A causes B,” it would be helpful for our leaders to highlight narratives such as “A causes B in some conditions and C in others.” Dissemination of this new narrative could take the form of traditional knowledge mobilization tools, such as books, popular press articles, interviews, podcasts, and essays. Our recent coedited book, Behavioral Science in the Wild, is one attempt at this.

2.Publish null and nonsurprising results 

Academic incentives usually create a body of work that (a) is replete with positive results, (b) overrepresents surprising results, (c) is not usually replicated, and (d) is focused on theory and phenomena and not on practical problems. As has been discussed elsewhere, this occurs because of the academic incentive structure, which favors surprising and positive results. We call on our field to change this culture by creating platforms that allow and encourage authors to publish null results, as well as unsurprising results…(More)”.

Use of science in public policy: Lessons from the COVID-19 pandemic efforts to ‘Follow the Science’

Curated on Posted on by Stefaan Verhulst

Paper by Barry Bozeman: “The paper asks: ‘What can we learn from COVID-19 pandemic about effective use of scientific and technical information (STI) in policymaking and how might the lessons be put to use?’ The paper employs the political rhetoric of ‘follow the science’ as a lens for examining contemporary concerns in the use of STI, including (1) ‘Breadth of Science Products’, the necessity of a broader concept of STI that includes by-products science, (2) ‘Science Dynamism’, emphasizing the uncertainty and impeachability of science, (3) ‘STI Urgency’ suggesting that STI use during widespread calamities differs from more routine applications, and (4) ‘Hyper-politicization of Science’, arguing that a step-change in the contentiousness of politics affects uses and misuses of STI. The paper concludes with a discussion, STI Curation, as a possible ingredient to improving effective use. With more attention to credibility and trust of STI and to the institutional legitimacy of curators, it should prove possible to improve the effective use of STI in public policy….(More)”.

Facial Expressions Do Not Reveal Emotions

Curated on Posted on by Stefaan Verhulst

Lisa Feldman Barrett at Scientific American: “Do your facial movements broadcast your emotions to other people? If you think the answer is yes, think again. This question is under contentious debate. Some experts maintain that people around the world make specific, recognizable faces that express certain emotions, such as smiling in happiness, scowling in anger and gasping with widened eyes in fear. They point to hundreds of studies that appear to demonstrate that smiles, frowns, and so on are universal facial expressions of emotion. They also often cite Charles Darwin’s 1872 book The Expression of the Emotions in Man and Animals to support the claim that universal expressions evolved by natural selection.

Other scientists point to a mountain of counterevidence showing that facial movements during emotions vary too widely to be universal beacons of emotional meaning. People may smile in hatred when plotting their enemy’s downfall and scowl in delight when they hear a bad pun. In Melanesian culture, a wide-eyed gasping face is a symbol of aggression, not fear. These experts say the alleged universal expressions just represent cultural stereotypes. To be clear, both sides in the debate acknowledge that facial movements vary for a given emotion; the disagreement is about whether there is enough uniformity to detect what someone is feeling.

This debate is not just academic; the outcome has serious consequences. Today you can be turned down for a job because a so-called emotion-reading system watching you on camera applied artificial intelligence to evaluate your facial movements unfavorably during an interview. In a U.S. court of law, a judge or jury may sometimes hand down a harsher sentence, even death, if they think a defendant’s face showed a lack of remorse. Children in preschools across the country are taught to recognize smiles as happiness, scowls as anger and other expressive stereotypes from books, games and posters of disembodied faces. And for children on the autism spectrum, some of whom have difficulty perceiving emotion in others, these teachings do not translate to better communication….Emotion AI systems, therefore, do not detect emotions. They detect physical signals, such as facial muscle movements, not the psychological meaning of those signals. The conflation of movement and meaning is deeply embedded in Western culture and in science. An example is a recent high-profile study that applied machine learning to more than six million internet videos of faces. The human raters, who trained the AI system, were asked to label facial movements in the videos, but the only labels they were given to use were emotion words, such as “angry,” rather than physical descriptions, such as “scowling.” Moreover there was no objective way to confirm what, if anything, the anonymous people in the videos were feeling in those moments…(More)”.

Open data: The building block of 21st century (open) science

Curated on Posted on by Stefaan Verhulst

Paper by Corina Pascu and Jean-Claude Burgelman: “Given this irreversibility of data driven and reproducible science and the role machines will play in that, it is foreseeable that the production of scientific knowledge will be more like a constant flow of updated data driven outputs, rather than a unique publication/article of some sort. Indeed, the future of scholarly publishing will be more based on the publication of data/insights with the article as a narrative.

For open data to be valuable, reproducibility is a sine qua non (King2011; Piwowar, Vision and Whitlock2011) and—equally important as most of the societal grand challenges require several sciences to work together—essential for interdisciplinarity.

This trend correlates with the already ongoing observed epistemic shift in the rationale of science: from demonstrating the absolute truth via a unique narrative (article or publication), to the best possible understanding what at that moment is needed to move forward in the production of knowledge to address problem “X” (de Regt2017).

Science in the 21st century will be thus be more “liquid,” enabled by open science and data practices and supported or even co-produced by artificial intelligence (AI) tools and services, and thus a continuous flow of knowledge produced and used by (mainly) machines and people. In this paradigm, an article will be the “atomic” entity and often the least important output of the knowledge stream and scholarship production. Publishing will offer in the first place a platform where all parts of the knowledge stream will be made available as such via peer review.

The new frontier in open science as well as where most of future revenue will be made, will be via value added data services (such as mining, intelligence, and networking) for people and machines. The use of AI is on the rise in society, but also on all aspects of research and science: what can be put in an algorithm will be put; the machines and deep learning add factor “X.”

AI services for science 4 are already being made along the research process: data discovery and analysis and knowledge extraction out of research artefacts are accelerated with the use of AI. AI technologies also help to maximize the efficiency of the publishing process and make peer-review more objective5 (Table 1).

Table 1. Examples of AI services for science already being developed

Abbreviation: AI, artificial intelligence.

Source: Authors’ research based on public sources, 2021.

Ultimately, actionable knowledge and translation of its benefits to society will be handled by humans in the “machine era” for decades to come. But as computers are indispensable research assistants, we need to make what we publish understandable to them.

The availability of data that are “FAIR by design” and shared Application Programming Interfaces (APIs) will allow new ways of collaboration between scientists and machines to make the best use of research digital objects of any kind. The more findable, accessible, interoperable, and reusable (FAIR) data resources will become available, the more it will be possible to use AI to extract and analyze new valuable information. The main challenge is to master the interoperability and quality of research data…(More)”.

Facebook-owner Meta to share more political ad targeting data

Curated on Posted on by Stefaan Verhulst

Article by Elizabeth Culliford: “Facebook owner Meta Platforms Inc (FB.O) will share more data on targeting choices made by advertisers running political and social-issue ads in its public ad database, it said on Monday.

Meta said it would also include detailed targeting information for these individual ads in its “Facebook Open Research and Transparency” database used by academic researchers, in an expansion of a pilot launched last year.

“Instead of analyzing how an ad was delivered by Facebook, it’s really going and looking at an advertiser strategy for what they were trying to do,” said Jeff King, Meta’s vice president of business integrity, in a phone interview.

The social media giant has faced pressure in recent years to provide transparency around targeted advertising on its platforms, particularly around elections. In 2018, it launched a public ad library, though some researchers criticized it for glitches and a lack of detailed targeting data.Meta said the ad library will soon show a summary of targeting information for social issue, electoral or political ads run by a page….The company has run various programs with external researchers as part of its transparency efforts. Last year, it said a technical error meant flawed data had been provided to academics in its “Social Science One” project…(More)”.

The Bare Minimum of Theory: A Definitional Definition for the Social Sciences

Curated on Posted on by Stefaan Verhulst

Paper by Chitu Okoli: “The ongoing debates in the information systems (IS) discipline on the nature of theory are implicitly rooted in different epistemologies of the social sciences and in a lack of consensus on a definition of theory. Thus, we focus here on the much-neglected topic of what constitutes the bare minimum of what can possibly be considered theory—only by carefully understanding the bare minimum can we really understand the essence of what makes a theory a theory. We definitionally define a theory in the social sciences as an explanation of the relationship between two or more measurable concepts. (“Measurable” refers to qualitative coding and inference of mechanisms, as well as quantitative magnitudes.) The rigorous justification of each element of this definition helps to resolve issues such as providing a consistent basis of determining what qualifies as theory; the value of other knowledge contributions that are not theory; how to identify theories regardless of if they are named; and a call to recognize diverse forms of theorizing across the social science epistemologies of positivism, interpretivism, critical social theory, critical realism, and pragmatism. Although focused on IS, most of these issues are pertinent to any scholarly discipline within the social sciences…(More)”.

A Computational Inflection for Scientific Discovery

Curated on Posted on by Stefaan Verhulst

Paper by Tom Hope, Doug Downey, Oren Etzioni, Daniel S. Weld, and Eric Horvitz: “We stand at the foot of a significant inflection in the trajectory of scientific discovery. As society continues on its fast-paced digital transformation, so does humankind’s collective scientific knowledge and discourse. We now read and write papers in digitized form, and a great deal of the formal and informal processes of science are captured digitally — including papers, preprints and books, code and datasets, conference presentations, and interactions in social networks and communication platforms. The transition has led to the growth of a tremendous amount of information, opening exciting opportunities for computational models and systems that analyze and harness it. In parallel, exponential growth in data processing power has fueled remarkable advances in AI, including self-supervised neural models capable of learning powerful representations from large-scale unstructured text without costly human supervision. The confluence of societal and computational trends suggests that computer science is poised to ignite a revolution in the scientific process itself.
However, the explosion of scientific data, results and publications stands in stark contrast to the constancy of human cognitive capacity. While scientific knowledge is expanding with rapidity, our minds have remained static, with severe limitations on the capacity for finding, assimilating and manipulating information. We propose a research agenda of task-guided knowledge retrieval, in which systems counter humans’ bounded capacity by ingesting corpora of scientific knowledge and retrieving inspirations, explanations, solutions and evidence synthesized to directly augment human performance on salient tasks in scientific endeavors. We present initial progress on methods and prototypes, and lay out important opportunities and challenges ahead with computational approaches that have the potential to revolutionize science…(More)”.

How does research data generate societal impact?

Curated on Posted on by Stefaan Verhulst

Blog by Eric Jensen and Mark Reed: “Managing data isn’t exciting and it can feel like a hassle to deposit data at the end of a project, when you want to focus on publishing your findings.

But if you want your research to have impact, paying attention to data could make a big difference, according to new research we published recently in the journal PLOS ONE.

We analysed case studies from the UK Research Excellence Framework (REF) exercise in 2014 to show how data analysis and curation can generate benefits for policy and practice, and sought to understand the pathways through which data typically leads to impact. In this series of blog posts we will unpack this research and show you how you can manage your data for impact.

We were commissioned by the Australian Research Data Commons (ARDC) to investigate how research data contributes to demonstrable non-academic benefits to society from research, drawing on existing impact case studies from the REF. We then analyzed case studies from the Australian Research Council (ARC) Engagement and Impact Assessment 2018, a similar exercise to the UK’s…

The most prevalent type of research data-driven impact was benefits for professional practice (45% UK; 44% Australia).

This category of impact includes changing the ways professionals operate and improving the quality of products or services through better methods, technologies, and responses to issues through better understanding. It also includes changing organisational culture and improving workplace productivity or outcomes.

Government impacts were the next most prevalent category identified in this research (21% UK; 20% Australia).

These impacts include the introduction of new policies and changes to existing policies, as well as

  • reducing the cost to deliver government services
  • enhancing the effectiveness or efficiency of government services and operations
  • more efficient government planning

Other relatively common types of research data-driven impacts were economic impact (13% UK; 14% Australia) and public health impacts (10% UK; 8% Australia)…(More)”.