Paper by David Leslie: “In the current hype-laden climate surrounding the rapid proliferation of foundation models and generative AI systems like ChatGPT, it is becoming increasingly important for societal stakeholders to reach sound understandings of their limitations and potential transformative effects. This is especially true in the natural and applied sciences, where magical thinking among some scientists about the take-off of “artificial general intelligence” has arisen simultaneously as the growing use of these technologies is putting longstanding norms, policies, and standards of good research practice under pressure. In this analysis, I argue that a deflationary understanding of foundation models and generative AI systems can help us sense check our expectations of what role they can play in processes of scientific exploration, sense-making, and discovery. I claim that a more sober, tool-based understanding of generative AI systems as computational instruments embedded in warm-blooded research processes can serve several salutary functions. It can play a crucial bubble-bursting role that mitigates some of the most serious threats to the ethos of modern science posed by an unreflective overreliance on these technologies. It can also strengthen the epistemic and normative footing of contemporary science by helping researchers circumscribe the part to be played by machine-led prediction in communicative contexts of scientific discovery while concurrently prodding them to recognise that such contexts are principal sites for human empowerment, democratic agency, and creativity. Finally, it can help spur ever richer approaches to collaborative experimental design, theory-construction, and scientific world-making by encouraging researchers to deploy these kinds of computational tools to heuristically probe unbounded search spaces and patterns in high-dimensional biophysical data that would otherwise be inaccessible to human-scale examination and inference…(More)”.
Open-access reformers launch next bold publishing plan
Article by Layal Liverpool: “The group behind the radical open-access initiative Plan S has announced its next big plan to shake up research publishing — and this one could be bolder than the first. It wants all versions of an article and its associated peer-review reports to be published openly from the outset, without authors paying any fees, and for authors, rather than publishers, to decide when and where to first publish their work.
The group of influential funding agencies, called cOAlition S, has over the past five years already caused upheaval in the scholarly publishing world by pressuring more journals to allow immediate open-access publishing. Its new proposal, prepared by a working group of publishing specialists and released on 31 October, puts forward an even broader transformation in the dissemination of research.
It outlines a future “community-based” and “scholar-led” open-research communication system (see go.nature.com/45zyjh) in which publishers are no longer gatekeepers that reject submitted work or determine first publication dates. Instead, authors would decide when and where to publish the initial accounts of their findings, both before and after peer review. Publishers would become service providers, paid to conduct processes such as copy-editing, typesetting and handling manuscript submissions…(More)”.
Can Indigenous knowledge and Western science work together? New center bets yes
Article by Jeffrey Mervis: “For millennia, the Passamaquoddy people used their intimate understanding of the coastal waters along the Gulf of Maine to sustainably harvest the ocean’s bounty. Anthropologist Darren Ranco of the University of Maine hoped to blend their knowledge of tides, water temperatures, salinity, and more with a Western approach in a project to study the impact of coastal pollution on fish, shellfish, and beaches.
But the Passamaquoddy were never really given a seat at the table, says Ranco, a member of the Penobscot Nation, which along with the Passamaquoddy are part of the Wabanaki Confederacy of tribes in Maine and eastern Canada. The Passamaquoddy thought water quality and environmental protection should be top priority; the state emphasized forecasting models and monitoring. “There was a disconnect over who were the decision-makers, what knowledge would be used in making decisions, and what participation should look like,” Ranco says about the 3-year project, begun in 2015 and funded by the National Science Foundation (NSF).
Last month, NSF aimed to bridge such disconnects, with a 5-year, $30 million grant designed to weave together traditional ecological knowledge (TEK) and Western science. Based at the University of Massachusetts (UMass) Amherst, the Center for Braiding Indigenous Knowledges and Science (CBIKS) aims to fundamentally change the way scholars from both traditions select and carry out joint research projects and manage data…(More)”.
How ChatGPT and other AI tools could disrupt scientific publishing
Article by Gemma Conroy: “When radiologist Domenico Mastrodicasa finds himself stuck while writing a research paper, he turns to ChatGPT, the chatbot that produces fluent responses to almost any query in seconds. “I use it as a sounding board,” says Mastrodicasa, who is based at the University of Washington School of Medicine in Seattle. “I can produce a publication-ready manuscript much faster.”
Mastrodicasa is one of many researchers experimenting with generative artificial-intelligence (AI) tools to write text or code. He pays for ChatGPT Plus, the subscription version of the bot based on the large language model (LLM) GPT-4, and uses it a few times a week. He finds it particularly useful for suggesting clearer ways to convey his ideas. Although a Nature survey suggests that scientists who use LLMs regularly are still in the minority, many expect that generative AI tools will become regular assistants for writing manuscripts, peer-review reports and grant applications.
Those are just some of the ways in which AI could transform scientific communication and publishing. Science publishers are already experimenting with generative AI in scientific search tools and for editing and quickly summarizing papers. Many researchers think that non-native English speakers could benefit most from these tools. Some see generative AI as a way for scientists to rethink how they interrogate and summarize experimental results altogether — they could use LLMs to do much of this work, meaning less time writing papers and more time doing experiments…(More)”.
Seven routes to experimentation in policymaking: a guide to applied behavioural science methods
OECD Resource: “…offers guidelines and a visual roadmap to help policymakers choose the most fit-for-purpose evidence collection method for their specific policy challenge.
The seven applied behavioural science methods:
- Randomised Controlled Trials (RCTs) are experiments that can demonstrate a causal relationship between an intervention and an outcome, by randomly assigning individuals to an intervention group and a control group.
- A/B testing tests two or more manipulations (such as variants of a webpage) to assess which performs better in terms of a specific goal or metric.
- Difference-in-Difference is an experimental method that estimates the causal effect of an intervention by comparing changes in outcomes between an intervention group and a control group before and after the intervention.
- Before-After studies assess the impact of an intervention or event by comparing outcomes or measurements before and after its occurrence, without a control group.
- Longitudinal studies collect data from the same individuals or groups over an extended period to assess trends over time.
- Correlational studies help to investigate the relationship between two or more variables to determine if they vary together (without implying causation).
- Qualitative studies explore the underlying meanings and nuances of a phenomenon through interviews, focus group sessions, or other exploratory methods based on conversations and observations…(More)”.
Machine-assisted mixed methods: augmenting humanities and social sciences with artificial intelligence
Paper by Andres Karjus: “The increasing capacities of large language models (LLMs) present an unprecedented opportunity to scale up data analytics in the humanities and social sciences, augmenting and automating qualitative analytic tasks previously typically allocated to human labor. This contribution proposes a systematic mixed methods framework to harness qualitative analytic expertise, machine scalability, and rigorous quantification, with attention to transparency and replicability. 16 machine-assisted case studies are showcased as proof of concept. Tasks include linguistic and discourse analysis, lexical semantic change detection, interview analysis, historical event cause inference and text mining, detection of political stance, text and idea reuse, genre composition in literature and film; social network inference, automated lexicography, missing metadata augmentation, and multimodal visual cultural analytics. In contrast to the focus on English in the emerging LLM applicability literature, many examples here deal with scenarios involving smaller languages and historical texts prone to digitization distortions. In all but the most difficult tasks requiring expert knowledge, generative LLMs can demonstrably serve as viable research instruments. LLM (and human) annotations may contain errors and variation, but the agreement rate can and should be accounted for in subsequent statistical modeling; a bootstrapping approach is discussed. The replications among the case studies illustrate how tasks previously requiring potentially months of team effort and complex computational pipelines, can now be accomplished by an LLM-assisted scholar in a fraction of the time. Importantly, this approach is not intended to replace, but to augment researcher knowledge and skills. With these opportunities in sight, qualitative expertise and the ability to pose insightful questions have arguably never been more critical…(More)”.
On the culture of open access: the Sci-hub paradox
Paper by Abdelghani Maddi and David Sapinho: “Shadow libraries, also known as ”pirate libraries”, are online collections of copyrighted publications that have been made available for free without the permission of the copyright holders. They have gradually become key players of scientific knowledge dissemination, despite their illegality in most countries of the world. Many publishers and scientist-editors decry such libraries for their copyright infringement and loss of publication usage information, while some scholars and institutions support them, sometimes in a roundabout way, for their role in reducing inequalities of access to knowledge, particularly in low-income countries. Although there is a wealth of literature on shadow libraries, none of this have focused on its potential role in knowledge dissemination, through the open access movement. Here we analyze how shadow libraries can affect researchers’ citation practices, highlighting some counter-intuitive findings about their impact on the Open Access Citation Advantage (OACA). Based on a large randomized sample, this study first shows that OA publications, including those in fully OA journals, receive more citations than their subscription-based counterparts do. However, the OACA has slightly decreased over the seven last years. The introduction of a distinction between those accessible or not via the Scihub platform among subscription-based suggest that the generalization of its use cancels the positive effect of OA publishing. The results show that publications in fully OA journals are victims of the success of Sci-hub. Thus, paradoxically, although Sci-hub may seem to facilitate access to scientific knowledge, it negatively affects the OA movement as a whole, by reducing the comparative advantage of OA publications in terms of visibility for researchers. The democratization of the use of Sci-hub may therefore lead to a vicious cycle, hindering efforts to develop full OA strategies without proposing a credible and sustainable alternative model for the dissemination of scientific knowledge…(More)”.
Game Changing Tools for Evidence Synthesis: Generative AI, Data and Policy Design
Paper by Geoff Mulgan, and Sarah O’Meara: “Evidence synthesis aims to make sense of huge bodies of evidence from around the world and make it available for busy decision-makers. Google search was in some respects a game changer in that you could quickly find out what was happening in a field – but it turned out to be much less useful for judging which evidence was relevant, reliable or high quality. Now large language models (LLM) and generative AI are offering an alternative to Google and again appear to have the potential to dramatically improve evidence synthesis, in an instant bringing together large bodies of knowledge and making it available to policy-makers, members of parliament or indeed the public.
But again there’s a gap between the promise and the results. ChatGPT is wonderful for producing a rough first draft: but its inputs are often out of date, it can’t distinguish good from bad evidence and its outputs are sometimes made up. So nearly a year after the arrival of ChatGPT we have been investigating how generative AI can be used most effectively, and, linked to that, how new methods can embed evidence into the daily work of governments and provide ways to see if the best available evidence is being used.
We think that these will be game-changers: transforming the everyday life of policy-makers, and making it much easier to mobilise, and assess evidence – especially if human and machine intelligence are combined rather than being seen as alternatives. But they need to be used with care and judgement rather than being panaceas. [Watch IPPO’s recent discussion here]…(More)”.
Open Science and Data Protection: Engaging Scientific and Legal Contexts
Editorial Paper of Special Issue edited by Ludovica Paseri: “This paper analyses the relationship between open science policies and data protection. In order to tackle the research data paradox of the contemporary science, i.e., the tension between the pursuit of data-driven scientific research and the crisis of repeatability or reproducibility of science, a theoretical perspective suggests a potential convergence between open science and data protection. Both fields regard governance mechanisms that shall take into account the plurality of interests at stake. The aim is to shed light on the processing of personal data for scientific research purposes in the context of open science. The investigation supports a threefold need: that of broadening the legal debate; of expanding the territorial scope of the analysis, in addition to the extra-territoriality effects of the European Union’s law; and an interdisciplinary discussion. Based on these needs, four perspectives are then identified, that encompass the challenges related to data processing in the context of open science: (i) the contextual and epistemological perspectives; (ii) the legal coordination perspectives; (iii) the governance perspectives; and (iv) the technical perspectives…(More)”.
The Rapid Growth of Behavioral Science
Article by Steve Wendel: “It’s hard to miss the rapid growth of our field: into new sectors, into new countries, and into new collaborations with other fields. Over the years, I’ve sought to better understand that growth by collecting data about our field and sharing the results. A few weeks ago, I launched the most recent effort – a survey for behavioral science & behavioral design practitioners and one for behavioral researchers around the globe. Here, I’ll share a bit about what we’re seeing so far in the data, and ask for your help to spread it more widely.
First, our field has seen rapid growth since 2008 – which is, naturally, when Thaler and Sunstein’s Nudge first came out. The number of teams and practitioners in the space has grown more or less in tandem, though with a recent slowing in the creation of new teams since 2020. The most productive year was 2019, with 59 new teams starting; the subsequent three years have averaged 28 per year[1].
Behavioral science and design practitioners are also increasingly spread around the world. Just a few years ago, it was difficult to find practitioners outside of BeSci centers in the US, UK, and a few other countries. While we are still heavily concentrated in these areas, there are now active practitioners in 72 countries: from Paraguay to Senegal to Bhutan.
Figure 1: Where practitioners are located. Note – the live and interactive map is available on BehavioralTeams.com.
The majority of practitioners (52%) are in full-time behavioral science or behavioral design roles. The rest are working in other disciplines such as product design and marketing in which they aren’t dedicated to BeSci but have the opportunity to apply it in their work (38%). A minority of individuals have BeSci side jobs (9%).
Among respondents thus far, the most common challenge they are facing is making the case for behavioral science with senior leaders in their organizations (63%) and being able to measure the impact of their inventions (65%). Anecdotally, many practitioners in the field complain that they are asked for their recommendations on what to do, but aren’t given the opportunity to follow up and see if those recommendations were implemented or, when implemented, were actually effective.
The survey asks many more questions about the experiences and backgrounds of practitioners, but we’re still gathering data and will release new results when we have them…(More)”.