Mind the (Language) Gap: Mapping the Challenges of LLM Development in Low-Resource Language Contexts

Curated on Posted on by Stefaan Verhulst

White Paper by the Stanford Institute for Human-Centered AI (HAI), the Asia Foundation and the University of Pretoria: “…maps the LLM development landscape for low-resource languages, highlighting challenges, trade-offs, and strategies to increase investment; prioritize cross-disciplinary, community-driven development; and ensure fair data ownership…

  • Large language model (LLM) development suffers from a digital divide: Most major LLMs underperform for non-English—and especially low-resource—languages; are not attuned to relevant cultural contexts; and are not accessible in parts of the Global South.
  • Low-resource languages (such as Swahili or Burmese) face two crucial limitations: a scarcity of labeled and unlabeled language data and poor quality data that is not sufficiently representative of the languages and their sociocultural contexts.
  • To bridge these gaps, researchers and developers are exploring different technical approaches to developing LLMs that better perform for and represent low-resource languages but come with different trade-offs:
    • Massively multilingual models, developed primarily by large U.S.-based firms, aim to improve performance for more languages by including a wider range of (100-plus) languages in their training datasets.
    • Regional multilingual models, developed by academics, governments, and nonprofits in the Global South, use smaller training datasets made up of 10-20 low-resource languages to better cater to and represent a smaller group of languages and cultures.
    • Monolingual or monocultural models, developed by a variety of public and private actors, are trained on or fine-tuned for a single low-resource language and thus tailored to perform well for that language…(More)”

Working With Cracks

Curated on Posted on by Stefaan Verhulst

An excerpt from Everyday Habits for Transforming Systems by Adam Kahane: “Systems are structured to keep producing the behaviors and results they are producing, and therefore often seem solid and unchangeable—but they are not. They are built, and they collapse. They crack and are cracked, which opens up new possibilities that some people find frightening and others find hopeful. Radical engagement involves looking for, moving toward, and working with these cracks—not ignoring or shying away from them. We do this by seeking out and working with openings, alongside others who are doing the same…

Al Etmanski has pioneered the transformation of the living conditions of Canadians with disabilities, away from segregation, dependency, and second-class status toward connection, agency, and justice. I have spoken with him and studied what he and others have written about his decades of experience, and especially about how his strategy and approach have evolved and enabled him to make the contributions he has. He has advanced through repeatedly searching out and working with openings or cracks (breakdowns and bright spots) in the social-economic-political-institutional-cultural “disability system.”..(More)”.

How is AI augmenting collective intelligence for the SDGs?

Curated on Posted on by Stefaan Verhulst

Article by UNDP: “Increasingly AI techniques like natural language processing, machine learning and predictive analytics are being used alongside the most common methods in collective intelligence, from citizen science and crowdsourcing to digital democracy platforms.

At its best, AI can be used to augment and scale the intelligence of groups. In this section we describe the potential offered by these new combinations of human and machine intelligence. First we look at the applications that are most common, where AI is being used to enhance efficiency and categorize unstructured data, before turning to the emerging role of AI – where it helps us to better understand complex systems.

These are the three main ways AI and collective intelligence are currently being used together for the SDGs:

1. Efficiency and scale of data processing

AI is being effectively incorporated into collective intelligence projects where timing is paramount and a key insight is buried deep within large volumes of unstructured data. This combination of AI and collective intelligence is most useful when decision makers require an early warning to help them manage risks and distribute public resources more effectively. For example, Dataminr’s First Alert system uses pre-trained machine learning models to sift through text and images scraped from the internet, as well as other data streams, such as audio broadcasts, to isolate early signals that anticipate emergency events…(More)”. (See also: Where and when AI and CI meet: exploring the intersection of artificial and collective intelligence towards the goal of innovating how we govern).

Developing countries are struggling to achieve their technology aims. Shared digital infrastructure is the answer

Curated on Posted on by Stefaan Verhulst

Article by Nii Simmonds: “The digital era offers remarkable prospects for both economic advancement and social development. Yet for emerging economies lacking energy, this potential often seems out of reach. The harsh truths of inconsistent electricity supply and scarce resources looms large over their digital ambitions. Nevertheless, a ray of hope shines through a strategy I call shared digital infrastructure (SDI). This cooperative model has the ability to turn these obstacles into opportunities for growth. By collaborating through regional country partnerships and bodies such as the Association of Southeast Asian Nations (ASEAN), the African Union (AU) and the Caribbean Community (CARICOM), these countries can harness the revolutionary power of digital technology, despite the challenges.

The digital economy is a critical driver of global GDP, with innovations in artificial intelligence, e-commerce and financial technology transforming industries at an unprecedented pace. At the heart of this transformation are data centres, which serve as the backbone of digital services, cloud computing and AI-driven applications. Yet many developing nations struggle to establish and maintain such facilities due to high energy costs, inadequate grid reliability and limited investment capital…(More)”.

Robotics for Global development

Curated on Posted on by Stefaan Verhulst

Report by the Frontier Tech Hub: “Robotics could enable progress on 46% of SDG targets  yet this potential remains largely untapped in low and middle-income countries. 

While technological developments and new-found applications of artificial intelligence (AI) keep captivating significant attention and investments, using robotics to advance the Sustainable Development Goals (SDGs) is consistently overlooked. This is especially true when the focus moves from aerial robotics (drones) to robotic arms, ground robotics, and aquatic robotics. How might these types of robots accelerate global development in the least developed countries? 

We aim to answer this question and inform the UK Foreign, Commonwealth & Development Office’s (FCDO) investment and policy towards robotics in the least developed countries (LDCs). In an emergent space, the UK FCDO has a unique opportunity to position itself as a global leader in leveraging robotics technology to accelerate sustainable development outcomes…(More)”.

From Insights to Action: Amplifying Positive Deviance within Somali Rangelands

Curated on Posted on by Stefaan Verhulst

Article by Basma Albanna, Andreas Pawelke and Hodan Abdullahi: “In every community, some individuals or groups achieve significantly better outcomes than their peers, despite having similar challenges and resources. Finding these so-called positive deviants and working with them to diffuse their practices is referred to as the Positive Deviance approach. The Data-Powered Positive Deviance (DPPD) method follows the same logic as the Positive Deviance approach but leverages existing, non-traditional data sources, in conjunction with traditional data sources to identify and scale the solutions of positive deviants. The UNDP Somalia Accelerator Lab was part of the first cohort of teams that piloted the application of DPPD trying to tackle the rangeland health problem in the West Golis region. In this blog post we’re reflecting on the process we designed and tested to go from the identification and validation of successful practices to helping other communities adopt them.

Uncovering Rangeland Success

Three years ago we embarked on a journey to identify pastoral communities in Somaliland that demonstrated resilience in the face of adversity. Using a mix of traditional and non-traditional data sources, we wanted to explore and learn from communities that managed to have healthy rangelands despite the severe droughts of 2016 and 2017.

We engaged with government officials from various ministries, experts from the University of Hargeisa, international organizations like the FAO and members of agro-pastoral communities to learn more about rangeland health. We then selected the West Golis as our region of interest with a majority pastoral community and relative ease of access. Employing the Soil-Adjusted Vegetation Index (SAVI) and using geospatial and earth observation data allowed us to identify an initial group of potential positive deviants illustrated as green circles in Figure 1 below.

From Insights to Action: Amplifying Positive Deviance within Somali Rangelands
Figure 1: Measuring the vegetation health within 5 km community buffer zones based on SAVI.

Following the identification of potential positive deviants, we engaged with 18 pastoral communities from the Togdheer, Awdal, and Maroodijeex regions to validate whether the positive deviants we found using earth observation data were indeed doing better than the other communities.

The primary objective of the fieldwork was to uncover the existing practices and strategies that could explain the outperformance of positively-deviant communities compared to other communities. The research team identified a range of strategies, including soil and water conservation techniques, locally-produced pesticides, and reseeding practices as summarized in Figure 2.

From Insights to Action
Figure 2: Strategies and practices that emerged from the fieldwork

Data-Powered Positive Deviance is not just about identifying outperformers and their successful practices. The real value lies in the diffusion, adoption and adaptation of these practices by individuals, groups or communities facing similar challenges. For this to succeed, both the positive deviants and those learning about their practices must take ownership and drive the process. Merely presenting the uncommon but successful practices of positive deviants to others will not work. The secret to success is in empowering the community to take charge, overcome challenges, and leverage their own resources and capabilities to effect change…(More)”.

A US-run system alerts the world to famines. It’s gone dark after Trump slashed foreign aid

Curated on Posted on by Stefaan Verhulst

Article by Lauren Kent: “A vital, US-run monitoring system focused on spotting food crises before they turn into famines has gone dark after the Trump administration slashed foreign aid.

The Famine Early Warning Systems Network (FEWS NET) monitors drought, crop production, food prices and other indicators in order to forecast food insecurity in more than 30 countries…Now, its work to prevent hunger in Sudan, South Sudan, Somalia, Yemen, Ethiopia, Afghanistan and many other nations has been stopped amid the Trump administration’s effort to dismantle the US Agency for International Development (USAID).

“These are the most acutely food insecure countries around the globe,” said Tanya Boudreau, the former manager of the project.

Amid the aid freeze, FEWS NET has no funding to pay staff in Washington or those working on the ground. The website is down. And its treasure trove of data that underpinned global analysis on food security – used by researchers around the world – has been pulled offline.

FEWS NET is considered the gold-standard in the sector, and it publishes more frequent updates than other global monitoring efforts. Those frequent reports and projections are key, experts say, because food crises evolve over time, meaning early interventions save lives and save money…The team at the University of Colorado Boulder has built a model to forecast water demand in Kenya, which feeds some data into the FEWS NET project but also relies on FEWS NET data provided by other research teams.

The data is layered and complex. And scientists say pulling the data hosted by the US disrupts other research and famine-prevention work conducted by universities and governments across the globe.

“It compromises our models, and our ability to be able to provide accurate forecasts of ground water use,” Denis Muthike, a Kenyan scientist and assistant research professor at UC Boulder, told CNN, adding: “You cannot talk about food security without water security as well.”

“Imagine that that data is available to regions like Africa and has been utilized for years and years – decades – to help inform divisions that mitigate catastrophic impacts from weather and climate events, and you’re taking that away from the region,” Muthike said. He cautioned that it would take many years to build another monitoring service that could reach the same level…(More)”.

Human Development and the Data Revolution

Curated on Posted on by Stefaan Verhulst

Book edited by Sanna Ojanperä, Eduardo López, and Mark Graham: “…explores the uses of large-scale data in the contexts of development, in particular, what techniques, data sources, and possibilities exist for harnessing large datasets and new online data to address persistent concerns regarding human development, inequality, exclusion, and participation.

Employing a global perspective to explore the latest advances at the intersection of big data analysis and human development, this volume brings together pioneering voices from academia, development practice, civil society organizations, government, and the private sector. With a two-pronged focus on theoretical and practical research on big data and computational approaches in human development, the volume covers such themes as data acquisition, data management, data mining and statistical analysis, network science, visual analytics, and geographic information systems and discusses them in terms of practical applications in development projects and initiatives. Ethical considerations surrounding these topics are visited throughout, highlighting the tradeoffs between benefitting and harming those who are the subjects of these new approaches…(More)”

A Roadmap to Accessing Mobile Network Data for Statistics

Curated on Posted on by Stefaan Verhulst

Guide by Global Partnership for Sustainable Development Data: “… introduces milestones on the path to mobile network data access. While it is aimed at stakeholders in national statistical systems and across national governments in general, the lessons should resonate with others seeking to take this route. The steps in this guide are written in the order in which they should be taken, and some readers who have already embarked on this journey may find they have completed some of these steps. 

This roadmap is meant to be followed in steps, and readers may start, stop, and return to points on the path at any point. 

The path to mobile network data access has three milestones:

  1. Evaluating the opportunity – setting clear goals for the desired impact of data innovation.
  2. Engaging with stakeholders – getting critical stakeholders to support your cause.
  3. Executing collaboration agreements – signing a written agreement among partners…(More)”

The 2026 Aid Transparency Index is canceled. Here’s what it means

Curated on Posted on by Stefaan Verhulst

Article by Gary Forster: “As things stand, we will not be running the 2026 Aid Transparency Index. Not because it isn’t needed. Not because it isn’t effective. But because, in spite of our best efforts, we haven’t been able to secure the funding for it.

This is not a trivial loss. The Aid Transparency Index has been the single most powerful mechanism driving improvements in the quantity and quality of aid data that is published to the International Aid Transparency Initiative, or IATI, Standard. Since 2012, every two years, it has independently assessed and ranked the transparency of the world’s 50 largest aid agencies — organizations responsible for 92% of all spending published in IATI, amounting to $237 billion in 2023 alone.

The index works because it shapes agency behavior. It has encouraged reluctant agencies to start publishing data; motivated those already engaged to improve data quantity and quality; and provided a crucial, independent check on the state of global aid transparency…(More)”.