More Questions Than Flags: Reality Check on DSA’s Trusted Flaggers

Curated on Posted on by Stefaan Verhulst

Article by Ramsha Jahangir, Elodie Vialle and Dylan Moses: “It’s been 100 days since the Digital Services Act (DSA) came into effect, and many of us are still wondering how the Trusted Flagger mechanism is taking shape, particularly for civil society organizations (CSOs) that could be potential applicants.

With an emphasis on accountability and transparency, the DSA requires national coordinators to appoint Trusted Flaggers, who are designated entities whose requests to flag illegal content must be prioritized. “Notices submitted by Trusted Flaggers acting within their designated area of expertise . . . are given priority and are processed and decided upon without undue delay,” according to the DSA. Trusted flaggers can include non-governmental organizations, industry associations, private or semi-public bodies, and law enforcement agencies. For instance, a private company that focuses on finding CSAM or terrorist-type content, or tracking groups that traffic in that content, could be eligible for Trusted Flagger status under the DSA. To be appointed, entities need to meet certain criteria, including being independent, accurate, and objective.

Trusted escalation channels are a key mechanism for civil society organizations (CSOs) supporting vulnerable users, such as human rights defenders and journalists targeted by online attacks on social media, particularly in electoral contexts. However, existing channels could be much more efficient. The DSA is a unique opportunity to redesign these mechanisms for reporting illegal or harmful content at scale. They need to be rethought for CSOs that hope to become Trusted Flaggers. Platforms often require, for instance, content to be translated into English and context to be understood by English-speaking audiences (due mainly to the fact that the key decision-makers are based in the US), which creates an added burden for CSOs that are resource-strapped. The lack of transparency in the reporting process can be distressing for the victims for whom those CSOs advocate. The lack of timely response can lead to dramatic consequences for human rights defenders and information integrity. Several CSOs we spoke with were not even aware of these escalation channels – and platforms are not incentivized to promote mechanisms given the inability to vet, prioritize and resolve all potential issues sent to them….(More)”.

Participatory mapping as a social digital tool

Curated on Posted on by Stefaan Verhulst

Blog by María de los Ángeles Briones: “…we will use 14 different examples from different continents and contexts to explore the goals and methods used for participatory mapping as a social digital tool. Despite looking very different and coming from a range of cultural backgrounds, there are a number of similarities in these different case studies.

Although the examples have different goals, we have identified four main focus areas: activism, conviviality, networking and urban planning. More localised mapping projects often had a focus on activism. We also see from that maps are not isolated tools, they are complementary to work with other communication tools and platforms.

The internet has transformed communications and networks across the globe – allowing for interconnectivity and scalability of information among and between different groups of society. This allows voices, regardless of their location, to be amplified and listened to by many other voices achieving collective goals. This has great potential in a global world where it is evident that top-down initiatives are not enough to handle many of the social needs that local people experience. However, though the internet makes sharing and collaborating between people easier, offline maps are still valuable, as shown in some of our examples.

The similarity between the different maps that we explored is that they are social digital tools. They are social because they are related to projects that are seeking to solve social needs; and they are digital because they are based on digital platforms that permit them to be alive, spread, shared and used. These characteristics also refer to their function and design.

A tool can be defined as a device or implement, especially one held in the hand, used to carry out a particular function. So when we speak of a tool there are four things involved: an actor, an object, a function and a purpose. Just as a hammer is a tool that a carpenter (actor) use to hammer nails (function) and thus build something (purpose) we understand that social tools are used by one or more people for taking actions where the final objective is to meet a social need…(More)”.

Crowded Out: The True Costs of Crowdfunding Healthcare

Curated on Posted on by Stefaan Verhulst

Book by Nora Kenworthy: “Over the past decade, charitable crowdfunding has exploded in popularity across the globe. Sites such as GoFundMe, which now boasts a “global community of over 100 million” users, have transformed the ways we seek and offer help. When faced with crises—especially medical ones—Americans are turning to online platforms that promise to connect them to the charity of the crowd. What does this new phenomenon reveal about the changing ways we seek and provide healthcare? In Crowded Out, Nora Kenworthy examines how charitable crowdfunding so quickly overtook public life, where it is taking us, and who gets left behind by this new platformed economy.

Although crowdfunding has become ubiquitous in our lives, it is often misunderstood: rather than a friendly free market “powered by the kindness” of strangers, crowdfunding is powerfully reinforcing inequalities and changing the way Americans think about and access healthcare. Drawing on extensive research and rich storytelling, Crowded Out demonstrates how crowdfunding for health is fueled by—and further reinforces—financial and moral “toxicities” in market-based healthcare systems. It offers a unique and distressing look beneath the surface of some of the most popular charitable platforms and helps to foster thoughtful discussions of how we can better respond to healthcare crises both small and large…(More)”.

Disfactory Project: How to Detect Illegal Factories by Open Source Technology and Crowdsourcing

Curated on Posted on by Stefaan Verhulst

Article by Peii Lai: “…building illegal factories on farmlands is still a profitable business, because the factory owners thus obtain the means of production at a lower price and can easily get away with penalties by simply ignoring their legal responsibility. Such conduct simply shifts the cost of production onto the environment in an irresponsible way. As we can imagine, such violations has been increasing year by year. On average, Taiwan loses 1,500 hectares of farmland each year due to illegal use, which demonstrates that illegal factories are an ongoing and escalating problem that people cannot ignore.

It’s clearly that the problem of illegal factories are caused by dysfunction of the previous land management regulations. In response to that, Citizens of Earth Taiwan (CET) started seeking solutions to tackle the illegal factories. CET soon realized that the biggest obstacle they faced was that no one saw the violations as a big deal. Local governments avoided standing on the opposite side of the illegal factories. For local governments, imposing penalties is an arduous and thankless task…

Through the collaboration of CET and g0v-zero, the Disfactory project combines the knowledge they have accumulated through advocacy and the diverse techniques brought by the passionate civic contributors. In 2020, the Disfactory project team delivered its first product: disfactory.tw. They built a website with geographic information that whistle blowers can operate on the ground by themselves. Through a few simple steps: identifying the location of the target illegal factory, taking a picture of it, uploading the photos, any citizen can easily register the information on Disfactory’s website….(More)”

Crowdsourcing for collaborative crisis communication: a systematic review

Curated on Posted on by Stefaan Verhulst

Paper by Maria Clara Pestana, Ailton Ribeiro and Vaninha Vieira: “Efficient crisis response and support during emergency scenarios rely on collaborative communication channels. Effective communication between operational centers, civilian responders, and public institutions is vital. Crowdsourcing fosters communication and collaboration among a diverse public. The primary objective is to explore the state-of-the-art in crowdsourcing for collaborative crisis communication guided by a systematic literature review. The study selected 20 relevant papers published in the last decade. The findings highlight solutions to facilitate rapid emergency responses, promote seamless coordination between stakeholders and the general public, and ensure data credibility through a rigorous validation process…(More)”.

Millions of gamers advance biomedical research

Curated on Posted on by Stefaan Verhulst

Article by McGill: “…4.5 million gamers around the world have advanced medical science by helping to reconstruct microbial evolutionary histories using a minigame included inside the critically and commercially successful video game, Borderlands 3. Their playing has led to a significantly refined estimate of the relationships of microbes in the human gut. The results of this collaboration will both substantially advance our knowledge of the microbiome and improve on the AI programs that will be used to carry out this work in future.

By playing Borderlands Science, a mini-game within the looter-shooter video game Borderlands 3, these players have helped trace the evolutionary relationships of more than a million different kinds of bacteria that live in the human gut, some of which play a crucial role in our health. This information represents an exponential increase in what we have discovered about the microbiome up till now. By aligning rows of tiles which represent the genetic building blocks of different microbes, humans have been able to take on tasks that even the best existing computer algorithms have been unable to solve yet…(More) (and More)”.

Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracy

Curated on Posted on by Stefaan Verhulst

Paper by Philipp Schoenegger, Indre Tuminauskaite, Peter S. Park, and Philip E. Tetlock: “Human forecasting accuracy in practice relies on the ‘wisdom of the crowd’ effect, in which predictions about future events are significantly improved by aggregating across a crowd of individual forecasters. Past work on the forecasting ability of large language models (LLMs) suggests that frontier LLMs, as individual forecasters, underperform compared to the gold standard of a human crowd forecasting tournament aggregate. In Study 1, we expand this research by using an LLM ensemble approach consisting of a crowd of twelve LLMs. We compare the aggregated LLM predictions on 31 binary questions to that of a crowd of 925 human forecasters from a three-month forecasting tournament. Our preregistered main analysis shows that the LLM crowd outperforms a simple no-information benchmark and is not statistically different from the human crowd. In exploratory analyses, we find that these two approaches are equivalent with respect to medium-effect-size equivalence bounds. We also observe an acquiescence effect, with mean model predictions being significantly above 50%, despite an almost even split of positive and negative resolutions. Moreover, in Study 2, we test whether LLM predictions (of GPT-4 and Claude 2) can be improved by drawing on human cognitive output. We find that both models’ forecasting accuracy benefits from exposure to the median human prediction as information, improving accuracy by between 17% and 28%: though this leads to less accurate predictions than simply averaging human and machine forecasts. Our results suggest that LLMs can achieve forecasting accuracy rivaling that of human crowd forecasting tournaments: via the simple, practically applicable method of forecast aggregation. This replicates the ‘wisdom of the crowd’ effect for LLMs, and opens up their use for a variety of applications throughout society…(More)”.

Eat, Click, Judge: The Rise of Cyber Jurors on China’s Food Apps

Curated on Posted on by Stefaan Verhulst

Article from Ye Zhanhang: “From unwanted ingredients in takeaway meals and negative restaurant reviews to late deliveries and poor product quality, digital marketplaces teem with minor frustrations. 

But because they affect customer satisfaction and business reputations, several Chinese online shopping platforms have come up with a unique solution: Ordinary users can become “cyber jurors” to deliberate and cast decisive votes in resolving disputes between buyers and sellers.

Though introduced in 2020, the concept has surged in popularity among young Chinese in recent months, primarily fueled by viral cases that users eagerly follow, scrutinizing every detail and deliberation online…

To be eligible for the role, a user must meet certain criteria, including having a verified account, maintaining consumption records within the past three months, and successfully navigating five mock cases as part of an entry test. Cyber jurors don’t receive any money for completing cases but may be rewarded with coupons.

Xianyu, an online secondhand shopping platform, has also introduced a “court” system that assembles a jury of 17 volunteer users to adjudicate disputes between buyers and sellers. 

Miao Mingyu, a law professor at the University of Chinese Academy of Social Sciences, told China Youth Daily that this public jury function, with its impartial third-party perspective, has the potential to enhance transaction transparency and the fairness of the platform’s evaluation system.

Despite Chinese law prohibiting platforms from removing user reviews of products, Miao noted that this feature has enabled the platform to effectively address unfair negative reviews without violating legal constraints…(More)”.

How to improve economic forecasting

Curated on Posted on by Stefaan Verhulst

Article by Nicholas Gruen: “Today’s four-day weather forecasts are as accurate as one-day forecasts were 30 years ago. Economic forecasts, on the other hand, aren’t noticeably better. Former Federal Reserve chair Ben Bernanke should ponder this in his forthcoming review of the Bank of England’s forecasting.

There’s growing evidence that we can improve. But myopia and complacency get in the way. Myopia is an issue because economists think technical expertise is the essence of good forecasting when, actually, two things matter more: forecasters’ understanding of the limits of their expertise and their judgment in handling those limits.

Enter Philip Tetlock, whose 2005 book on geopolitical forecasting showed how little experts added to forecasting done by informed non-experts. To compare forecasts between the two groups, he forced participants to drop their vague weasel words — “probably”, “can’t be ruled out” — and specify exactly what they were forecasting and with what probability. 

That started sorting the sheep from the goats. The simple “point forecasts” provided by economists — such as “growth will be 3.0 per cent” — are doubly unhelpful in this regard. They’re silent about what success looks like. If I have forecast 3.0 per cent growth and actual growth comes in at 3.2 per cent — did I succeed or fail? Such predictions also don’t tell us how confident the forecaster is.

By contrast, “a 70 per cent chance of rain” specifies a clear event with a precise estimation of the weather forecaster’s confidence. Having rigorously specified the rules of the game, Tetlock has since shown how what he calls “superforecasting” is possible and how diverse teams of superforecasters do even better. 

What qualities does Tetlock see in superforecasters? As well as mastering necessary formal techniques, they’re open-minded, careful, curious and self-critical — in other words, they’re not complacent. Aware, like Socrates, of how little they know, they’re constantly seeking to learn — from unfolding events and from colleagues…(More)”.

The Crowdless Future? How Generative AI Is Shaping the Future of Human Crowdsourcing

Curated on Posted on by Stefaan Verhulst

Paper by Leonard Boussioux, Jacqueline Lane, Miaomiao Zhang, Vladimir Jacimovic, and Karim Lakhani: “This study investigates the capability of generative artificial intelligence (AI) in creating innovative business solutions compared to human crowdsourcing methods. We initiated a crowdsourcing challenge focused on sustainable, circular economy business opportunities. The challenge attracted a diverse range of solvers from a myriad of countries and industries. Simultaneously, we employed GPT-4 to generate AI solutions using three different prompt levels, each calibrated to simulate distinct human crowd and expert personas. 145 evaluators assessed a randomized selection of 10 out of 234 human and AI solutions, a total of 1,885 evaluator-solution pairs. Results showed comparable quality between human and AI-generated solutions. However, human ideas were perceived as more novel, whereas AI solutions delivered better environmental and financial value. We use natural language processing techniques on the rich solution text to show that although human solvers and GPT-4 cover a similar range of industries of application, human solutions exhibit greater semantic diversity. The connection between semantic diversity and novelty is stronger in human solutions, suggesting differences in how novelty is created by humans and AI or detected by human evaluators. This study illuminates the potential and limitations of both human and AI crowdsourcing to solve complex organizational problems and sets the groundwork for a possible integrative human-AI approach to problem-solving…(More)”.