Measuring Scientific Impact Beyond Citation Counts


Robert M. Patton, Christopher G. Stahl and Jack C. Wells at DLib Magazine: “Measuring scientific progress remains elusive. There is an intuitive understanding that, in general, science is progressing forward. New ideas and theories are formed, older ideas and theories are confirmed, rejected, or modified. Progress is made. But, questions such as how is it made, by whom, how broadly, or how quickly present significant challenges. Historically, scientific publications reference other publications if the former publication in some way shaped the work that was performed. In other words, one publication “impacted” a latter one. The implication of this impact revolves around the intellectual content of the idea, theory, or conclusion that was formed. Several metrics such as h-index or journal impact factor (JIF) are often used as a means to assess whether an author, article, or journal creates an “impact” on science. The implied statement behind high values for such metrics is that the work must somehow be valuable to the community, which in turn implies that the author, article, or journal somehow has influenced the direction, development, or progress of what others in that field do. Unfortunately, the drive for increased publication revenue, research funding, or global recognition has lead to a variety of external factors completely unrelated to the quality of the work that can be used to manipulate key metric values. In addition, advancements in computing and data sciences field have further altered the meaning of impact on science.

The remainder of this paper will highlight recent advancements in both cultural and technological factors that now influence scientific impact as well as suggest new factors to be leveraged through full content analysis of publications….(More)”

Next big thing: The ‘uberfication’ of crowdsourced news


Ken Doctor at Politico: “Get ready to hear a lot about the “uberfication” of user-generated content.

Yes, it’s a mouthful. But it’s also the next big thing. Fresco News, a two-year-old New York start-up, sees itself becoming a hot property as it cracks the code on local amateur content generation….Fresco News now enables local TV stations to assign, receive and quickly get on air and online lots of amateur-shot newsy videos in their metro area.

Its secret sauce: Uberizing the supply chain process from station assignment to Fresco “qualified” shooter to shooting smartphone video to uploading and optimizing its quality for quick delivery to consumers, online or on the air.

Meyer’s team of 40, which includes numerous part-timers, has assiduously worked through the many frictions. That’s one hallmark of successful Uberfication.

“We just did a tremendous amount of just non-stop testing,” he says. “I would say, even with simple things like user acquisition, which is a major part of our process and entering new markets. We’ve tested hundreds of different ad types, graphics that we’ve designed internally that effectively, and I would say cheaply, bring in prospective citizen journalists.”

Stations can assign easily. Would-be shooters can see assignments, geographically displayed, on a single screen. The upload works well and stations’ ability to quickly use the videos is a strong selling point. Fresco, then, handles the billing and payment processes, much as Uber does.

As with once-taxi rides, individual transaction amounts compute small. Shooters get $50 for each video used by a TV station or $20 for a still photo. Stations pay $75 for a video and $30 for a still. As a standalone business, Fresco News is a scale play.

It’s not a new idea.

UGC – or user-generated content – was supposed to be huge. The late ’90s notion: the Internet could make anyone and everyone a reporter, and make it easy for them to share their work widely and cheaply. Many newspaper chains bought into the idea, and tested it unevenly, hoping that UGC could provide what was, for awhile called, “local-local” content. Local-local meaning neighborhood plus, that kind of locally differentiating news coverage that publishers thought readers wanted, but coverage publishers believed cost too much if they had to pay professional reporters to do it.

Short story: It didn’t work for the chains. In part, the technology was immature. More importantly, it turns out that reporting – and writing – remains, even the Internet age, largely a professional skill. Publishers couldn’t find enough dependable local amateurs, and besides, they never really iterated a business model around the idea.

Then, there were the national start-up efforts. NowPublic, one memorable one partnered with the Associated Press, launched in 2005 …but never found traction. Today, several other companies ply the territory, with Storyful a standard of quality. Importantly, Storyful focuses on national and global content. Fresco News aims squarely at local – first across the 3,000-mile breadth of the U.S.

The dots tell the story

Take a look at the many dots on the Philly map above. Each blue dot represents an active, signed-up Fresco video shooter in the area. Each yellow dot shows current assignments. In this August visualization, visually, you get a sense of quickly and energetically local TV station Fox 29, WTXF, has deployed – and uses – Fresco News.as earned “preferred” status at Fresco, has been around journalistic operations for a long time and looks forward to contributing news tips as Fresco might expand what its tech can do for local stations…(More)”

Designing the Next Generation of Open Data Policy


Andrew Young and Stefaan Verhulst at the Open Data Charter Blog: “The international Open Data Charter has emerged from the global open data community as a galvanizing document to place open government data directly in the hands of citizens and organizations. To drive this process forward, and ensure that the outcomes are both systemic and transformational, new open data policy needs to be based on evidence of how and when open data works in practice. To support this work, the GovLab, in collaboration with Omidyar Network, has recently completed research which provides vital evidence of open data projects around the world, including an analysis of 19 in-depth, impact-focused case studies and a key findings paper. All of the research is now available in an eBook published by O’Reilly Media.

The research found that open data is making an impact in four core ways, including:…(More)”

Twitter, UN Global Pulse announce data partnership


PressRelease: “Twitter and UN Global Pulse today announced a partnership that will provide the United Nations with access to Twitter’s data tools to support efforts to achieve the Sustainable Development Goals, which were adopted by world leaders last year.

Every day, people around the world send hundreds of millions of Tweets in dozens of languages. This public data contains real-time information on many issues including the cost of food, availability of jobs, access to health care, quality of education, and reports of natural disasters. This partnership will allow the development and humanitarian agencies of the UN to turn these social conversations into actionable information to aid communities around the globe.

“The Sustainable Development Goals are first and foremost about people, and Twitter’s unique data stream can help us truly take a real-time pulse on priorities and concerns — particularly in regions where social media use is common — to strengthen decision-making. Strong public-private partnerships like this show the vast potential of big data to serve the public good,” said Robert Kirkpatrick, Director of UN Global Pulse.

“We are incredibly proud to partner with the UN in support of the Sustainable Development Goals,” said Chris Moody, Twitter’s VP of Data Services. “Twitter data provides a live window into the public conversations that communities around the world are having, and we believe that the increased potential for research and innovation through this partnership will further the UN’s efforts to reach the Sustainable Development Goals.”

Organizations and business around the world currently use Twitter data in many meaningful ways, and this unique data source enables them to leverage public information at scale to better inform their policies and decisions. These partnerships enable innovative uses of Twitter data, while protecting the privacy and safety of Twitter users.

UN Global Pulse’s new collaboration with Twitter builds on existing R&D that has shown the power of social media for social impact, like measuring the impact of public health campaigns, tracking reports of rising food prices, or prioritizing needs after natural disasters….(More)”

How Technology is Crowd-Sourcing the Fight Against Hunger


Beth Noveck at Media Planet: “There is more than enough food produced to feed everyone alive today. Yet access to nutritious food is a challenge everywhere and depends on getting every citizen involved, not just large organizations. Technology is helping to democratize and distribute the job of tackling the problem of hunger in America and around the world.

Real-time research

One of the hardest problems is the difficulty of gaining real-time insight into food prices and shortages. Enter technology. We no longer have to rely on professional inspectors slowly collecting information face-to-face. The UN World Food Programme, which provides food assistance to 80 million people each year, together with Nielsen is conducting mobile phone surveys in 15 countries (with plans to expand to 30), asking people by voice and text about what they are eating. Formerly blank maps are now filled in with information provided quickly and directly by the most affected people, making it easy to prioritize the allocation of resources.

Technology helps the information flow in both directions, enabling those in need to reach out, but also to become more effective at helping themselves. The Indian Ministry of Agriculture, in collaboration with Reuters Market Light, provides information services in nine Indian languages to 1.4 million registered farmers in 50,000 villages across 17 Indian states via text and voice messages.

“In the United States, 40 percent of the food produced here is wasted, and yet 1 in 4 American children (and 1 in 6 adults) remain food insecure…”

Data to the people

New open data laws and policies that encourage more transparent publication of public information complement data collection and dissemination technologies such as phones and tablets. About 70 countries and hundreds of regions and cities have adopted open data policies, which guarantee that the information these public institutions collect be available for free use by the public. As a result, there are millions of open datasets now online on websites such as the Humanitarian Data Exchange, which hosts 4,000 datasets such as country-by-country stats on food prices and undernourishment around the world.

Companies are compiling and sharing data to combat food insecurity, too. Anyone can dig into the data on the Global Open Data for Agriculture and Nutrition platform, a data collaborative where 300 private and public partners are sharing information.

Importantly, this vast quantity of open data is available to anyone, not only to governments. As a result, large and small entrepreneurs are able to create new apps and programs to combat food insecurity, such as Plantwise, which uses government data to offer a knowledge bank and run “plant clinics” that help farmers lose less of what they grow to pests. Google uses open government data to show people the location of farmers markets near their homes.

Students, too, can learn to play a role. For the second summer in a row, the Governance Lab at New York University, in partnership with the United States Department of Agriculture (USDA), mounted a two-week open data summer camp for 40 middle and high school students. The next generation of problem solvers is learning new data science skills by working on food safety and other projects using USDA open data.

Enhancing connection

Ultimately, technology enables greater communication and collaboration among the public, social service organizations, restaurants, farmers and other food producers who must work together to avoid food crises. The European Food Safety Authority in Italy has begun exploring how to use internet-based collaboration (often called citizen science or crowdsourcing) to get more people involved in food and feed risk assessment.

In the United States, 40 percent of the food produced here is wasted, and yet 1 in 4 American children (and 1 in 6 adults) remain food insecure, according to the Rockefeller Foundation. Copia, a San Francisco based smartphone app facilitates donations and deliveries of those with excess food in six cities in the Bay Area. Zero Percent in Chicago similarly attacks the distribution problem by connecting restaurants to charities to donate their excess food. Full Harvest is a tech platform that facilitates the selling of surplus produce that otherwise would not have a market.

Mobilizing the world

Prize-backed challenges create the incentives for more people to collaborate online and get involved in the fight against hunger….(More)”

What we see when we see transparency


Matthew Taylor at the RSA: “…social coordination theory…. is one of the most powerful ways to analyse complex social problems and to develop effective solutions. I spoke this morning to the Transparency Task Force and it gave me an opportunity to apply the theory to the issue of corporate openness.

First, a very brief recap of the theory:

My own approach focusses on the four modes as approaches to the challenge of social coordination. Human beings are complex social beings who have to work together to survive and flourish. The hierarchical perspective puts emphasis on leadership, strategy and expertise as the way to coordinate human activity. The solidaristic view emphasises the glue of belonging and shared values. The individualistic view sees coordination as emerging largely spontaneously and its goal being to provide a platform of individual ambition and competitive endeavour. The fatalistic view sees effective coordination, variously, as intractable, unlikely to deliver intended outcomes or irrelevant to the things that make it hardest to be human.

Each mode has a substrate in human evolution and psychology – these competing theories have emerged from who we are as a species. Each mode or combination of modes has been dominant at different times in our history. Also, each mode generates behavioural and ideological predispositions: Solidaristic views – which emphasise membership and values – are often associated with rigid ideologies (on both the left and right); individualism goes with liberal attitudes (again of both left and right varieties).

In a kind of fractal which stretches from individual preferences to global problem solving, each mode is available as a perspective on complex social problems. Crucially, for the kind of problems which the modern world increasingly generates, the best solutions will combine aspects of each method. But there is a problem: each mode is, in part, a critique of the others. Each has both benign and malign aspects; hierarchy can be strategic and overbearing, individualism can be enterprising and selfish, solidarity can be altruistic and tribal, fatalism can be stoical and defeatist. The theory explains why success can so quickly turn to failure. Even when the modes are successfully combined – what I call a ‘fully engaged’ solution – internal dynamics or external shocks will sooner or later upset the balance.

This is all rather theoretical so let me be more concrete. If we are trying to solve a problem like encouraging and enabling a corporation to be more transparent we need to understand the arguments both in favour and against such a move from each of the perspectives. If we don’t many people, many ideas and many approaches will be ignored. In short, we will be much more likely to fail.

Here is a simple guide to what might be seen as the pros and cons of greater transparency from the four standpoints:

perspectives on transparency Matthew Taylor

People in authority and those who see things from a hierarchical perspective (we all see things from different perspectives at different times and in different places) will worry that transparency will make decision making more difficult and that, by opening up things like the underlying business model to scrutiny by customers and competitors, it would threaten the interests of the organisation. Conversely, the hierarchical case for transparency is that it can increase trust and understanding towards leaders and the challenges they face and it can aid alignment, clarity and commitment by exposing practices that don’t fit with corporate strategy.

Those who approach things from a solidaristic perspective are often the most outspoken champions of transparency. They see it as increasing integrity as companies have to live up to their stated values and fairness as unfair practices are exposed.  However, there are also solidaristic concerns; what if transparency exposes vulnerable people or if transparency makes life harder for the team to which I belong (the company or my part of the company)?

From an individualistic starting point transparency can be viewed suspiciously as promoting a focus on process rather than outcomes and also being adverse to risk and reward (you can justify the means by the ends only if you have been able to achieve the ends without someone looking over your shoulder). On the other hand, the individualistic case for transparency can cite its contribution to innovation (by looking under the bonnet and seeing all the working parts we have better insights into what can be improved) and the promotion of rewards based on fair competition instead of covert rent-seeking or organisational nepotism.

Finally, those in a fatalistic mind set will tend to see transparency as either irrelevant or – and this is a more forensic critique – illusory (the secret stuff will just get hidden better). However there is also an appeal to be made to fatalists that transparency can help reveal warning signs of future dangers and make it easier to mount a defence when things go wrong (‘even if we failed, we can show that we tried’).

To win the case for transparency and also to implement it effectively its advocates need to stress the positives from the different perspectives and also address the legitimate concerns; both of which should make it easier to confront objections that are not reasonable. …(More)”

Living in the World of Both/And


Essay by Adene Sacks & Heather McLeod Grant  in SSIR: “In 2011, New York Times data scientist Jake Porway wrote a blog post lamenting the fact that most data scientists spend their days creating apps to help users find restaurants, TV shows, or parking spots, rather than addressing complicated social issues like helping identify which teens are at risk of suicide or creating a poverty index of Africa using satellite data.

That post hit a nerve. Data scientists around the world began clamoring for opportunities to “do good with data.” Porway—at the center of this storm—began to convene these scientists and connect them to nonprofits via hackathon-style events called DataDives, designed to solve big social and environmental problems. There was so much interest, he eventually quit his day job at the Times and created the organization DataKind to steward this growing global network of data science do-gooders.

At the same time, in the same city, another movement was taking shape—#GivingTuesday, an annual global giving event fueled by social media. In just five years, #GivingTuesday has reshaped how nonprofits think about fundraising and how donors give. And yet, many don’t know that 92nd Street Y (92Y)—a 140-year-old Jewish community and cultural center in Manhattan, better known for its star-studded speaker series, summer camps, and water aerobics classes—launched it.

What do these two examples have in common? One started as a loose global network that engaged data scientists in solving problems, and then became an organization to help support the larger movement. The other started with a legacy organization, based at a single site, and catalyzed a global movement that has reshaped how we think about philanthropy. In both cases, the founding groups have incorporated the best of both organizations and networks.

Much has been written about the virtues of thinking and acting collectively to solve seemingly intractable challenges. Nonprofit leaders are being implored to put mission above brand, build networks not just programs, and prioritize collaboration over individual interests. And yet, these strategies are often in direct contradiction to the conventional wisdom of organization-building: differentiating your brand, developing unique expertise, and growing a loyal donor base.

A similar tension is emerging among network and movement leaders. These leaders spend their days steering the messy process required to connect, align, and channel the collective efforts of diverse stakeholders. It’s not always easy: Those searching to sustain movements often cite the lost momentum of the Occupy movement as a cautionary note. Increasingly, network leaders are looking at how to adapt the process, structure, and operational expertise more traditionally associated with organizations to their needs—but without co-opting or diminishing the energy and momentum of their self-organizing networks…

Welcome to the World of “Both/And”

Today’s social change leaders—be they from business, government, or nonprofits—must learn to straddle the leadership mindsets and practices of both networks and organizations, and know when to use which approach. Leaders like Porway, and Henry Timms and Asha Curran of 92Y can help show us the way.

How do these leaders work with the “both/and” mindset?

First, they understand and leverage the strengths of both organizations and networks—and anticipate their limitations. As Timms describes it, leaders need to be “bilingual” and embrace what he has called “new power.” Networks can be powerful generators of new talent or innovation around complex multi-sector challenges. It’s useful to take a network approach when innovating new ideas, mobilizing and engaging others in the work, or wanting to expand reach and scale quickly. However, networks can dissipate easily without specific “handrails,” or some structure to guide and support their work. This is where they need some help from the organizational mindset and approach.

On the flip side, organizations are good at creating centralized structures to deliver products or services, manage risk, oversee quality control, and coordinate concrete functions like communications or fundraising. However, often that efficiency and effectiveness can calcify over time, becoming a barrier to new ideas and growth opportunities. When organizational boundaries are too rigid, it is difficult to engage the outside world in ideating or mobilizing on an issue. This is when organizations need an infusion of the “network mindset.”

 

…(More)

Beware of the gaps in Big Data


Edd Gent at E&T: “When the municipal authority in charge of Boston, Massachusetts, was looking for a smarter way to find which roads it needed to repair, it hit on the idea of crowdsourcing the data. The authority released a mobile app called Street Bump in 2011 that employed an elegantly simple idea: use a smartphone’s accelerometer to detect jolts as cars go over potholes and look up the location using the Global Positioning System. But the approach ran into a pothole of its own.The system reported a disproportionate number of potholes in wealthier neighbourhoods. It turned out it was oversampling the younger, more affluent citizens who were digitally clued up enough to download and use the app in the first place. The city reacted quickly, but the incident shows how easy it is to develop a system that can handle large quantities of data but which, through its own design, is still unlikely to have enough data to work as planned.

As we entrust more of our lives to big data analytics, automation problems like this could become increasingly common, with their errors difficult to spot after the fact. Systems that ‘feel like they work’ are where the trouble starts.

Harvard University professor Gary King, who is also founder of social media analytics company Crimson Hexagon, recalls a project that used social media to predict unemployment. The model was built by correlating US unemployment figures with the frequency that people used words like ‘jobs’, ‘unemployment’ and ‘classifieds’. A sudden spike convinced researchers they had predicted a big rise in joblessness, but it turned out Steve Jobs had died and their model was simply picking up posts with his name. “This was an example of really bad analytics and it’s even worse because it’s the kind of thing that feels like it should work and does work a little bit,” says King.

Big data can shed light on areas with historic information deficits, and systems that seem to automatically highlight the best course of action can be seductive for executives and officials. “In the vacuum of no decision any decision is attractive,” says Jim Adler, head of data at Toyota Research Institute in Palo Alto. “Policymakers will say, ‘there’s a decision here let’s take it’, without really looking at what led to it. Was the data trustworthy, clean?”…(More)”

Coming soon: The Conversation Global


Screen Shot 2016-09-22 at 8.54.58 AMThe Conversation, an independent news and commentary website produced by academics and journalists, launches its Global edition this month.

The Conversation Global will publish commentary, analysis and research from the academic community worldwide. We will engage scholars from across the world, featuring perspectives from the Global South and North on the most pressing international issues. All content will be published under Creative Commons.

The site is open and free for everyone to read.

The Wisdom of the Crowd is what science really needs


Science/Disrupt: “In a world where technology allows for global collaboration, and in a time when we’re finally championing diversity of thought, there are few barriers to getting the right people together to work on some of our most pressing problems. Governments and research labs are attempting to apply this mentality to science through what is known as ‘Citizen Science’ – research conducted in part by the public (amateur scientists) in partnership with the professionals.

The concept of Citizen Science is brilliant: moving science forward, faster, by utilising the wisdom and volume of the crowd. …

But Citizen Science goes beyond working directly with people with specific data to share. Zooniverse – the home of Citizen Science online – lists hundreds of projects which anyone can get involved with to help advance science. From mapping the galaxy and looking for comets, to seeking outAustralian wildlife and helping computers understand animal faces, the projects span across many subjects.

But when you dig deeper into the tasks being asked of these CitizenScientists, you find that – really – it’s a simple data capture activity. There’s no skill involved other than engaging your eyes to see and fingers to click and type. It’s not the wisdom of the crowd which is being tapped into.

You could argue that people are interested purely in being a part of important research – which of course is true for many – but it misses the point that scientists are simply missing out on a great resource of intellect at their fingertips.

There has been a rise of crowdsourced solutions over the last few years. rLoopis an organisation formed over Reddit to propose a Hyperloop transportation capsule; Techfugees is a Global community of technologists who team up to propose and build solutions to problems facing the increasing numbers of refugees around the world;  and XPRIZE is an open competition offering winning teams large sums of money and support to solve the global problems they select each year.

The difference between crowdsourcing and Citizen Science is that in the former, a high value is placed on ideas. There’s a general understanding that‘two minds are better than one’ and that by empowering a larger, more diverse pool of people to engage with important and purposeful work, a better solution will be found faster.

With Citizen Science, the mood is that of the public only being capable of playing hide and seek with pictures and completing menial, time consuming work that the scientists are simply too busy to do. …(More)”