Online and social media data as a flawed continuous panel survey


Fernando Diaz, Michael Gamon, Jake Hofman, Emre Kıcıman, and David Rothschild from Microsoft Research: “There is a large body of research on utilizing online activity to predict various real world outcomes, ranging from outbreaks of influenza to outcomes of elections. There is considerably less work, however, on using this data to understand topic-specific interest and opinion amongst the general population and specific demographic subgroups, as currently measured by relatively expensive surveys. Here we investigate this possibility by studying a full census of all Twitter activity during the 2012 election cycle along with comprehensive search history of a large panel of internet users during the same period, highlighting the challenges in interpreting online and social media activity as the results of a survey. As noted in existing work, the online population is a non-representative sample of the offline world (e.g., the U.S. voting population). We extend this work to show how demographic skew and user participation is non-stationary and unpredictable over time. In addition, the nature of user contributions varies wildly around important events. Finally, we note subtle problems in mapping what people are sharing or consuming online to specific sentiment or opinion measures around a particular topic. These issues must be addressed before meaningful insight about public interest and opinion can be reliably extracted from online and social media data…”

 

Data Mining Reddit Posts Reveals How to Ask For a Favor–And Get it


Emerging Technology From the arXiv: “There’s a secret to asking strangers for something and getting it. Now data scientists say they’ve discovered it by studying successful requests on the web

One of the more extraordinary phenomena on the internet is the rise of altruism and of websites designed to enable it. The Random Acts of Pizza section of the Reddit website is a good example.

People leave messages asking for pizza which others fulfil if they find the story compelling. As the site says: “because… who doesn’t like helping out a stranger? The purpose is to have fun, eat pizza and help each other out. Together, we aim to restore faith in humanity, one slice at a time.”

A request might go something like this: “It’s been a long time since my mother and I have had proper food. I’ve been struggling to find any kind of work so I can supplement my mom’s social security… A real pizza would certainly lift our spirits”. Anybody can then fulfil the order which is then marked on the site with a badge saying “got pizza’d”, often with notes of thanks.

That raises an interesting question. What kinds of requests are most successful in getting a response? Today, we get an answer thanks to the work of Tim Althoff at Stanford University and a couple of pals who lift the veil on the previously murky question of how to ask for a favour—and receive it.

They analysed how various features might be responsible for the success of a post, such as the politeness of the post; its sentiment, whether positive or negative for example; its length. The team also looked at the similarity of the requester to the benefactor; and also the status of the requester.

Finally, they examined whether the post contained evidence of need in the form of a narrative that described why the requester needed free pizza.

Althoff and co used a standard machine learning algorithm to comb through all the possible correlations in 70 per cent of the data, which they used for training. Having found various correlations, they tested to see whether this had predictive power in the remaining 30 per cent of the data. In other words, can their algorithm predict whether a previously unseen request will be successful or not?

It turns out that their algorithm makes a successful prediction about 70 per cent of the time. That’s far from perfect but much better than random guessing which is right only half the time.

So what kinds of factors are important? Narrative is a key part of many of the posts, so Althoff and co spent some time categorising the types of stories people use.

They divided the narratives into five types, those that mention: money; a job; being a student; family; and a final group that includes mentions of friends, being drunk, celebrating and so on, which Althoff and co call ‘craving’.

Of these, narratives about jobs, family and money increase the probability of success. Student narratives have no effect while craving narratives significantly reduce the chances of success. In other words, narratives that communicate a need are more successful than those that do not.

 “We find that clearly communicating need through the narrative is essential,” say Althoff and co. And evidence of reciprocation helps too.

(Given these narrative requirements, it is not surprising that longer requests tend to be more successful than short ones.)

So for example, the following request was successful because it clearly demonstrates both need and evidence of reciprocation.

“My gf and I have hit some hard times with her losing her job and then unemployment as well for being physically unable to perform her job due to various hand injuries as a server in a restaurant. She is currently petitioning to have unemployment reinstated due to medical reasons for being unable to perform her job, but until then things are really tight and ANYTHING would help us out right now.

I’ve been both a giver and receiver in RAOP before and would certainly return the favor again when I am able to reciprocate. It took everything we have to pay rent today and some food would go a long ways towards making our next couple of days go by much better with some food.”

By contrast, the ‘craving’ narrative below demonstrates neither and was not successful.

“My friend is coming in town for the weekend and my friends and i are so excited because we haven’t seen him since junior high. we are going to a high school football game then to the dollar theater after and it would be so nice if someone fed us before we embarked :)”

Althoff and co also say that the status of the requester is an important factor too. “We find that Reddit users with higher status overall (higher karma) or higher status within the subcommunity (previous posts) are significantly more likely to receive help,” they say.

But surprisingly, being polite does not help (except by offering thanks).

That’s interesting work. Until now, psychologists have never understood the factors that make requests successful, largely because it has always been difficult to separate the influence of the request from what is being requested.

The key here is that everybody making requests in this study wants the same thing—pizza. In one swoop, this makes the data significantly easier to tease apart.

An important line of future work will be in using his work to understand altruistic behaviour in other communities too…

Ref:  http://arxiv.org/abs/1405.3282 : How to Ask for a Favor: A Case Study on the Success of Altruistic Requests”

Digital Social Innovation


Nesta: Digital technologies and the internet play an increasingly important role in how social innovation happens. We call this phenomenon digital social innovation (DSI) and created a network map that we’re inviting you to join.
But what do we really mean by the term DSI? Peter Baeck and Alice Casey take a closer look at the tools and platforms you use to help you start your own digital social innovation project or get involved in those that others have already begun.
As part of our DSI research project, we have been looking across Europe, and beyond, to find out more about how people are using digital technology to make a social impact. We’re inviting people involved in creating these new social innovations to map their activities over at our open data community map www.digitalsocial.eu. We hope this will give everyone working on digital social innovation more exposure and help funders and researchers to shape their work to support this exciting field.

Below, we highlight our top 11 DSI trends to watch. Although you can read about each one separately, many of the most exciting innovations come from combining several of these trends to form entirely new systems. We’d love to gather more examples, so please add those you may have to our crowdmap here.

A civic-social platform for a new kind of citizen duty


Dirk Jan van der Wal at OpenSource.com: “In the Netherlands a community of civil servants has developed an open source platform for collaboration within the public sector. What began as a team of four has grown to over 75,000 registered users. What happened? And, why was open source key to the project’s success?
Society is rapidly changing. One change is the tremendous development of Internet and Web-based tools. These tools have opened up new ways for collaboration and sharing information. This is a big change for our society and democracy, having an impact on our politics. How does government change along with it?
A need to change the way government organizations worked and civil servants interacted too could not be ignored. Take for example, politicians resigning because of one tweet! Meanwhile, government organizations continually face the challenge of doing more with less funds. I think this increased the need to cooperate and share knowledge; it was not longer feasible for smaller communities to maintain knowledge on their own.
The question became: How do we cooperate in an efficient manner?
In the Netherlands, we have over 500 different government organizations: departments, city councils, provinces, and so on. All these organizations have their own information and communications technology (ICT) environment. So, with a growing network and discussions around multiple themes, it became clear that one of the basic requirements for cooperating efficiently is having a government-wide platform for people to communicate and work from.
So, a small team of four started Pleio for Dutch civil servants and citizens to meet each other, have discussions, and work together on things that matter to them.
(Pleio translates loosely in English to “government square.”)
As in real life, citizens and government officials work together across various teams, groups, and networks to think about and do work on projects that matter. Using the Pleio online platform, citizens and government officials can find and then engage with the right people to collaborate on a project or problem…”

Data.gov Turns Five


NextGov: “When government technology leaders first described a public repository for government data sets more than five years ago, the vision wasn’t totally clear.
“I just didn’t understand what they were talking about,” said Marion Royal of the General Services Administration, describing his first introduction to the project. “I was thinking, ‘this is not going to work for a number of reasons.’”
A few minutes later, he was the project’s program director. He caught onto and helped clarify that vision and since then has worked with a small team to help shepherd online and aggregate more than 100,000 data sets compiled and hosted by agencies across federal, state and local governments.
Many Americans still don’t know what Data.gov is, but chances are good they’ve benefited from the site, perhaps from information such as climate or consumer complaint data. Maybe they downloaded the Red Cross’ Hurricane App after Superstorm Sandy or researched their new neighborhood through a real estate website that drew from government information.
Hundreds of companies pull data they find on the site, which has seen 4.5 million unique visitors from 195 countries, according to GSA. Data.gov has proven a key part of President Obama’s open data policies, which aim to make government more efficient and open as well as to stimulate economic activity by providing private companies, organizations and individuals machine-readable ingredients for new apps, digital tools and programs.”

Open Data at Core of New Governance Paradigm


GovExec: “Rarely are federal agencies compared favorably with Facebook, Instagram, or other modern models of innovation, but there is every reason to believe they can harness innovation to improve mission effectiveness. After all, Aneesh Chopra, former U.S. Chief Technology Officer, reminded the Excellence in Government 2014 audience that government has a long history of innovation. From nuclear fusion to the Internet, the federal government has been at the forefront of technological development.
According to Chopra, the key to fueling innovation and economic prosperity today is open data. But to make the most of open data, government needs to adapt its culture. Chopra outlined three essential elements of doing so:

  1. Involve external experts – integrating outside ideas is second to none as a source of innovation.
  2. Leverage the experience of those on the front lines – federal employees who directly execute their agency’s mission often have the best sense of what does and does not work, and what can be done to improve effectiveness.
  3. Look to the public as a value multiplier – just as Facebook provides a platform for tens of thousands of developers to provide greater value, federal agencies can provide the raw material for many more to generate better citizen services.

In addition to these three broad elements, Chopra offered four specific levers government can use to help enact this paradigm shift:

  1. Democratize government data – opening government data to the public facilitates innovation. For example, data provided by the National Oceanic and Atmospheric Administration helps generate a 5 billion dollar industry by maintaining almost no intellectual property constraints on its weather data.
  2. Collaborate on technical standards – government can act as a convener of industry members to standardize technological development, and thereby increase the value of data shared.
  3. Issue challenges and prizes – incentivizing the public to get involved and participate in efforts to create value from government data enhances the government’s ability to serve the public.
  4. Launch government startups – programs like the Presidential Innovation Fellows initiative helps challenge rigid bureaucratic structures and permeate a culture of innovation.

Federal leaders will need a strong political platform to sustain this shift. Fortunately, this blueprint is also bipartisan, says Chopra. Political leaders on both sides of the aisle are already getting behind the movement to bring innovation to the core of government..

Three projects meet the European Job Challenge and receive the Social Innovation Prize


EU Press Release: “Social innovation can be a tool to create new or better jobs, while giving an answer to pressing challenges faced by Europe. Today, Michel Barnier, European Commissioner, has awarded three European Social Innovation prizes to ground-breaking ideas to create new types of work and address social needs. The winning projects aim to help disadvantaged women by employing them to create affordable and limited fashion collections, create jobs in the sector of urban farming, and convert abandoned social housing into learning spaces and entrepreneurship labs.

After the success of the first edition in 2013, the European Commission launched a second round of the Social Innovation Competition in memory of Diogo Vasconcelos1. Its main goal was to invite Europeans to propose new solutions to answer The Job Challenge. The Commission received 1,254 ideas out of which three were awarded with a prize of €30,000 each.

Commissioner Michel Barnier said: “We believe that the winning projects can take advantage of unmet social needs and create sustainable jobs. I want these projects to be scaled up and replicated and inspire more social innovations in Europe. We need to tap into this potential to bring innovative solutions to the needs of our citizens and create new types of work.”

More informationon the Competition page

More jobs for Europe – three outstanding ideas

The following new and exceptional ideas are the winners of the second edition of the European Social Innovation Competition:

  • ‘From waste to wow! QUID project’ (Italy): fashion business demands perfection, and slightly damaged textile cannot be used for top brands. The project intends to recycle this first quality waste into limited collections and thereby provide jobs to disadvantaged women. This is about creating highly marketable products and social value through recycling.

  • ‘Urban Farm Lease’ (Belgium): urban agriculture could provide 6,000 direct jobs in Brussels, and an additional 1,500 jobs considering indirect employment (distribution, waste management, training or events). The project aims at providing training, connection and consultancy so that unemployed people take advantage of the large surfaces available for agriculture in the city (e.g. 908 hectares of land or 394 hectares of suitable flat roofs).

  • ‘Voidstarter’ (Ireland): all major cities in Europe have “voids”, units of social housing which are empty because city councils have insufficient budgets to make them into viable homes. At the same time these cities also experience pressure with social housing provision and homelessness. Voidstarter will provide unemployed people with learning opportunities alongside skilled tradespersons in the refurbishing of the voids.”

The Collective Intelligence Handbook: an open experiment


Michael Bernstein: “Is there really a wisdom of the crowd? How do we get at it and understand it, utilize it, empower it?
You probably have some ideas about this. I certainly do. But I represent just one perspective. What would an economist say? A biologist? A cognitive or social psychologist? An artificial intelligence or human-computer interaction researcher? A communications scholar?
For the last two years, Tom Malone (MIT Sloan) and I (Stanford CS) have worked to bring together all these perspectives into one book. We are nearing completion, and the Collective Intelligence Handbook will be published by the MIT Press later this year. I’m still relatively dumbfounded by the rockstar lineup we have managed to convince to join up.

It’s live.

Today we went live with the authors’ current drafts of the chapters. All the current preprints are here: http://cci.mit.edu/CIchapterlinks.html

And now is when you come in.

But we’re not done. We’d love for you — the crowd — to help us make this book better. We envisioned this as an open process, and we’re excited that all the chapters are now at a point where we’re ready for critique, feedback, and your contributions.
There are two ways you can help:

  • Read the current drafts and leave comments inline in the Google Docs to help us make them better.
  • Drop suggestions in the separate recommended reading list for each chapter. We (the editors) will be using that material to help us write an introduction to each chapter.

We have one month. The authors’ final chapters are due to us in mid-June. So off we go!”

Here’s what’s in the book:

Chapter 1. Introduction
Thomas W. Malone (MIT) and Michael S. Bernstein (Stanford University)
What is collective intelligence, anyway?
Chapter 2. Human-Computer Interaction and Collective Intelligence
Jeffrey P. Bigham (Carnegie Mellon University), Michael S. Bernstein (Stanford University), and Eytan Adar (University of Michigan)
How computation can help gather groups of people to tackle tough problems together.
Chapter 3. Artificial Intelligence and Collective Intelligence
Daniel S. Weld (University of Washington), Mausam (IIT Delhi), Christopher H. Lin (University of Washington), and Jonathan Bragg (University of Washington)
Mixing machine intelligence with human intelligence could enable a synthesized intelligent actor that brings together the best of both worlds.
Chapter 4. Collective Behavior in Animals: An Ecological Perspective
Deborah M. Gordon (Stanford University)
How do groups of animals work together in distributed ways to solve difficult problems?
Chapter 5. The Wisdom of Crowds vs. the Madness of Mobs
Andrew W. Lo (MIT)
Economics has studied a collectively intelligent forum — the market — for a long time. But are we as smart as we think we are?
Chapter 6. Collective Intelligence in Teams and Organizations
Anita Williams Woolley (Carnegie Mellon University), Ishani Aggarwal (Georgia Tech), Thomas W. Malone (MIT)
How do the interactions between groups of people impact how intelligently that group acts?
Chapter 7. Cognition and Collective Intelligence
Mark Steyvers (University of California, Irvine), Brent Miller (University of California, Irvine)
Understanding the conditions under which people are smart individually can help us predict when they might be smart collectively.

Chapter 8. Peer Production: A Modality of Collective Intelligence
Yochai Benkler (Harvard University), Aaron Shaw (Northwestern University), Benjamin Mako Hill (University of Washington)
What have collective efforts such as Wikipedia taught us about how large groups come together to create knowledge and creative artifacts?

Public service workers will have to become Jacks and Jills of all trades


Catherine Needham in the Guardian: “When Kent county council was looking to save money a couple of years ago, it hit upon the idea of merging the roles of library manager and registrar. Library managers were expected to register births and deaths on top of their existing duties, and registrars took on roles in libraries. One former library manager chose to leave the service as a result. It wasn’t, he said, what he signed up for: “I don’t associate the skills in running a library with those of a registrar. I don’t have the emotional skill to do it.”
Since the council was looking to cut staff numbers, it was probably not too troubled by his departure. But this does raise questions about how to support staff who are being asked to work well beyond their professional boundaries.
In our 21st Century Public Servant project at the University of Birmingham, we have found that this trend is evident across public services. We interviewed local government managers who said staff needed to think differently about their skills. As one put it: “We need to use people’s latent talent – if you are a librarian, for example, a key skill will be working with people from the local community. It’s about a different background mindset: ‘I am not just here to do a specific job, but to help the people of this town.'”

The skills of this generic public service worker include interpersonal skills (facilitation, empathy, political skills), analysing skills (sorting evidence, making judgements, offering critique and being creative), organisation (particularly for group work and collaboration) and communication skills (such as using social media and multimedia resources).
The growing interest in genericism seems to have two main drivers. The first, of course, is austerity. Cost cutting on an unprecedented scale in local authorities requires those staff that survive the waves of redundancies to be willing to take on new roles and work in multi-purpose settings. The second is the drive for whole-person approaches in which proper engagement with the public might require staff to cross traditional sector boundaries.
It is good that public service workers are being granted greater flexibility. But there are two main limitations to this move to greater genericism. The first is that multi-tasking in an era of cost cutting can look a lot like deprofessionalisation. Within social work, for example, concerns have been expressed about the downgrading of social work posts (by appointing brokers in their place, say) and the resulting loss of professional skills and knowledge.
A second limitation is that skills training continues to be sectoral, failing to catch up with the move to genericism….”

New Research Suggests Collaborative Approaches Produce Better Plans


JPER: “In a previous blog post (see, http://goo.gl/pAjyWE), we discussed how many of the most influential articles in the Journal of Planning Education and Research (and in peer publications, like JAPA) over the last two decades have focused on communicative or collaborative planning. Proponents of these approaches, most notably Judith Innes, Patsy Healey, Larry Susskind, and John Forester, developed the idea that the collaborative and communicative structures that planners use impact the quality, legitimacy, and equity of planning outcomes. In practice, communicative theory has led to participatory initiatives, such as those observed in New Orleans (post-Katrina, http://goo.gl/A5J5wk), Chattanooga (to revitalize its downtown and riverfront, http://goo.gl/zlQfKB), and in many other smaller efforts to foment wider involvement in decision making. Collaboration has also impacted regional governance structures, leading to more consensus based forms of decision making, notably CALFED (SF Bay estuary governance, http://goo.gl/EcXx9Q) and transportation planning with Metropolitan Planning Organizations (MPOs)….
Most studies testing the implementation of collaborative planning have been case studies. Previous work by authors such as Innes and Booher, has provided valuable qualitative data about collaboration in planning, but few studies have attempted to empirically test the hypothesis that consensus building and participatory practices lead to better planning outcomes.
Robert Deyle (Florida State) and Ryan Weidenman (Atkins Global) build on previous case study research by surveying officials in involved in developing long-range transportation plans in 88 U.S. MPOs about the process and outcomes of those plans. The study tests the hypothesis that collaborative processes provide better outcomes and enhanced long-term relationships in situations where “many stakeholders with different needs” have “shared interests in common resources or challenges” and where “no actor can meet his/her interests without the cooperation of many others (Innes and Booher 2010, 7; Innes and Gruber 2005, 1985–2186). Current theory posits that consensus-based collaboration requires 1) the presence of all relevant interests, 2) mutual interdependence for goal achievement, and 3) honest and authentic dialog between participants (Innes and Booher 2010, 35–36, Deyle and Weidenmann, 2014).

Figure 2 Deyle and Weidenman (2014)
By surveying planning authorities, the authors found that most of the conditions (See Figure 2, above) posited in collaborative planning literature had statistically significant impacts on planning outcomes.These included perceptions of plan quality, participant satisfaction with the plan, as well as intangible outcomes that benefit both the participants and their ongoing collaboration efforts. However, having a planning process in which all or most decisions were made by consensus did not improve outcomes.  ….
Deyle, Robert E., and Ryan E. Wiedenman. “Collaborative Planning by Metropolitan Planning Organizations A Test of Causal Theory.” Journal of Planning Education and Research (2014): 0739456X14527621.
To access this article FREE until May 31 click the following links: Online, http://goo.gl/GU9inf, PDF, http://goo.gl/jehAf1.”