education

Data Could Help Scholars Persuade, If Only They Were Willing to Use It

Curated on February 17, 2016August 15, 2018 by Stefaan Verhulst

Paul Basken at the Chronicle of Higher Education: “Thanks to what they’ve learned from university research, consultants like Matthew Kalmans have become experts in modern political persuasion. A co-founder of Applecart, a New York data firm, Mr. Kalmans specializes in shaping societal attitudes by using advanced analytical techniques to discover and exploit personal connections and friendships. His is one of a fast-growing collection of similar companies now raising millions of dollars, fattening businesses, and aiding political campaigns with computerized records of Facebook exchanges, high-school yearbooks, even neighborhood gossip.

Applecart uses that data to try to persuade people on a range of topics by finding voices they trust to deliver endorsements. “You can use this sort of technology to get people to purchase insurance at higher rates, get people to purchase a product, get people to do all sorts of other things that they might otherwise not be inclined to do,” said Mr. Kalmans, a 2014 graduate of the University of Pennsylvania. And in building such a valuable service, he’s found that the intellectual underpinnings are often free. “We are constantly reading academic papers to get ideas on how to do things better,” Mr. Kalmans said. That’s because scholars conduct the field experiments and subsequent tests that Mr. Kalmans needs to build and refine his models. “They do a lot of the infrastructural work that, frankly, a lot of commercial companies don’t have the in-house expertise to do,” he said of university researchers. Yet the story of Applecart stands in contrast to the dominant attitude and approach among university researchers themselves. Universities are full of researchers who intensively study major global problems such as environmental destruction and societal violence, then stop short when their conclusions point to the need for significant change in public behavior.

Some in academe consider that boundary a matter of principle rather than a systematic failure or oversight. “The one thing that we have to do is not be political,” Michael M. Crow, the usually paradigm-breaking president of Arizona State University, said this summer at a conference on academic engagement in public discourse. “Politics is a process that we are informing. We don’t have to be political to inform politicians or political actors.” But other academics contemplate that stance and see a missed opportunity to help convert the millions of taxpayer dollars spent on research into meaningful societal benefit. They include Dan M. Kahan, a professor of law and of psychology at Yale University who has been trying to help Florida officials cope with climate change. Mr. Kahan works with the four-county Southeast Florida Regional Climate Change Compact, which wants to redesign roads, expand public transit, and build pumping stations to prepare for harsher weather. But Mr. Kahan says he and his Florida partners have had trouble getting enough

But Mr. Kahan says he and his Florida partners have had trouble getting enough policy makers to seriously consider the scale of the problem and the necessary solutions. It’s frustrating, Mr. Kahan said, to see so much university research devoted to work inside laboratories on problems like climate, and comparatively little spent on real-world needs such as sophisticated messaging strategies. “There really is a kind of deficit in the research relating to actually operationalizing the kinds of insights that people have developed from research,” he said. That deficit appears to stem from academic culture, said Utpal M. Dholakia, a professor of marketing at Rice University whose work involves testing people’s self-control in areas such as eating and shopping. He then draws conclusions about whether regulations or taxes aimed at changing behaviors will be effective. Companies find advanced personal behavioral data highly useful, said Mr. Dholakia, who works on the side to help retailers devise sales strategies. But his university, he said, appears more interested in seeing him publish his findings than take the time to help policy makers make real-world use of them. “My dean gets very worried if I don’t publish a lot.” Because universities h

That deficit appears to stem from academic culture, said Utpal M. Dholakia, a professor of marketing at Rice University whose work involves testing people’s self-control in areas such as eating and shopping. He then draws conclusions about whether regulations or taxes aimed at changing behaviors will be effective. Companies find advanced personal behavioral data highly useful, said Mr. Dholakia, who works on the side to help retailers devise sales strategies. But his university, he said, appears more interested in seeing him publish his findings than take the time to help policy makers make real-world use of them. “My dean gets very worried if I don’t publish a lot.” …(More)

Improving government effectiveness: lessons from Germany

Curated on February 11, 2016July 19, 2019 by Stefaan Verhulst

Tom Gash at Global Government Forum: “All countries face their own unique challenges but advanced democracies also have much in common: the global economic downturn, aging populations, increasingly expensive health and pension spending, and citizens who remain as hard to please as ever.

At an event last week in Bavaria, attended by representatives of Bavaria’s governing party, the Christian Social Union (CSU) and their guests, it also became clear that there is a growing consensus that governments face another common problem. They have relied for too long on traditional legislation and regulation to drive change. The consensus was that simply prescribing in law what citizens and companies can and can’t do will not solve the complex problems governments are facing, that governments cannot legislate their way to improved citizen health, wealth and wellbeing….

…a number of developments …from which both UK and international policymakers and practitioners can learn to improve government effectiveness.

Behavioural economics: The Behavioural Insights Team (BIT), which span out of government in 2013 and is the subject of a new book by one of its founders and former IfG Director of Research, David Halpern, is being watched carefully by many countries abroad. Some are using its services, while others – including the New South Wales Government in Australia –are building their own skills in this area. BIT and others using similar principles have shown that using insights from social psychology – alongside an experimental approach – can help save money and improve outcomes. Well known successes include increasing the tax take through changing wording of reminder letters (work led by another IfG alumni Mike Hallsworth) and increasing pension take-up through auto-enrolment.
Market design: There is an emerging field of study which is examining how algorithms can be used to match people better with services they need – particularly in cases where it is unfair or morally repugnant to let allow a free market to operate. Alvin Roth, the Harvard Professor and Nobel prize winner, writes about these ‘matching markets’ in his book Who Gets What and Why – in which he also explains how the approach can ensure that more kidneys reach compatible donors, and children find the right education.
Big data: Large datasets can now be mined far more effectively, whether it is to analyse crime patterns to spot where police patrols might be useful or to understand crowd flows on public transport. The use of real-time information allows far more sophisticated deployment of public sector resources, better targeted at demand and need, and better tailored to individual preferences.
Transparency: Transparency has the potential to enhance both the accountability and effectiveness of governments across the world – as shown in our latest Whitehall Monitor Annual Report. The UK government is considered a world-leader for its transparency – but there are still areas where progress has stalled, including in transparency over the costs and performance of privately provided public services.
New management models: There is a growing realisation that new methods are best harnessed when supported by effective management. The Institute’s work on civil service reform highlights a range of success factors from past reforms in the UK – and the benefits of clear mechanisms for setting priorities and sticking to them, as is being attempted by governments new(ish) Implementation Taskforces and the Departmental Implementation Units currently cropping up across Whitehall. I looked overseas for a different model that clearly aligns government activities behind citizens’ concerns – in this case the example of the single non-emergency number system operating in New York City and elsewhere. This system supports a powerful, highly responsive, data-driven performance management regime. But like many performance management regimes it can risk a narrow and excessively short-term focus – so such tools must be combined with the mind-set of system stewardship that the Institute has long championed in its policymaking work.
Investment in new capability: It is striking that all of these developments are supported by technological change and research insights developed outside government. But to embed new approaches in government, there appear to be benefits to incubating new capacity, either in specialist departmental teams or at the centre of government….(More)”

A Government of the Future

Curated on February 10, 2016August 3, 2018 by Stefaan Verhulst

White House Fact Sheet on The President’s Fiscal Year 2017 Budget: “…The President is committed to driving lasting change in how Government works – change that makes a significant, tangible, and positive difference in the economy and the lives of the American people. Over the past seven years, the Administration has launched successful efforts to modernize and improve citizen-facing services, eliminate wasteful spending, reduce the Federal real property footprint, improve the use of evidence to improve program performance, and spur innovation in the private sector by opening to the public tens of thousands of Federal data sets and innovation assets at the national labs.

Supporting the President’s Management Agenda. The Budget includes investments to continue driving the President’s Management Agenda by improving the service we provide to the American public; leveraging the Federal Government’s buying power to bring more value and efficiency to how we use taxpayer dollars; opening Government data and research to the private sector to drive innovation and economic growth; promoting smarter information technology; modernizing permitting and environmental review processes; creating new Idea Labs to support employees with promising ideas; and, attracting and retaining the best talent in the Federal workforce.

Supporting Digital Service Delivery for Citizens. In 2014 the Administration piloted the U.S. Digital Service, a unit of innovators, entrepreneurs, and engineers. This team of America’s best digital experts has worked in collaboration with Federal agencies to implement streamlined and effective digital technology practices on the Nation’s highest priority programs. This work includes collaborating with the Department of Education to launch the new College Scorecard to give students, parents, and their advisors most reliable national data to help with college choice and supporting the U.S. Citizenship and Immigration Services (USCIS) transition to launch the new myUSCIS which makes it easier for users to access information about the immigration process and immigration services. To institutionalize the dramatic improvements that this approach has demonstrated, the Budget supports the Administration’s aggressive goal of hiring and placing 500 top technology and design experts to serve in the Government by January 2017.

Strengthening Federal Cybersecurity. As outlined above, the Budget provides $19 billion in resources for cybersecurity. This includes the creation of a new $3.1 billion revolving fund, the Information Technology Modernization Fund (ITMF), to retire the Government’s antiquated IT systems and transition to more secure and efficient modern IT systems, funding to streamline governance and secure Federal networks, and investments to strengthen the cybersecurity workforce and cybersecurity education across society.

Building Evidence and Encouraging Innovation. The President has made it clear that policy decisions should be driven by evidence so that the Federal government can do more of what works and less of what does not. The Administration’s evidence-based approaches have resulted in important gains in areas ranging from reducing veteran homelessness, to improving educational outcomes, to enhancing the effectiveness of international development programs. The Budget invests in expanding evidence-based approaches, developing and testing effective practices, and enhancing government’s capacity to build and use evidence, in particular by expanding access to administrative data and further developing Federal, State, local, and tribal data infrastructure.

Reorganizing Government to Succeed in the Global Economy. The Budget also includes proposals to consolidate and reorganize Government agencies to make them leaner and more efficient, and it increases the use of evidence and evaluation to ensure that taxpayer dollars are spent wisely on programs that work….(More). See also President Barack Obama’s FY 2017 Budget for the U.S. Government

The Metric Tide

Curated on February 6, 2016August 3, 2018 by Stefaan Verhulst

Book by James Wilsdon: Metrics evoke a mixed reaction from the research community. A commitment to using data and evidence to inform decisions makes many of us sympathetic, even enthusiastic, about the prospect of granular, real-time analysis of our own activities. Yet we only have to look around us, at the blunt use of metrics to be reminded of the pitfalls. Metrics hold real power: they are constitutive of values, identities and livelihoods.

How to exercise that power to positive ends is the focus of this book. Using extensive evidence-gathering, analysis and consultation, the authors take a thorough look at potential uses and limitations of research metrics and indicators. They explore the use of metrics across different disciplines, assess their potential contribution to the development of research excellence and impact and consider the changing ways in which universities are using quantitative indicators in their management systems. Finally, they consider the negative or unintended effects of metrics on various aspects of research culture.

Including an updated introduction from James Wilsdon, the book proposes a framework for responsible metrics and makes a series of targeted recommendations to show how responsible metrics can be applied in research management, by funders, and in the next cycle of the Research Excellence Framework.

The metric tide is certainly rising. Unlike King Canute, we have the agency and opportunity – and in this book, a serious body of evidence – to influence how it washes through higher education and research….(More)”.

Direct democracy may be key to a happier American democracy

Curated on January 30, 2016August 15, 2018 by Stefaan Verhulst

Benjamin Radcliff and Michael Krassa in the Conversation: “Is American democracy still “by the people, for the people?” According to recent research, it may not be. Martin Gilens at Princeton University confirms that the wishes of the American working and middle class play essentially no role in our nation’s policy making. A BBC story rightly summarized this with the headline: US Is an Oligarchy, Not a Democracy.

However new research by Benjamin Radcliff and Gregory Shufeldt suggests a ray of hope.

Ballot initiatives, they argue, may better serve the interests of ordinary Americans than laws passed by elected officials….

Today, 24 states allow citizens to directly vote on policy matters.

This year, more than 42 initiatives already are approved for the ballot in 18 states.

Voters in California will decide diverse questions including banning plastic bags, voter approval of state expenses greater than US$2 billion dollars, improving school funding, and the future of bilingual education.

The people of Colorado will vote on replacing their current medical insurance programs with a single payer system, and in Massachusetts people may consider legalizing recreational marijuana….

However, many have pointed to problems with direct democracy in the form of ballot initiatives.

Maxwell Sterns at the University of Maryland, for example, writes that legislatures are better because initiatives are the tools of special interests and minorities. In the end, initiatives are voted upon by an unrepresentative subset of the population, Sterns concludes.

Others like Richard Ellis of Willamette University argue that the time-consuming process of gathering signatures introduces a bias toward moneyed interests. Some suggest this has damaged direct democracy in California, where professional petition writers andpaid signature gatherers dominate the process. Moneyed interests also enjoy a natural advantage in having the resources that ordinary people lack to mount media campaigns to support their narrow interests.

To curb this kind of problem, bans on paying people per signature are proposed in many states, but have not yet passed any legislature. However, because Californians like direct democracy in principle, they have recently amended the process to allow for a review and revision, and they require mandatory disclosures about the funding and origins of ballot initiatives.

Finally, some say initiatives can be confusing for voters, like the two recent Ohio propositions concerning marijuana, where one ballot proposition essentially canceled out the other. Similarly, Mississippi’s Initiative 42 required marking the ballot in two places for approval but only one for disapproval, resulting in numerous nullified “yes” votes.

Routes to happiness

Despite these flaws, our research shows that direct democracy might improve happiness in two ways.

One is through its psychological effect on voters, making them feel they have a direct impact on policy outcomes. This holds even if they may not like, and thus vote against, a particular proposition. The second is that it may indeed produce policies more consistent with human well being.

The psychological benefits are obvious. By allowing people literally to be the government, just as in ancient Athens, people develop higher levels of political efficacy. In short, they may feel they have some control over their lives. Direct democracy can give people political capital because it offers a means by which citizens may place issues on the ballot for popular vote, giving them an opportunity both to set the agenda and to vote on the outcome.

We think this is important today given America’s declining faith in government. Overall today only 19 percent believe the government is run for all citizens. The same percentage trusts government to mostly do what is right. The poor and working classes are even more alienated….(More)”

The Crusade Against Multiple Regression Analysis

Curated on January 26, 2016August 3, 2018 by Stefaan Verhulst

Richard Nisbett at the Edge: (VIDEO) “…The thing I’m most interested in right now has become a kind of crusade against correlational statistical analysis—in particular, what’s called multiple regression analysis. Say you want to find out whether taking Vitamin E is associated with lower prostate cancer risk. You look at the correlational evidence and indeed it turns out that men who take Vitamin E have lower risk for prostate cancer. Then someone says, “Well, let’s see if we do the actual experiment, what happens.” And what happens when you do the experiment is that Vitamin E contributes to the likelihood of prostate cancer. How could there be differences? These happen a lot. The correlational—the observational—evidence tells you one thing, the experimental evidence tells you something completely different.

In the case of health data, the big problem is something that’s come to be called the healthy user bias, because the guy who’s taking Vitamin E is also doing everything else right. A doctor or an article has told him to take Vitamin E, so he does that, but he’s also the guy who’s watching his weight and his cholesterol, gets plenty of exercise, drinks alcohol in moderation, doesn’t smoke, has a high level of education, and a high income. All of these things are likely to make you live longer, to make you less subject to morbidity and mortality risks of all kinds. You pull one thing out of that correlate and it’s going to look like Vitamin E is terrific because it’s dragging all these other good things along with it.

This is not, by any means, limited to health issues. A while back, I read a government report in The New York Times on the safety of automobiles. The measure that they used was the deaths per million drivers of each of these autos. It turns out that, for example, there are enormously more deaths per million drivers who drive Ford F150 pickups than for people who drive Volvo station wagons. Most people’s reaction, and certainly my initial reaction to it was, “Well, it sort of figures—everybody knows that Volvos are safe.”

Let’s describe two people and you tell me who you think is more likely to be driving the Volvo and who is more likely to be driving the pickup: a suburban matron in the New York area and a twenty-five-year-old cowboy in Oklahoma. It’s obvious that people are not assigned their cars. We don’t say, “Billy, you’ll be driving a powder blue Volvo station wagon.” Because of this self-selection problem, you simply can’t interpret data like that. You know virtually nothing about the relative safety of cars based on that study.

I saw in The New York Times recently an article by a respected writer reporting that people who have elaborate weddings tend to have marriages that last longer. How would that be? Maybe it’s just all the darned expense and bother—you don’t want to get divorced. It’s a cognitive dissonance thing.

Let’s think about who makes elaborate plans for expensive weddings: people who are better off financially, which is by itself a good prognosis for marriage; people who are more educated, also a better prognosis; people who are richer; people who are older—the later you get married, the more likelihood that the marriage will last, and so on.

The truth is you’ve learned nothing. It’s like saying men who are a somebody III or IV have longer-lasting marriages. Is it because of the suffix there? No, it’s because those people are the types who have a good prognosis for a lengthy marriage.

A huge range of science projects are done with multiple regression analysis. The results are often somewhere between meaningless and quite damaging….(More)”

How Measurement Fails Doctors and Teachers

Curated on January 20, 2016August 3, 2018 by Stefaan Verhulst

Robert M. Wachter at the New York Times: “Two of our most vital industries, health care and education, have become increasingly subjected to metrics and measurements. Of course, we need to hold professionals accountable. But the focus on numbers has gone too far. We’re hitting the targets, but missing the point.

Through the 20th century, we adopted a hands-off approach, assuming that the pros knew best. Most experts believed that the ideal “products” — healthy patients and well-educated kids — were too strongly influenced by uncontrollable variables (the sickness of the patient, the intellectual capacity of the student) and were too complex to be judged by the measures we use for other industries.

By the early 2000s, as evidence mounted that both fields were producing mediocre outcomes at unsustainable costs, the pressure for measurement became irresistible. In health care, we saw hundreds of thousands of deaths from medical errors, poor coordination of care and backbreaking costs. In education, it became clear that our schools were lagging behind those in other countries.

So in came the consultants and out came the yardsticks. In health care, we applied metrics to outcomes and processes. Did the doctor document that she gave the patient a flu shot? That she counseled the patient about smoking? In education, of course, the preoccupation became student test scores.

All of this began innocently enough. But the measurement fad has spun out of control. There are so many different hospital ratings that more than 1,600 medical centers can now lay claim to being included on a “top 100,” “honor roll,” grade “A” or “best” hospitals list. Burnout rates for doctors top 50 percent, far higher than other professions. A 2013 study found that the electronic health record was a dominant culprit. Another 2013 study found that emergency room doctors clicked a mouse 4,000 times during a 10-hour shift. The computer systems have become the dark force behind quality measures.

Education is experiencing its own version of measurement fatigue. Educators complain that the focus on student test performance comes at the expense of learning. Art, music and physical education have withered, because, really,why bother if they’re not on the test?…

Thoughtful and limited assessment can be effective in motivating improvements and innovations, and in weeding out the rare but disproportionately destructive bad apples.

But in creating a measurement and accountability system, we need to tone down the fervor and think harder about the unanticipated consequences….(More)”

State of the Commons

Curated on December 9, 2015August 3, 2018 by Stefaan Verhulst

Creative Commons: “Creative Commoners have known all along that collaboration, sharing, and cooperation are a driving force for human evolution. And so for many it will come as no surprise that in 2015 we achieved a tremendous milestone: over 1.1 billion CC licensed photos, videos, audio tracks, educational materials, research articles, and more have now been contributed to the shared global commons…..

Whether it’s open education, open data, science, research, music, video, photography, or public policy, we are putting sharing and collaboration at the heart of the Web. In doing so, we are much closer to realizing our vision: unlocking the full potential of the Internet to drive a new era of development, growth, and productivity.

I am proud to share with you our 2015 State of the Commons report, our best effort to measure the immeasurable scope of the commons by looking at the CC licensed content, along with content marked as public domain, that comprise the slice of the commons powered by CC tools. We are proud to be a leader in the commons movement, and we hope you will join us as we celebrate all we have accomplished together this year. ….Report at https://stateof.creativecommons.org/2015/”

ClearGov aims to bring town finances into the 21st century

Curated on December 8, 2015July 19, 2019 by Stefaan Verhulst

Dan Adams at the Boston Globe: “Earlier this year, Hopkinton resident Chris Bullock was deciding how to vote on a tax increase that would fund a new school. He wanted to know how much the town spent on education, and how that compared to other nearby towns — reasonable questions that any engaged voter might ask.

But the information was surprisingly hard to find. Charts buried in the town’s 230-page annual report were inscrutable, the money scattered across various funds labeled with jargon. Even after piecing together a few figures, Bullock had no way to make sense of them. Was his town spending a lot or only a little on education?
That frustrating exercise was the genesis of ClearGov , a startup founded by Bullock that takes towns’ raw financial data and turns them into visually appealing online infographics, along with comparisons to similar towns nearby.

For residents, Bullock said, ClearGov aims to make local governments more approachable, transparent, and accountable.

And for officials, the software should help them parse voluminous budget spreadsheets to better compare their finances to those of nearby towns.

The site also encourages officials to answer questions posted on the site by residents and annotate their numbers with plain-English explanations of the policies behindthem….

Easton is one of five Massachusetts municipalities — along with Athol, Northfield, Oxford, and Warwick — to sign up for new paid service, inking a $1,500 deal with ClearGov in September that runs through June 2016, according to town officials.

The town’s ClearGov page gives a snapshot of its population and median home values and incomes, plus graphs of Easton’s debt load and rainy day reserve fund. There’s also a detailed, per-capita breakdown of where Easton’s revenues come from…(More).