This algorithm can predict a revolution

Curated on Posted on by Stefaan Verhulst

Russell Brandom at the Verge: “For students of international conflict, 2013 provided plenty to examine. There was civil war in Syria, ethnic violence in China, and riots to the point of revolution in Ukraine. For those working at Duke University’s Ward Lab, all specialists in predicting conflict, the year looks like a betting sheet, full of predictions that worked and others that didn’t pan out.

Guerrilla campaigns intensified, proving out the prediction

When the lab put out their semiannual predictions in July, they gave Paraguay a 97 percent chance of insurgency, largely based on reports of Marxist rebels. The next month, guerrilla campaigns intensified, proving out the prediction. In the case of China’s armed clashes between Uighurs and Hans, the models showed a 33 percent chance of violence, even as the cause of each individual flare-up was concealed by the country’s state-run media. On the other hand, the unrest in Ukraine didn’t start raising alarms until the action had already started, so the country was left off the report entirely.

According to Ward Lab’s staff, the purpose of the project isn’t to make predictions but to test theories. If a certain theory of geopolitics can predict an uprising in Ukraine, then maybe that theory is onto something. And even if these specialists could predict every conflict, it would only be half the battle. “It’s a success only if it doesn’t come at the cost of predicting a lot of incidents that don’t occur,” says Michael D. Ward, the lab’s founder and chief investigator, who also runs the blog Predictive Heuristics. “But it suggests that we might be on the right track.”

If a certain theory of geopolitics can predict an uprising in Ukraine, maybe that theory is onto something

Forecasting the future of a country wasn’t always done this way. Traditionally, predicting revolution or war has been a secretive project, for the simple reason that any reliable prediction would be too valuable to share. But as predictions lean more on data, they’ve actually become harder to keep secret, ushering in a new generation of open-source prediction models that butt against the siloed status quo.

Will this country’s government face an acute existential threat in the next six months?

The story of automated conflict prediction starts at the Defense Advance Research Projects Agency, known as the Pentagon’s R&D wing. In the 1990s, DARPA wanted to try out software-based approaches to anticipating which governments might collapse in the near future. The CIA was already on the case, with section chiefs from every region filing regular forecasts, but DARPA wanted to see if a computerized approach could do better. They looked at a simple question: will this country’s government face an acute existential threat in the next six months? When CIA analysts were put to the test, they averaged roughly 60 percent accuracy, so DARPA’s new system set the bar at 80 percent, looking at 29 different countries in Asia with populations over half a million. It was dubbed ICEWS, the Integrated Conflict Early Warning System, and it succeeded almost immediately, clearing 80 percent with algorithms built on simple regression analysis….

On the data side, researchers at Georgetown University are cataloging every significant political event of the past century into a single database called GDELT, and leaving the whole thing open for public research. Already, projects have used it to map the Syrian civil war and diplomatic gestures between Japan and South Korea, looking at dynamics that had never been mapped before. And then, of course, there’s Ward Lab, releasing a new sheet of predictions every six months and tweaking its algorithms with every development. It’s a mirror of the same open-vs.-closed debate in software — only now, instead of fighting over source code and security audits, it’s a fight over who can see the future the best.”

The Problem With Serious Games–Solved

Curated on Posted on by Stefaan Verhulst
Emerging Technology From the arXiv:” Serious games are becoming increasingly popular but the inability to generate realistic new content has hampered their progress. Until now.

Here’s an imaginary scenario: you’re a law enforcement officer confronted with John, a 21-year-old male suspect who is accused of breaking into a private house on Sunday evening and stealing a laptop, jewellery and some cash. Your job is to find out whether John has an alibi and if so whether it is coherent and believable.
That’s exactly the kind of scenario that police officers the world over face on a regular basis. But how do you train for such a situation? How do you learn the skills necessary to gather the right kind of information?
An increasingly common way of doing this is with serious games, those designed primarily for purposes other than entertainment. In the last 10 years or so, medical, military and commercial organisations all over the world began to experiment with game-based scenarios that are designed to teach people how to perform their jobs and tasks in realistic situations.
But there is a problem with serious games which require realistic interaction is with another person. It’s relatively straightforward to design one or two scenarios that are coherent, lifelike and believable but it’s much harder to generate them continually on an ongoing basis.
Imagine in the example above, that John is a computer-generated character. What kind of activities could he describe that would serve as a believable, coherent alibi for Sunday evening? And how could he do it a thousand times, each describing a different realistic alibi. Therein lies the problem.
Today, Sigal Sina at Bar-Ilan University in Israel, and a couple pals, say they’ve solved this probelm. These guys have come up with a novel way of generating ordinary, realistic scenarios that can be cut and pasted into a serious game to serve exactly this purpose. The secret sauce in their new approach is to crowdsource the new scenarios from real people using Amazon’s Mechanical Turk service.
The approach is straightforward. Sina and co simply ask Turkers to answer a set of questions asking what they did during each one-hour period throughout various days, offering bonuses to those who provide the most varied detail.
They then analyse the answers, categorising activities by factors such as the times they are performed, the age and sex of the person doing it, the number of people involved and so on.
This then allows a computer game to cut and paste activities into the action at appropriate times. So for example, the computer can select an appropriate alibi for John on a Sunday evening by choosing an activity described by a male Turker for the same time while avoiding activitiesthat a woman might describe for a Friday morning, which might otherwise seem unbelievable. The computer also changes certain details in the narrative, such as names, locations and so on to make the narrative coherent with John’s profile….
That solves a significant problem with serious games. Until now, developers have had to spend an awful lot of time producing realistic content, a process known as procedural content generation. That’s always been straightforward for things like textures, models and terrain in game settings. Now, thanks to this new crowdsourcing technique, it can be just as easy for human interactions in serious games too.
Ref:  arxiv.org/abs/1402.5034 : Using the Crowd to Generate Content for Scenario-Based Serious-Games”

Four Threats to American Democracy

Curated on Posted on by Stefaan Verhulst

Jared Diamond in Governance: “The U.S. government has spent the last two years wrestling with a series of crises over the federal budget and debt ceiling. I do not deny that our national debt and the prospect of a government shutdown pose real problems. But they are not our fundamental problems, although they are symptoms of them. Instead, our fundamental problems are four interconnected issues combining to threaten a breakdown of effective democratic government in the United States.
Why should we care? Let’s remind ourselves of the oft-forgotten reasons why democracy is a superior form of government (provided that it works), and hence why its deterioration is very worrisome. (Of course, I acknowledge that there are many countries in which democracy does not work, because of the lack of a national identity, of an informed electorate, or of both). The advantages of democracy include the following:

  • In a democracy, one can propose and discuss virtually any idea, even if it is initially unpalatable to the government. Debate may reveal the idea to be the best solution, whereas in a dictatorship the idea would not have gotten debated, and its virtues would not have been discovered.
  • In a democracy, citizens and their ideas get heard. Hence, without democracy, people are more likely to feel unheard and frustrated and to resort to violence.
  • Compromise is essential to a democracy. It enables us to avoid tyranny by the majority or (conversely) paralysis of government through vetoes exercised by a frustrated minority.
  • In modern democracies, all citizens can vote. Hence, government is motivated to invest in all citizens, who thereby receive the opportunity to become productive, rather than just a small dictatorial elite receiving that opportunity.

Why should we Americans keep reminding ourselves of those fundamental advantages of democracies? I would answer: not only in order to motivate ourselves to defend our democratic processes, but also because increasing numbers of Americans today are falling into the trap of envying the supposed efficiency of China’s dictatorship. Yes, it is true that dictatorships, by closing debate, can sometimes implement good policies faster than can the United States, as has China in quickly converting to lead-free gasoline and building a high-speed rail network. But dictatorships suffer from a fatal disadvantage. No one, in the 5,400 years of history of centralized government on all the continents, has figured out how to ensure that a dictatorship will embrace only good policies. Dictatorships also prevent the public debate that helps to avert catastrophic policies unparalleled in any large modern First World democracy—such as China’s quickly abolishing its educational system, sending its teachers out into the fields, and creating the world’s worst air pollution.
That is why democracy, given the prerequisites of an informed electorate and a basic sense of common interest, is the best form of government—at least, better than all the alternatives that have been tried, as Winston Churchill quipped. Our form of government is a big part of the explanation why the United States has become the richest and most powerful country in the world. Hence, an undermining of democratic processes in the United States means throwing away one of our biggest advantages. Unfortunately, that is what we are now doing, in four ways.
First, political compromise has been deteriorating in recent decades, and especially in the last five years. That deterioration can be measured as the increase in Senate rejections of presidential nominees whose approvals used to be routine, the increasing use of filibusters by the minority party, the majority party’s response of abolishing filibusters for certain types of votes, and the decline in number of laws passed by Congress to the lowest level of recent history. The reasons for this breakdown in political compromise, which seems to parallel increasing levels of nastiness in other areas of American life, remain debated. Explanations offered include the growth of television and then of the Internet, replacing face-to-face communication, and the growth of many narrowly partisan TV channels at the expense of a few broad-public channels. Even if these reasons hold a germ of truth, they leave open the question why these same trends operating in Canada and in Europe have not led to similar deterioration of political compromise in those countries as well.
Second, there are increasing restrictions on the right to vote, weighing disproportionately on voters for one party and implemented at the state level by the other party. Those obstacles include making registration to vote difficult and demanding that registered voters show documentation of citizenship when they present themselves at the polls. Of course, the United States has had a long history of denying voting rights to blacks, women, and other groups. But access to voting had been increasing in the last 50 years, so the recent proliferation of restrictions reverses that long positive trend. In addition to those obstacles preventing voter registration, the United States has by far the lowest election turnout among large First World democracies: under 60% of registered voters in most presidential elections, 40% for congressional elections, and 20% for the recent election for mayor of my city of Los Angeles. (A source of numbers for this and other comparisons that I shall cite is an excellent recent book by Howard Steven Friedman, The Measure of a Nation). And, while we are talking about elections, let’s not forget the astronomical recent increase in costs and durations of election campaigns, their funding by wealthy interests, and the shift in campaign pitches to sound bites. Those trends, unparalled in other large First World democracies, undermine the democratic prerequisite of a well-informed electorate.
A third contributor to the growing breakdown of democracy is our growing gap between rich and poor. Among our most cherished core values is our belief that the United States is a “land of opportunity,” and that we uniquely offer to our citizens the potential for rising from “rags to riches”—provided that citizens have the necessary ability and work hard. This is a myth. Income and wealth disparity in the United States (as measured by the Gini index of equality/inequality, and in other ways) is much higher in the United States than in any other large First World democracy. So is hereditary socioeconomic immobility, that is, the probability that a son’s relative income will just mirror his father’s relative income, and that sons of poor fathers will not become wealthy. Part of the reason for those depressing facts is inequality of educational opportunities. Children of rich Americans tend to receive much better educations than children of poor Americans.
That is bad for our economy, because it means that we are failing to develop a large fraction of our intellectual capital. It is also bad for our political stability, because poor parents who correctly perceive that their children are not being given the opportunity to succeed may express their resulting frustration in violence. Twice during my 47 years of residence in Los Angeles, in 1964 and 1993, frustration in poor areas of Los Angeles erupted into violence, lootings, and killings. In the 1993 riots, when police feared that rioters would spill into the wealthy suburb of Beverly Hills, all that the outnumbered police could do to protect Beverly Hills was to string yellow plastic police tape across major streets. As it turned out, the rioters did not try to invade Beverly Hills in 1993. But if present trends causing frustration continue, there will be more riots in Los Angeles and other American cities, and yellow plastic police tape will not suffice to contain the rioters.
The remaining contributor to the decline of American democracy is the decline of government investment in public purposes, such as education, infrastructure, and nonmilitary research and development. Large segments of the American populace deride government investment as “socialism.” But it is not socialism. On the contrary, it is one of the longest established functions of government. Ever since the rise of the first governments 5,400 years ago, governments have served two main functions: to maintain internal peace by monopolizing force, settling disputes, and forbidding citizens to resort to violence in order to settle disputes themselves; and to redistribute individual wealth for investing in larger aims—in the worst cases, enriching the elite; in the best cases, promoting the good of society as a whole. Of course, some investment is private, by wealthy individuals and companies expecting to profit from their investments. But many potential payoffs cannot attract private investment, either because the payoff is so far off in the future (such as the payoff from universal primary school education), or because the payoff is diffused over all of society rather than concentrated in areas profitable to the private investor (such as diffused benefits of municipal fire departments, roads, and broad education). Even the most passionate American supporters of small government do not decry as socialism the funding of fire departments, interstate highways, and public schools.

Algorithmic Accountability Reporting: On the Investigation of Black Boxes

Curated on Posted on by Stefaan Verhulst

New report by by Nicholas Diakopoulos: “The past three years have seen a small profusion of websites, perhaps as many as 80, spring up to capitalize on the high interest that mug shot photos generate online.1 Mug shots are public record, artifacts of an arrest, and these websites collect, organize, and optimize the photos so that they’re found more easily online. Proponents of such sites argue that the public has a right to know if their neighbor, romantic date, or colleague has an arrest record. Still, mug shots are not proof of conviction; they don’t signal guilt.
Having one online is likely to result in a reputational blemish; having that photo ranked as the first result when someone searches for your name on Google turns that blemish into a garish reputational wound, festering in facile accessibility. Some of these websites are exploiting this, charging peo- ple to remove their photo from the site so that it doesn’t appear in online searches. It’s reputational blackmail. And remember, these people aren’t necessarily guilty of anything.
To crack down on the practice, states like Oregon, Georgia, and Utah have passed laws requiring these sites to take down the photos if the person’s record has been cleared. Some credit card companies have stopped processing payments for the seediest of the sites. Clearly both legal and market forces can help curtail this activity, but there’s another way to deal with the issue too: algorithms. Indeed, Google recently launched updates to its ranking algorithm that down-weight results from mug shot websites, basically treating them more as spam than as legitimate information sources.2 With a single knock of the algorithmic gavel, Google declared such sites illegitimate.
At the turn of the millennium, 14 years ago, Lawrence Lessig taught us that “code is law”—that the architecture of systems, and the code and algorithms that run them, can be powerful influences on liberty.3 We’re living in a world now where algorithms adjudicate more and more consequential decisions in our lives. It’s not just search engines either; it’s everything from online review systems to educational evaluations, the operation of markets to how political campaigns are run, and even how social services like welfare and public safety are managed. Algorithms, driven by vast troves of data, are the new power brokers in society.
As the mug shots example suggests, algorithmic power isn’t necessarily detrimental to people; it can also act as a positive force. The intent here is not to demonize algorithms, but to recognize that they operate with biases like the rest of us.4 And they can make mistakes. What we generally lack as a public is clarity about how algorithms exercise their power over us. With that clarity comes an increased ability to publicly debate and dialogue the merits of any particular algorithmic power. While legal codes are available for us to read, algorithmic codes are more opaque, hidden behind layers of technical complexity. How can we characterize the power that various algorithms may exert on us? And how can we better understand when algo- rithms might be wronging us? What should be the role of journalists in holding that power to account?
In the next section I discuss what algorithms are and how they encode power. I then describe the idea of algorithmic accountability, first examining how algorithms problematize and sometimes stand in tension with transparency. Next, I describe how reverse engineering can provide an alternative way to characterize algorithmic power by delineating a conceptual model that captures different investigative scenarios based on reverse engineering algorithms’ input-output relationships. I then provide a number of illustrative cases and methodological details on how algorithmic accountability reporting might be realized in practice. I conclude with a discussion about broader issues of human resources, legality, ethics, and transparency.”

Focus on Migration: A tech ‘wiki’ site could improve lives

Curated on Posted on by Stefaan Verhulst

Max Martin in SciDev: “Wikipedia is probably the best example of a website that allows users to share and edit information in real time. But several other sites based on the ‘wiki’ model provide a sharing platform specifically for technologies that could help improve lives in the developing world.
One such site, Appropedia, is aimed at collaborative solutions in sustainability, appropriate technology and poverty reduction. Appropedia has had 50 million hits since its 2006 inception and is getting a facelift that will allow it to reach more people.
Such a one-stop information point offers tremendous scope for informing people on the move about green, low-cost and locally owned technologies. A website like Appropedia could function as a clearing house for information on technologies that could make life easier for migrants who are forced to travel and live rough in poor settings — as long as the information is reliable.
For example, displaced people building new homes after a disaster has struck face many choices over the materials they use, as I’ve written previously. The wiki site could be a place for them to swap experiences and learn what has worked for others in different settings.
It could also host advice for people on the move about affordable transport, healthcare and humanitarian aid locations, plus tips for staying safe while travelling in unfamiliar territory and what to pack when camping out in the open.
It could also help channel relevant innovations from other settings to migrants. For example, some villagers in flood-prone areas of Bangladesh grow crops on ‘floating gardens’ made using bamboo-pole rafts lined with soil water hyacinths and cow dung. [1] A local group in India’s frequently flooded Bihar state has shown how to make a life jacket using just plastic bottles, sticky tape, fast-drying cotton and thread. [2] Both of these concepts could be useful for other peoples affected by floods and a dedicated wiki could help disseminate know-how and review the technologies’ safety, reliability and suitability for different locations.
Of course, an information wiki for migrants must offer reliable information. This could be achieved by involving a specialist agency or a consortium of humanitarian groups who could invite experts and local practitioners to review and edit posts.”

Tim Berners-Lee: we need to re-decentralise the web

Curated on Posted on by Stefaan Verhulst

Wired:  “Twenty-five years on from the web’s inception, its creator has urged the public to re-engage with its original design: a decentralised internet that at its very core, remains open to all.
Speaking with Wired editor David Rowan at an event launching the magazine’s March issue, Tim Berners-Lee said that although part of this is about keeping an eye on for-profit internet monopolies such as search engines and social networks, the greatest danger is the emergence of a balkanised web.
“I want a web that’s open, works internationally, works as well as possible and is not nation-based,” Berners-Lee told the audience… “What I don’t want is a web where the  Brazilian government has every social network’s data stored on servers on Brazilian soil. That would make it so difficult to set one up.”
It’s the role of governments, startups and journalists to keep that conversation at the fore, he added, because the pace of change is not slowing — it’s going faster than ever before. For his part Berners-Lee drives the issue through his work at the Open Data Institute, World Wide Web Consortium and World Wide Web Foundation, but also as an MIT professor whose students are “building new architectures for the web where it’s decentralised”. On the issue of monopolies, Berners-Lee did say it’s concerning to be “reliant on big companies, and one big server”, something that stalls innovation, but that competition has historically resolved these issues and will continue to do so.
The kind of balkanised web he spoke about, as typified by Brazil’s home-soil servers argument or Iran’s emerging intranet, is partially being driven by revelations of NSA and GCHQ mass surveillance. The distrust that it has brewed, from a political level right down to the threat of self-censorship among ordinary citizens, threatens an open web and is, said Berners-Lee,  a greater threat than censorship. Knowing the NSA  may be breaking commercial encryption services could result in the emergence of more networks like China’s Great Firewall, to “protect” citizens. This is why we need a bit of anti-establishment push back, alluded to by Berners-Lee.”

Unbundling the nation state

Curated on Posted on by Stefaan Verhulst

The Economist on Government-to-government trade: “NIGERIAN pineapple for breakfast, Peruvian quinoa for lunch and Japanese sushi for dinner. Two centuries ago, when David Ricardo advocated specialisation and free trade, the notion that international exchange in goods and services could make such a cosmopolitan diet commonplace would have seemed fanciful.
Today another scenario may appear equally unlikely: a Norwegian government agency managing Algeria’s sovereign-wealth fund; German police overseeing security in the streets of Mumbai; and Dubai playing the role of the courthouse of the Middle East. Yet such outlandish possibilities are more than likely if a new development fulfils its promise. Ever more governments are trading with each other, from advising lawmakers to managing entire services. They are following businesses, which have long outsourced much of what they do. Is this the dawn of the government-to-government era?
Such “G2G” trade is not new, though the name may be. After the Ottoman empire defaulted on its debt in 1875 foreign lenders set up an “Ottoman Public Debt Administration”, its governing council packed with European government officials. At its peak it had 9,000 employees, more than the empire’s finance ministry. And the legacy of enforced G2G trade—colonialism, as it was known—is still visible even today. Britain’s Privy Council is the highest court of appeal for many Commonwealth countries. France provides a monetary-policy service to several west African nations by managing their currency, the CFA franc.
One reason G2G trade is growing is that it is a natural extension of the trend for governments to pinch policies from each other. “Policymaking now routinely occurs in comparative terms,” says Jamie Peck of the University of British Columbia, who refers to G2G advice as “fast policy”. Since the late 1990s Mexico’s pioneering policy to make cash benefits for poor families conditional on things like getting children vaccinated and sending them to school has been copied by almost 50 other countries….Budget cuts can provide another impetus for G2G trade. The Dutch army recently sold its Leopard II tanks and now sends tank crews to train with German forces. That way it will be able to reform its tank squadrons quickly if they are needed. Britain, with a ten-year gap between scrapping old aircraft-carriers and buying new ones, has sent pilots to train with the American marines on the F-35B, which will fly from both American and British carriers.
…
No one knows the size of the G2G market. Governments rarely publicise deals, not least because they fear looking weak. And there are formidable barriers to trade. The biggest is the “Westphalian” view of sovereignty, says Stephen Krasner of Stanford University: that states should run their own affairs without foreign interference. In 2004 Papua New Guinea’s parliament passed a RAMSI-like delegation agreement, but local elites opposed it and courts eventually declared it unconstitutional. Honduras attempted to create independent “charter cities”, a concept developed by Paul Romer of New York University (NYU), whose citizens would have had the right of appeal to the supreme court of Mauritius. But in 2012 this scheme, too, was deemed unconstitutional.
Critics fret about accountability and democratic legitimacy. The 2005 Paris Declaration on Aid Effectiveness, endorsed by governments and aid agencies, made much of the need for developing countries to design their own development strategies. And providers open themselves to reputational risk. British police, for instance, have trained Bahraini ones. A heavy-handed crackdown by local forces during the Arab spring reflected badly on their foreign teachers…
When San Francisco decided to install wireless control systems for its streetlights, it posted a “call for solutions” on Citymart, an online marketplace for municipal projects. In 2012 it found a Swiss firm, Paradox Engineering, which had built such systems for local cities. But though members often share ideas, says Sascha Haselmayer, Citymart’s founder, most still decide to implement their chosen policies themselves.
Weak government services are the main reason poor countries fail to catch up with rich ones, says Mr Romer. One response is for people in poorly run places to move to well governed ones. Better would be to bring efficient government services to them. In a recent paper with Brandon Fuller, also of NYU, Mr Romer argues that either response would bring more benefits than further lowering the barriers to trade in privately provided goods and services. Firms have long outsourced activities, even core ones, to others that do them better. It is time governments followed suit.”

"Natural Cities" Emerge from Social Media Location Data

Curated on Posted on by Stefaan Verhulst

Emerging Technology From the arXiv: “Nobody agrees on how to define a city. But the emergence of “natural cities” from social media data sets may change that, say computational geographers…
A city is a large, permanent human settlement. But try and define it more carefully and you’ll soon run into trouble. A settlement that qualifies as a city in Sweden may not qualify in China, for example. And the reasons why one settlement is classified as a town while another as a city can sometimes seem almost arbitrary.
City planners know this problem well.  They tend to define cities by administrative, legal or even historical boundaries that have little logic to them. Indeed, the same city can sometimes be defined in various different ways.
That causes all kinds of problems from counting the total population to working out who pays for the upkeep of the place.  Which definition do you use?
Now help may be at hand thanks to the work of Bin Jiang and Yufan Miao at the University of Gävle in Sweden. These guys have found a way to use people’s location recorded by social media to define the boundaries of so-called natural cities which have a close resemblance to real cities in the US.
Jiang and Miao began with a dataset from the Brightkite social network, which was active between 2008 and 2010. The site encouraged users to log in with their location details so that they could see other users nearby. So the dataset consists of almost 3 million locations in the US and the dates on which they were logged.
To start off, Jiang and Miao simply placed a dot on a map at the location of each login. They then connected these dots to their neighbours to form triangles that end up covering the entire mainland US.
Next, they calculated the size of each triangle on the map and plotted this size distribution, which turns out to follow a power law. So there are lots of tiny triangles but only a few  large ones.
Finally, the calculated the average size of the triangles and then coloured in all those that were smaller than average. The coloured areas are “natural cities”, say Jiang and Miao.
It’s easy to imagine that resulting map of triangles is of little value.  But to the evident surprise of ther esearchers, it produces a pretty good approximation of the cities in the US. “We know little about why the procedure works so well but the resulting patterns suggest that the natural cities effectively capture the evolution of real cities,” they say.
That’s handy because it suddenly gives city planners a way to study and compare cities on a level playing field. It allows them to see how cities evolve and change over time too. And it gives them a way to analyse how cities in different parts of the world differ.
Of course, Jiang and Miao will want to find out why this approach reveals city structures in this way. That’s still something of a puzzle but the answer itself may provide an important insight into the nature of cities (or at least into the nature of this dataset).
A few days ago, this blog wrote about how a new science of cities is emerging from the analysis of big data.  This is another example and expect to see more.
Ref:  http://arxiv.org/abs/1401.6756 : The Evolution of Natural Cities from the Perspective of Location-Based Social Media”

Shedding Light on Projects Through Contract Transparency

Curated on Posted on by Stefaan Verhulst

OpenAidMap: “In all countries, whether rich or poor, contracts are at the nexus of revenue generation, budget planning, resource management and the delivery of public goods. Open contracting refers to norms and practices for increased disclosure and participation in public contracting at all stages of the contracting process.
There are very good reasons for making procurement processes transparent. Public posting of tender notices and “requests for proposals” helps support free and fair competitive bidding – increasing citizen trust while also improving the likelihood of securing the best possible supplier. Once procurement is finished, public posting of contract awards gives important assurance for citizens, development partners, and competing companies that procurement processes are open and fair. Increasingly, open contracting in procurement transparency through portals like this one is becoming the norm for governments around the world. There is also a global initiative at work to establish a common standard for contracting data….
With so much momentum behind procurement transparency, there is an untapped opportunity to leverage data from public procurement processes to provide operational insight into activities. Procurement data can help answer two of the most important questions in project-level aid transparency: (1) Where are projects taking place? (2) How much money is being invested at each location?
Take an example from Nepal. Consulting the government’s aid management system yields some basic, but already useful, information about a particular transportation project. This type of information can be useful for anyone trying to assess patterns of transportation investment in the country or, for that matter, patterns of development partner financing….
Open contracting data have intrinsic value for transparency and accountability. However, they also have significant value for planners – even those who only care about getting greater insight into project activities. At the moment though, contracting data are too difficult to access. While contracting data are increasingly becoming available, they are often posted on stand-alone websites, in diverse data formats and without structured access. By standardizing around a core contracting data format and accessibility approach, we can unlock the potential to use contracting data at scale not only for transparency, but also as an effort-free addition to the arsenal of available data for project-level planning, coordination, and accountability. The utility could be even higher if combined with performance and results data.
When developing any public data standard, there are opportunities and risks. For open contracting data, there is a huge opportunity to make those data equally relevant for project planners as for those more purely interested in transparency and accountability. The pilot conducted by the Open Aid Partnership and AidData has explored this potential for overlap, yielding key insights that we hope can be used in the future development of an open and broadly relevant open contracting data standard.”

The Age of ‘Infopolitics’

Curated on Posted on by Stefaan Verhulst

Colin Koopman in the New York Times: “We are in the midst of a flood of alarming revelations about information sweeps conducted by government agencies and private corporations concerning the activities and habits of ordinary Americans. After the initial alarm that accompanies every leak and news report, many of us retreat to the status quo, quieting ourselves with the thought that these new surveillance strategies are not all that sinister, especially if, as we like to say, we have nothing to hide.
One reason for our complacency is that we lack the intellectual framework to grasp the new kinds of political injustices characteristic of today’s information society. Everyone understands what is wrong with a government’s depriving its citizens of freedom of assembly or liberty of conscience. Everyone (or most everyone) understands the injustice of government-sanctioned racial profiling or policies that produce economic inequality along color lines. But though nearly all of us have a vague sense that something is wrong with the new regimes of data surveillance, it is difficult for us to specify exactly what is happening and why it raises serious concern, let alone what we might do about it.
Our confusion is a sign that we need a new way of thinking about our informational milieu. What we need is a concept of infopolitics that would help us understand the increasingly dense ties between politics and information. Infopolitics encompasses not only traditional state surveillance and data surveillance, but also “data analytics” (the techniques that enable marketers at companies like Target to detect, for instance, if you are pregnant), digital rights movements (promoted by organizations like the Electronic Frontier Foundation), online-only crypto-currencies (like Bitcoin or Litecoin), algorithmic finance (like automated micro-trading) and digital property disputes (from peer-to-peer file sharing to property claims in the virtual world of Second Life). These are only the tip of an enormous iceberg that is drifting we know not where.
Surveying this iceberg is crucial because atop it sits a new kind of person: the informational person. Politically and culturally, we are increasingly defined through an array of information architectures: highly designed environments of data, like our social media profiles, into which we often have to squeeze ourselves. The same is true of identity documents like your passport and individualizing dossiers like your college transcripts. Such architectures capture, code, sort, fasten and analyze a dizzying number of details about us. Our minds are represented by psychological evaluations, education records, credit scores. Our bodies are characterized via medical dossiers, fitness and nutrition tracking regimens, airport security apparatuses. We have become what the privacy theorist Daniel Solove calls “digital persons.” As such we are subject to infopolitics (or what the philosopher Grégoire Chamayou calls “datapower,” the political theorist Davide Panagia “datapolitik” and the pioneering thinker Donna Haraway “informatics of domination”).
Today’s informational person is the culmination of developments stretching back to the late 19th century. It was in those decades that a number of early technologies of informational identity were first assembled. Fingerprinting was implemented in colonial India, then imported to Britain, then exported worldwide. Anthropometry — the measurement of persons to produce identifying records — was developed in France in order to identify recidivists. The registration of births, which has since become profoundly important for initiating identification claims, became standardized in many countries, with Massachusetts pioneering the way in the United States before a census initiative in 1900 led to national standardization. In the same era, bureaucrats visiting rural districts complained that they could not identify individuals whose names changed from context to context, which led to initiatives to universalize standard names. Once fingerprints, biometrics, birth certificates and standardized names were operational, it became possible to implement an international passport system, a social security number and all other manner of paperwork that tells us who someone is. When all that paper ultimately went digital, the reams of data about us became radically more assessable and subject to manipulation, which has made us even more informational.
We like to think of ourselves as somehow apart from all this information. We are real — the information is merely about us. But what is it that is real? What would be left of you if someone took away all your numbers, cards, accounts, dossiers and other informational prostheses? Information is not just about you — it also constitutes who you are….”