Sentiment analysis

 

Source:

The web and social-networking platforms have resulted in an explosion of words. Many firms apply artificial intelligence technology to get the gist, and use that as a trading signal. One study in 2010 by researchers at Indiana University analyzed millions of tweets to predict the movement of the stock market three days later with an 87% accuracy. Such success has unleashed a new fashion for Wall Street quants to plug so-called "sentiment analysis" of social media into their massive models. 

Until now these indicators have been fairly blunt, usually tracking a handful of companies on a two-dimensional scale of positive or negative sentiment. But on June 25th Thomson Reuters unleashed no fewer than 18,864 new indices, updated each minute. The system, developed by MarketPsych, a start-up in California, can analyze as many as 55,000 news sites and 4.5m social media sites, blogs and tweets (though on an everyday basis, the number it crunches will be much smaller). The indices quantify emotional states like optimism, gloom, joy, fear, anger—even things like innovation, litigation and conflict. And it does it across a slew of assets: 40 equity sectors, 29 currencies, 22 types of energy and materials, 12 agricultural commodities and 119 countries.

The world's center of gravity

Source:

IT IS not exactly news that the world's economic centre of gravity is shifting east. But it is striking how fast this seems to be happening. In a new study on the economic impact of urbanisation the McKinsey Global Institute, the research arm of the eponymous consultancy, has attempted to calculate how this centre of gravity has moved since AD 1 and how it is likely to move until 2025. Although the underlying maths (which involves weighting the approximate centre of landmass of a country by its GDP) has to be taken with a pinch of salt, the calculations show that the centre is rapidly shifting east—at a speed of 140 kilometres a year and thus faster than ever before in human history, according to Richard Dobbs, one of the authors of the study. The main reason for this is rapid urbanisation in developing countries, in particular China.

PAGASA data fees

Today I inquired at the PAGASA for climate data, and I was told that the fee is currently at P36/month/parameter/weather station. That's P432/year/parameter/weather station. Data is available for most climate- and weather-related parameters you can think of, and data can go back as far as the 1950s. To obtain data, one has to write a request letter addressed to Edna L. Juanillo, OIC of the Climatology and Agrometeorology Division.

Graph of the day: World GDP

It was just a matter of time

Source:

ASIANS recently passed Hispanics as the largest group of new immigrants to the United States, according to a survey today predicting a demographic trend bringing powerful economic, social and political changes. 

The survey by the Pew Research Center found the ranking switch of two largest groups of newcomers to the United States started in 2009. 

And it determined that the growing Asian population in the United States not only is large, but it is thriving. 

"Asian Americans are the highest-income, best-educated and fastest-growing racial group in the United States," the study concluded. 

The Pew centre said the change in immigration figures is largely due to declining immigration from Mexico – the source of more US immigrants than any other country – amid weakness in the US job market and a crackdown on illegal immigration.... 

Overall, Hispanics still far outnumber Asians in America. In fact, there are already more Hispanics than the 41 million Asians Pew predicts will live in the United States by 2050, if the current trend continues. 

Currently there are 18.2 million Asians in the United States or 5.8 per cent of population, up from one per cent in 1965, mostly from China, the Philippines, India, Vietnam, South Korea and Japan. 

This compares to 52 million Hispanics or 16.7 per cent of the population.

China's coming back

Nations' share of world GDP from 1-2008 AD.

Addendum: I knew right off the bat that there was something wrong about the horizontal axis. Luckily The Economist has improved on this with the ff., much more correct graph:

Another illustration of the world's economic history from last year, also by The Economist:

Political centralization is good (and bad)

So near yet so different.

From Why Nations Fail:

One of the most dramatic borders in Africa is at the north of Lake Kivu when you cross from Gisenyi in Rwanda into Goma in the Democratic Republic of the Congo....

Why is Gisenyi so different from Goma? Both were part of Belgian colonies. If anything you might have expected Rwanda to be much poorer and less functional. After all they don’t have the mineral wealth and Congo does. And wasn’t there a huge genocide in 1994 in which maybe 800,000 people perished? Why do things seem to work so much better in Rwanda? 

The explanation for these differences lies not in the last 50 years when both countries were independent. Neither does it lie in the colonial period. The roots are much deeper. The key fact is that around 1700 a powerful centralized state appeared in Rwanda while such a state never formed to the west of Lake Kivu in the Eastern Congo.

The state, the seminal study of which is Jan Vansina’s Antecedents to Modern Rwanda, mythically emerged on Gasabo Hill northeast of the modern day capital of Kigali. By the 19th century it had spread to most of modern Rwanda, making Rwanda one of the few modern African countries whose borders correspond closely to a pre-colonial polity. The Rwandan state was highly militarized and run by a king and a cattle owning elite which became associated with the so-called Tutsis. Historically the king had constantly moved their capital but in the 1890s he settled at Nyanza where parts of the traditional palace have been restored. You can also visit the king’s heard of cattle (if you want to see how the court was in the late 1940s it, and much of the aristocracy, they featured in the Hollywood classic King Solomon’s Mines, or the 1985 remake).

The historical Rwandan state was not a “developmental state”. It was highly militarized and in the 1870s succeeded in turning most of the rural population of farmers into serfs who had to pay heavy dues and do free labor services for their chiefs for half of the week. It was this act which helped to institutionalize the differences between Tutsis and Hutus, the latter bearing the brunt of this new set of economic institutions. But developmental or not the state brought order and rules and heavily influenced the behavior of people in Rwanda. Most striking is the impact this had on the genocide in 1994. As pointed out in every treatment (e.g., the Human Rights Watch’s study led by Alison Des Forges Leave None to Tell the Story), this was planned and executed from above. Order came down to kill....

As we have argued in Why Nations Fail, political centralization is key not only for the development of inclusive institutions but for even the most basic form of growth under extractive institutions, and the orderliness and the economic revival in Rwanda since 1994 are both due to this type of growth under extractive institutions. 

But Rwanda also illustrates that political centralization can also have perverse consequences, particularly when the state turns genocidal. And that of course is not unique to Africa. Just think of the Nazi state.

Addendum: Income inequality as seen from Google Earth satellite imagery.

Winners and losers from free trade

From the IGM Forum:

Question A: "Trade with China makes most Americans better off because, among other advantages, they can buy goods that are made or assembled more cheaply in China."

• Strongly agree: 45%
• Agree: 40%
• Uncertain: 0%
• Disagree: 0%
• Strongly disagree: 0%
• No opinion: 0%

Question B: "Some Americans who work in the production of competing goods, such as clothing and furniture, are made worse off by trade with China."

• Strongly agree: 33%
• Agree: 50%
• Uncertain: 3%
• Disagree: 0%
• Strongly disagree: 0%
• No opinion: 0%

Regulation and drug shortages

From the Marginal Revolution blog:

The FDA began to ramp up GMP rules and regulations under the new commissioner in 2010 and 2011. In fact, the report indicates that FDA threats shut down some 30% of the manufacturing capacity at the big producers of generic injectables. The safety of these lines was not a large problem and could have been handled with a targeted approach but instead the FDA launched a sweep against all the major manufacturers at the same time. These problem have been exacerbated by a change in Medicare reimbursement rules and by the rise of GPOs (buying groups) which reduced the prices of generics. Thus, in response to the cut in capacity, firms have shifted production from less profitable generics to more profitable branded drugs, so we get shortages of generics rather than of branded drugs. 

Add to these major factors a few unique events such as the FDA now requiring pre-1938 and pre-62 drugs to go through expensive clinical trials, the slowdown of ANDAs and crazy stuff such as DEA control over pharmaceutical manufacturing and you get very extensive shortages.

Skip the burgers and sodas!

Source: "If the whole world had the same proportion of overweight and obese people as America, its biomass would rise by 20%."

Market design in stock exchanges

Source:

"The Securities and Exchange Commission, which oversees the capital markets, has proposed a "consolidated audit trail" requiring exchanges to report every trade to a central repository, where they could later be analyzed. 

"The project is a "very high priority" for the SEC, says an official, but the agency doesn't know when the rules for it will be completed. The main obstacle: agreeing on how to standardize the various formats that brokers and exchanges use to gather trading data. 

"It will cost money to improve and modernize market structure," says Bryan Harkins, chief operating officer at Direct Edge, the fourth-largest stock exchange. "But the short-term money pales in comparison to boosting investor confidence in the long term." 

"The SEC has estimated that a centralized order-tracking system would cost approximately $4 billion to set up and $2.1 billion a year to maintain. 

"Mr. Leinweber of Berkeley has a simpler, and probably cheaper, solution in mind. He proposes that supercomputers—like those at national laboratories such as Berkeley's—should track every trade in real time. If volume began surging dangerously, the system would flash a "yellow light." Regulators or stock exchanges could then slow trading down, giving the market time to clear and potentially averting a crisis."

Elinor Ostrom passes away

Elinor Ostrom

1933-2012

The Indiana University's statement:

The Royal Swedish Academy of Sciences awarded the 2009 Nobel Prize in Economic Sciences to Ostrom “for her analysis of economic governance, especially the commons.” Through a multidisciplinary approach that combined theory, field studies and laboratory experiments, she showed that ordinary people are capable of creating rules and institutions that allow for the sustainable and equitable management of shared resources. Her work countered the conventional wisdom that only private ownership or top-down regulation could prevent a “tragedy of the commons,” in which users would inevitably destroy the resources that they held in common.

Graph of the day: FDIs

Sir James Mirrlees on mathematical models

Featured paper of the day: Banks and securities markets

"The evolving importance of banks and securities markets" (Demirguc-Kunt, Feyen, Levine)

This paper examines the evolving importance of banks and securities markets during the process of economic development. We find that as countries develop economically, (1) the size of both banks and securities markets increases relative to the size of the economy, (2) the association between an increase in economic output and an increase in bank development becomes smaller, and (3) the association between an increase in economic output and an increase in securities market development becomes larger. The results are consistent with theories predicting that as economies develop, the services provided by securities markets become more important for economic activity, while those provided by banks become less important.

Nice paper using quantile regression techniques. (I wish I could have written this for work!) At least now I have a greater appreciation of the applications of quantile regression in finance.

US leads carbon dioxide reductions

Source:

"US emissions have now fallen by 430 Mt (7.7%) since 2006,the largest reduction of all countries or regions. This development has arisen from lower oil use in the transport sector … and a substantial shift from coal to gas in the power sector." 

How big is a cut of 430 million tons of CO2? It's equal to eliminating the combined emissions of ten western states: Alaska, Washington, Oregon, Idaho, Montana, North Dakota, South Dakota, Wyoming, Utah and Nevada. 

It seems the planet's biggest all-time CO2 polluter is finally reducing its emissions. Not only that, but as the chart above shows, US CO2 emissions are falling even faster than what President Obama pledged in the global Copenhagen Accord. 

Here is the biggest shocker of all: the average American's CO2 emissions are down to levels not seen since 1964 -- over half a century ago."

With great data come great responsibilities

A thoughtful piece on the powers and limits of data. While aimed more toward "data journalists", economists and econometricians may do well to take heed of the following advice:

Data is not a force unto itself. Data clearly does not literally create value or change in the world by itself. We talk of data changing the world metonymically – in more or less the same way that we talk of the print press changing the world. Databases do not knock on doors, make phonecalls, push for institutional reform, create new services for citizens, or educate the masses about the inner workings of the labyrinthine bureaucracies that surround us. The value that data can potentially deliver to society is to be realised by human beings who use data to do useful things. The value of these things is the result of the ingenuity, competence and (perhaps above all) hard work of human beings, not something that follows automatically from the mere presence and availability of datasets over the web in a form which permits their reuse. 

Data is not a perfect reflection of the world. Public datasets (unsurprisingly) do not give us perfect information about the world. They are representations of the world gathered, generated, selected, arranged, filtered, collated, analysed and corrected for particular purposes – purposes as diverse as public sector accounting, traffic control, weather prediction, urban planning, and policy evaluation. Data is often incomplete, imperfect, inaccurate or outdated. It is more like a shadow cast on the wall, generated by fallible human beings, refracted through layers of bureaucracy and official process. Despite this partiality and imperfection, data generated by public bodies can be the best source of information we have on a given topic and can be augmented with other data sources, documents and external expertise. Rather than taking them at face value or as gospel, datasets may often serve as an indicative springboard, a starting point or a supplementary source for understanding a topic. 

Data does not speak for itself. Sometimes items in a database will stand by themselves, and do not require additional context or documentation to help us interpret them – for example, when we consult transport timetables to find out when the next train leaves. But often data will require further research and analysis in order to make sense of it. In many ways official datasets resemble official texts: we need to learn how to read and interpret them critically, to read between the lines, to notice what is absent or omitted, to understand the gravity and implications of different figures, and so on. We should not imagine that anyone can easily understand any dataset, any more than we would think that anyone can easily read any policy document or academic article. 

Data is not power. Data may enable more people to scrutinise official activities and transactions through more detailed, data-driven reportage. In principle it might help more people participate in the formulation of more evidence based policy proposals. But the democratisation of information is different from the democratisation of power. Knowing that something is wrong or that there is a better way of doing things is not the same thing as being in a position to fix things or to affect change. For better or for worse flawless arguments and impeccable evidence are usually not sufficient in themselves to affect reform. If you want to change laws, policies or practices it usually helps to have things like implacable advocacy, influential or high profile supporters, positive press attention, hours of hard graft, bucketloads of cash and so on. Being able to see what happens in the corridors of power through public datasets does not mean you can waltz down them and move the furniture around. Open information about government is not the same as open government, participatory government or good government. 

Interpreting data is not easy. Furthermore there is a tendency to think that the widespread availability of data and data tools represent a democratisation of the analysis and interpretation of data. With the right tools and techniques, anyone can understand the contents of a dataset, right? Here it is important to distinguish between different orders of activity: while it is easier than ever before to do things with data on computers and on the web (scrape it, visualise it, publish it), this does not necessarily entail that it is easier to know what a given dataset means. Revolutionary content management systems that enable us to search and browse legal documents don't mean that it is easier for us to interpret the law. In this sense it isn't any easier to be a good data journalist than it is to be a good journalist, a good analyst, a good interpreter. Creating a good piece of data journalism or a good data-driven app is often more like an art than a science. Like photography, it involves selection, filtering, framing, composition and emphasis. It involves making sources sing and pursuing truth – and truth often doesn't come easily. Amid all of the services and widgets, libraries and plugins, talks and tutorials, there is no sure-fire technique to doing it well.

Policy musing: Mandatory updates from health professionals

One problem plaguing the Philippine health care system is that there is no reliable, accurate way of keeping track of nurses, physicians, and other health professionals in and out of the country. Once they pass the board exams of the Philippine Regulatory Commission (PRC), there's no mechanism to know whether they're practicing their profession, whether they've changed jobs, whether they're unemployed, whether they're still inside the country, or whether they're still alive!

I guess this problem also besets other professions (e.g., law) but this important implications in the development of our health system (and human capital development, in general), especially in knowing which regions are in most need for health manpower and how many are available to satisfy that need. Also, without an updating mechanism, it will be difficult to measure how many health professionals are unemployed and underemployed, which has far-reaching implications in understanding the so-called "brain drain" phenomenon and in the allocation of human capital resources (most notably in "oversubscribed" health courses such as nursing).

As such, it may do well for government (mainly the PRC) to require the mandatory submission of each and every registered health professional of his or her bio-data each year if possible (or once every 3 years at least, in consonance with the schedule most surveys made by the government). In addition, it could be made a requisite for the renewal of licenses. Of course, this should be an online process for easy and efficient data processing, and so that everyone (esp. those in the provinces or abroad) can participate.

The above-suggested information updating system is likely to be cost-effective and will surely result in a more informed assessment of the status of the Philippine health human resource sector.

Links worth reading

1. Alvin Roth on the future of repugnance (essay).

2. Dave Giles griping about linear probability models:

It's not the bias that worries me in this particular context - it's the inconsistency. After all, the MLE's for the Logit and Probit models are also biased in finite samples - but they're consistent. Given the sample sizes that we usually work with when modelling binary data, it's consistency and asymptotic efficiency that are of primary importance.

Economist of the day: Roland Fryer Jr.

Roland Fryer Jr. (of Freakonomics fame) on his MacArthur "Genius" Fellowship ($500,000 grant).

Deaths from non-communicable diseases

From the Daily Chart:

YOU are more likely to be killed by a non-communicable disease (NCD), like cancer or heart disease, than anything else. In 2008 they accounted for 63% of the 56m deaths worldwide. Growing populations and increased longevity are making the problem worse. By 2030, 22% of people in the OECD club of rich countries will be 65 or older, nearly double the share in 1990. China will catch up just six years later. About half of American adults already have a chronic condition, such as diabetes or high blood pressure, and as the world becomes richer the diseases of the rich spread farther. In the slums of Calcutta, although infectious diseases claim the young; for middle-aged adults, heart disease and cancer are the most common killers.