Software used to sift the news for government spook agences is getting so good it can predict revolutions.
According to Singularity Hub, software developed by Kalev Leetaru at the University of Illinois' Institute for Computing in the Humanities, Arts and Social Science was able to predict the Arab Spring to ten decibel places.
The software, which runs on a supercomputer, looked at news about Egypt, Libya and Tunisia. It detected compelling trends of negative tone in the decade leading up to the recent revolutions.
The software sifted through news reports from nearly every country and the story elements were woven together into a mind-boggling web of 100 trillion relationships.
Although the numbers were huge, all the 'sentiment mining' involved was counting the number of words in a document which were positive or negative.
Changes in the tone of a regions' news documents over time correlates with the sentiment of the people in that region. If there is a major dip in tone the natives are getting restless and any dictator should be shifting their fortunes and bases off-shore before their presidential palace is stormed.
But another technique, which is used alongside the negative word search, matches sentiment to geographic locations so it is possible to see where things are going to kick off.
It is not clear if censorship would undermine that. After all, if the evil regime is trotting out positive stories all the time it will be counted.
But the software also notices if stories are not being covered. For example, between 1 January and 24 January 2011, the global tone about Egypt dropped to an extent that had only been seen twice in the past 30 years. Those times were when US planes bombarded Iraqi troops in Kuwait in 1991 and the March 2003 US invasion of Iraq.
Leetaru said that it all works, despite claims that the uprising was a social media revolution. You did not need to monitor Facebook - it was all there in the mainstream press.
He said that the software was not powerful enough yet to predict progression or the timing of an event, but it can reveal periods of "increased potential for unrest."
This can help foreign ministers work out if they really want to support the despot or rally behind the people for a change of regime. Early on in Egyptian revolution, the US stood in support of Mubarak, which suggested very strongly that even the highest level analysis thought Mubarak was going to stay.
This is based on the logic that if you look at Egypt in the last 30 years Mubarak had been in power, so therefore he was going to stay in power. However the computer's tonal curve indicated that everything in the world was against him and getting worse so fast and so strongly that it was impossible for him to survive.
The same happened in Tunisia, Libya, and for the ethnic conflicts in the Balkans during the 1990s.