The intelligence community has seen the future, and the future is Google Trends. Actually, more like a highly sophisticated version of Google Trends, with Twitter and YouTube thrown in for good measure.

Iarpa, the blue-sky research arm of the intelligence community, recently announced a new program that aims to monitor, collect and analyze publicly available data to predict future events. The Open Source Indicator Program would be so sensitive to changes in the zeitgeist that it could “beat the news,” anticipating “political crises, disease outbreaks, economic instability, resource shortages and natural disasters” — to name a few.

The idea that web search trends, blogs, internet traffic and webcams could hint at the future is nothing new. Google Flu Trends was set up in 2009 to predict flu outbreaks, counting on the fact that hordes of people searching for “flu symptoms” might mean something. In some cases Google was faster than the government at identifying where the next outbreak would occur.

Last month, the Bank of England announced that it was using internet search data to understand economic trends. They found that searches for “estate agents” closely tracked house price inflation.

So it’s no surprise the intelligence community has over the last few years become more interested in open source intelligence, or information that’s freely available if you could just sort through the noise. Just recently, a West Point instructor teamed up with a Princeton researcher to investigate if Google search data could help analyze the situation in Egypt.

“Open source provides a critical lens, a critical opening into understanding the world around us in a much more dynamic way than traditional intelligence sources can provide,” Lieutenant Colonel Reid Sawyer, the head of West Point’s Combating Terrorism Center, told NPR.

Some of the latest investments in this more “open” direction include Visible Technologies, which scours the social web for posts and conversations on everything from Amazon to Flickr. Another Iarpa project, ALADDIN, would scan thousands of open source video clips and let you search for specific events of interest. Recorded Future taps thousands of websites, blogs and news alerts to somehow algorithmically calculate who knows whom, what’s getting attention and what will happen next.

Of course, gathering data and making educated predictions is one thing -– having those predictions become reality is quite another. Recorded Future’s prediction that Hosni Mubarak would flee to Saudi Arabia didn’t exactly pan out. There are many events -– 9/11, Fukushima, Osama bin Laden’s death -– that were unlikely to have popped up on a Google search, even one that underwent intense statistical analysis. As much as the internet can provide answers, without the right questions, it remains a flood of information. And there may be some questions we will never know to ask.

So the first step for Iarpa’s most recent program is to try to get a grip on all that information. Other current technologies have their specialties, whether it’s social media, search trends or YouTube videos. This program hopes to feed thousands of webcam streams, Twitter feeds, internet traffic numbers and search terms into a single pipeline, and have some semblance of sense come out the other side. Actually, more than just sense: a prediction about the future.

In other words, get every open source piece of information you can, and use it to predict every kind of outcome you can. The deadline for submitting an idea is July 27. But you probably predicted that already.

Photo: Lena Groeger