From OSINT Skeptic to Open-Source True Believer

I’ve recently become a believer in open-source intelligence (OSINT) but this hasn’t always been the case. I spent 34 years in the U.S. intelligence community as an analyst, during which time OSINT was largely an afterthought.  First and foremost, our job was to provide warfighters and policymakers insight and assessments gleaned from the U.S. intelligence community’s expensive and expansive classified collection systems.       

My colleagues and I always recognized the importance of information from academia, think tanks, and select media. But OSINT, which delivers insights based on the publicly available information (PAI) and commercially available information (CAI) sources, seemed at the time like background noise, not actionable intelligence.

The tipping point: Online data expands exponentially     

The information space changed radically in the last two decades; the world is now awash in data. In 2020, for example, about 2.5 quintillion bytes of data were created every day, according to Techjury. This year, 70% of the globe’s GDP will have undergone digitization. The amount of data in the world was estimated to be 44 zettabytes at the beginning of 2020, according to the World Economic Forum, and by 2025, it is estimated there will be 175 zettabytes of data in the global datasphere. These numbers are so huge they’re hard to comprehend; a zettabyte is 8,000 bytes to the seventh power — that’s an “8” followed by 21 zeros.

This information space includes traditional and social media, blogs, videos, commercial imagery and much more, all uploaded to the web. Just by way of one example, as of July 2021, 350,000 tweets were posted every minute or 500 million per day, according to The Fact Site. Senior officials — including heads of state — in nearly every country are routinely tweeting, providing insights into the world views, threat perceptions and policies of allies and adversaries alike. Clearly, this explosion of real-time information is relevant to intelligence analysis, and just as clearly, OSINT is now much more than reading and translating newspapers and academic articles.  

Tools combined with subject matter experts are key

What has made OSINT so valuable for the intelligence professional is the ability to access these massive data streams via sophisticated IT platforms that can ingest, organize, and derive insight from the information. And let’s not forget the importance of expertise; subject matter experts are essential to understanding the data and providing assessments and actionable intelligence to policymakers and warfighters at the tactical to strategic level. 

The world of big data and OSINT is dynamic and rapidly evolving. New social media sites pop up routinely, necessitating a strong back-end of data scientists and technical experts to ensure access to new data streams. In many ways, these specialists are analogous to the intelligence community’s collectors. It is their job to ensure the tools can ingest the data needed to meet mission requirements.

I experienced the effectiveness and relevance of OSINT first-hand after I retired from the U.S. government. Beginning in 2018, I led the analytic effort for an open-source center where our team of subject matter experts and data scientists employed sophisticated OSINT tools to access PAI; we provided high-quality intelligence analysis daily to our customers on a wide range of operational and strategic topics. Over nearly three years at this center, OSINT helped us routinely provide indications and warnings, situational awareness, world-class intelligence analysis and decision advantage to our clients.  

OSINT – a key tool for staying ahead of threats

OSINT will not replace classified intelligence, which continues to provide exquisite data that only our classified intelligence collection platforms can provide. That said, the ability to tap into big data and derive understanding and insight is groundbreaking. 

Open-source intelligence has emerged as a true force multiplier — and an essential element for intelligence gathering and analysis. And the value of OSINT will only expand as data available for exploitation increases and new capabilities in artificial intelligence and machine learning come online to bolster our ability to make sense of this data. It’s true; I was skeptical of just how much real information was hiding in plain sight. Now that I’ve lived OSINT analysis — and seen for myself how it can augment traditional intelligence methods — I've become a believer.