OSINT’s journey from academia to decision-makers

STEPHEN COULTHART
If history rhymes, as Mark Twain would say, I would say probably we’re large organizations are going to run behind on automation, except for some, like, highly exemplary organizations that have lots of resources that are highly innovative.

JEFF PHILLIPS
Welcome to Needlestack, the podcast for professional online research. I’m Jeff Phillips, and I’ll be your host today.

AUBREY BYRON
And I’m Aubrey Byron, a producer on needlestack.

JEFF PHILLIPS
Today, we’re joined by Dr. Steven Coulthart, associate professor at University of Albany and the director of the Open Source Intelligence Laboratory, which is opening there in April. Steven, welcome to the show.

STEPHEN COULTHART
Thank you, Jeff. Thank you, Aubrey.

JEFF PHILLIPS
Okay, so we’re talking about this lab. Steven, your launch sounds like it’s just a few months away. Why don’t you tell us a little bit about the lab and what you hope to accomplish there.

STEPHEN COULTHART
Sure. Yeah. Thank you. So the overall purpose of the lab is to provide an academic home for the study of open source intelligence. And when I say the study of open source intelligence, I mean studying about OSINT, as well as how to do OSINT. So in terms of the about OSINT, it’s, for example, looking at the integration of OSINT into organizations, into the government, but also, too, looking at the wider impact on society as well. And then, as I was saying, we really will get into studying the actual practice of OSINT as well, looking at tradecraft and so forth.

AUBREY BYRON
Yeah. We talk a lot on this podcast about the fact that information is not intelligence. Can you tell us a little bit more about what offerings you plan to have for students, and especially not just on the information gathering side, but also on the analysis side?

STEPHEN COULTHART
Yeah, so down the road, one of the things that we’ll be doing is working for clients in the government. So one of the great things about the college that the lab is in, which the name of the college is quite long, it’s the College of Emergency Preparedness Homeland Security and Cybersecurity, or CEAC. One great thing about CEAC is that we’re located in Albany, and so we have some good connections to the New York State Department of Homeland Security, or Dishes. So we see ourselves, like, finding clients in state and local government, as well as in the private sector as well, because I’m sure, as you talk about here on, the clients in the private sector have a great need for OSINT in terms of activities for the students. I also see in terms on the analytical side, also doing academic research papers. In fact, I have a couple of interns now that are even before the lab is getting started, they’re helping me put together an academic paper, looking at everything that’s been written in the professional literature and academic literature about OSINT as a field. So they’re already starting to work on.

JEFF PHILLIPS
It, and that’s a big field. So that’s a mix. That’s a mix of you have actual projects coming in from the government side or private that they can work on in addition to kind of more traditional I’m being taught. And the reason I ask that is because thanks to Twitter, there’s a lot of amateur sleuths out there that have kind of come into it’s basically self taught OSINT. So maybe a little bit. What do you think about as far as what is there to be gained by approaching OSINT from a more traditional education style? I don’t know if these people, these kids are going to get a degree or a certificate. What do you think about turning it into a traditional education?

STEPHEN COULTHART
Yeah, if I can just kind of step back, Jeff, and just kind of say more broadly about an academic perspective on OSINT. I think that one of the great values of the OSI Lab is going to be that we’re going to act as also an integrator. Because one of the things that I’ve noticed in my career as an academic who does applied scholarship is that sometimes when I talk to folks who are outside of academia, there might be something that they’re curious about. And they may not necessarily know that there might be some good research that’s already out there. Maybe in another field, maybe something they haven’t heard of yet. And I think of an example. I was at a conference in Ottawa. It was a five Eyes conference. And I remember one of the speakers, he was a fantastic speaker, highly knowledgeable. But one of the things really that he said blew me away, he said we need to have remember, it was scales or measurements for what counted as rigorous intelligence analysis. And I was sitting there in the audience and I’m thinking to myself, but that exists. People have done that research with intelligence analysts.

STEPHEN COULTHART
But academics on the research side were not typically good at getting the word out, right? So along the lines of kind of the OSI Lab being this academic home is also being an integrator of different knowledge sources for things that might be out there. Because one of the things I’m already kind of realizing is I’m starting to scope out this huge field through this project. I mentioned a moment ago that my students are working on is just how diverse the different groups are. And I think you probably see this on the needlecast, right? Like, you have people that are working on human trafficking issues. You have people that are doing online research about national security issues and they’re working in a variety of different places. And so I think bringing together all this knowledge to bear and best practices is something that I think academia is well placed to do.

JEFF PHILLIPS
Well. That makes a lot of sense. And people are starved, to be honest with you, for those kinds of best practices. Whether that’s there’s the tools side, which we hear a lot from needle stack listeners asking for tips and tricks. But just what are the best practices for doing this stuff because they’re out just actually doing OSINT, trying to find out about how to do OSINT. Right. They’re out there researching to figure out what are the best practices, which is if not two thirds of the reason why we even started the podcast. So that’s great.

AUBREY BYRON
Yeah, that’s a lot of why this podcast exists. And also just to have the perspective of a fraud analyst might help someone like a journalist look at OSINT from a different perspective. Part of the mission on the OSI lab is to research the impact of OSINT on society. What do you think that impact is and what sort of studies do you think you’ll conduct on that?

STEPHEN COULTHART
That’s a good question, Aubrey. I share the view that probably you all do, and we see in a lot of commentators, I generally believe that OSINT has democratized the intelligence function for organizations that it would have been outside of the reach 2030 years ago or wouldn’t have been as insightful. So I think that’s broadly true. And we see this not only in, I guess, like traditional organizations at state, local and federal level and private sector, but we also see it too, I think interestingly in some of the groups and some of the networks. That have formed of people that are doing something at least like open source research and open source intelligence to support operations to help Afghan translators get to Afghanistan or to provide assistance overseas, say, in Ukraine. I think there’s kind of some interesting lines there that are blurred, too, between open source intelligence and almost like covert action where there’s actual activity on the ground to help, say, a person out of Afghanistan that leverages open source to enable that type of operation. So I think there’s kind of interesting democratization going on, and a lot of people are talking about that.

STEPHEN COULTHART
So I think that’s definitely true. I do think there is a little bit at times of just like, anything that’s very new and very shiny, sometimes there can be a little bit of exaggeration as well. Being an academic who does applied work and not being strictly a practitioner, I’ve had a little bit of taste of this in one of my classes. I had the students do open source intelligence research and do geographic profiling, which is typically a technique used in law enforcement to estimate the probable places that a suspect would be or their patterns of behavior and life and so forth. And we wanted to do that for trying to find Abu Bakar al Baghdadi, so the leader of ISIS when he was actively being sought out by the US. Government. And so we did decently. We published an academic paper on it for at least the first half of the manhunt. We were looking over the course of about the first six months of I’m trying to remember when this would have been. It was just before the pandemic, but we were basically doing well in terms of identifying probable places where he could be using a variety of different open source information.

STEPHEN COULTHART
But we totally went off the trail in the last like few months that he was on the road and fleeing. And when that was, was when he specifically went to Syria and kind of backtracking and looking at what happened. The information on what was on where he was, was, I think in theory out there through some reporting and human sources on the ground. The New York Times had but these weren’t published reports. And the reason why I’m telling this story is that I think at times we kind of get caught up with how far OSINT can go. But in reality, there’s almost more. You got to be there. You have to go a little further. And there’s really just one more quick example of this. There is a completely unrelated to OSINT, I would say, but relevant example of Box, the news outlet. They did a story looking at these weird circles that people had noticed that were on the floor of the desert in a North African country and they were basically trying to so this reporter from Box was trying to figure out what are these circles there for? And he did everything.

STEPHEN COULTHART
He contacted experts, he combed the Internet. I mean, he did everything. And ultimately what he had to do was he had to send someone there. He had to send someone into the desert. That was the only way to figure out that those circles were in fact something having to do with petroleum exploration. But there’s kind of a certain point where we only go so far, and I think sometimes with OSINT, we might over promise on what it’s capable of because there’s just so much well founded excitement around it.

JEFF PHILLIPS
Well, that makes a lot of sense, right? That’s kind of the opposite of sometimes what we’re seeing. If I think of some of the amateur sleuths that especially with the Ukraine war and what’s going on with Twitter, where they’re spending time saying, well, that person is on the ground and supposedly they’ve posted this picture of this image. But is that true? Are they where they say they are? Or is this from a conflict years and years ago? But you’re talking about it in the reverse way, which is eventually if I can’t get a satellite that’s going to go within 5ft of the ground there to understand what’s going on, then that you have to get someone physically there for the we’ll call it the last mile of the intelligence. That makes sense that you don’t want to over promise right now you touched on this slightly, but a few years ago you wrote an article about the need for a better flow of information from research labs to national security decision makers. So do you see that? Will the OSI lab fit into and communicate with the larger intelligence community.

STEPHEN COULTHART
Yeah, absolutely. Jeff, I’m impressed that you took the time to find that academic paper. It fights back about what I was saying about the difficulty of academics getting our work out there. So thank you for reading that or finding it. So, yeah, I mean, definitely OSI Lab will be engaging with the community of practice, the IC specifically, of course, as well. A major finding when I wrote that paper was that certainly the content matters, but what really matters is knowledge translation. So how do we present ideas in a way that will be found and read and used by practitioner audiences? And so, yeah, absolutely. On the OSINT side, one of the things I’m really curious about is looking at this question of when it comes to open source intelligence research, setting aside academic research for a second, how often do customers use OSINT based products, say, over other types of products? And are there other aspects of different types of OSINT products that are more persuasive to different types of customers? And people have studied this in other fields looking, say, at whether or not research evidence is useful, say, to government decision makers. But it hasn’t been done in the OSINT field that I’m aware of to really understand what makes an OSINT product persuasive to a particular customer.

STEPHEN COULTHART
I think that on the OSINT side would be quite interesting.

JEFF PHILLIPS
Sorry to interrupt. I just want to clarify for us, because at first when you said OSINT products, I was thinking of tools or feeds of OSINT information. But you’re talking about the product, the output from your investigation. What product form does that take? Right, that’s what we’re referring to. And so that someone after this investigation is research, how do they best get access to it or digest it? That’s what you mean, correct?

STEPHEN COULTHART
That’s what I mean. That’s right. Okay. And I’m holding that separate then, to talk about research for a second. Right. So let’s say we do research, trying to understand, say, the implementation of OSINT and a federal agency or so or something. In terms of knowledge translation, it’s going to be key that we’re going to engage with folks like yourself. We’re going to go on popular media, be able to do opinion articles and so forth to get that information out there. Because a lot of times, like I was saying, a lot of academic knowledge is siloed and it could potentially be useful. So I think getting out there, disseminating information, that is going to be absolutely key for the work of the OSINT Lab as well. Just to more directly answer your question.

JEFF PHILLIPS
Well, yeah, but by the way, I took this through my head in a couple of ways. One, I totally get it from the lab’s perspective, the OSI labs, you’re going to be doing great work. How do people make sure people get access to it and don’t duplicate it at the same time? By the way, I also started going down my head thinking about your traditional in the professional world. You’re doing OSINT and you do an investigation research and how am I going to publish this out? What does that actual write up look like or report that I disseminate that? What are the best practices if there’s what you’re doing online? I’m taking screenshots, I’m finding videos, I’m doing finding phone numbers. I’m doing all kinds of stuff or physical locations. And what’s the best way to then present that up the stack in terms of threat intelligence reports?

AUBREY BYRON
What are some of the trends you’re seeing within OSA and what do you think the role of automation will be for practitioners in 2023 and going forward?

STEPHEN COULTHART
Yeah, this is the million dollar question and I think probably a lot of listeners are thinking about Chat GPT and its role. Yeah, I would be, I think, overly confident if I knew exactly how AI, for example, is going to be shaping online research. One thing that I would say, and this is kind of based on my own research, is that when it comes to technologies, just because they exist and anyone in the government knows this, just because they exist doesn’t mean that they’re actually going to be using government. Right. So just because particular automation tools might be available, that doesn’t necessarily mean that they’ll be used. And a few years ago I did a study of the US. Border Patrol. And originally I was very interested in this idea of how is the US border Patrol using data science concepts, technologies? This is a massive organization. So I did this research project and I remember one of the first people that I spoke to, they told me, they said that the Border Patrol doesn’t really have a concern for data science because we haven’t mastered Excel spreadsheets yet. And I thought to myself, my God, especially, and this is true, I think of people that are commenting at a high level, is that once we really get down to the implementation level, you can see that there are distinct challenges there.

STEPHEN COULTHART
So in the case of the Border Patrol, there was the issue of there was a counterculture within parts of the Border Patrol against that type of innovation. There was, on the other hand, technical issues. So just because the tool is available, you have to be able to make it play within the current It infrastructure. So you had that. You had gosh, just a variety of different problems. So I guess to answer your question, Aubrey, in a fairly succinct way, I guess I would say that it’s kind of unclear what direction we’re going in. But if history rhymes, as Mark Twain would say, I would say probably large organizations are going to run behind on automation, except for some highly exemplar organizations that have lots of resources that are highly innovative. Most, I think, are going to lag a little bit like the Border Patrol.

AUBREY BYRON
Yeah, that’s a good point.

JEFF PHILLIPS
I want to go a little bit back. I’m picturing this lab and students entering into this field. Some basic questions for you. When you guys get a project or you’re giving them a project, how do you go about telling them about what sources to use for collecting open source intelligence or what kind of process to use for their analysis on finding information and taking them from step to step as they go about the project? So is there certain models they’re following? Are there certain sources that you’re teaching them about that are the best starting points and ways to go about engaging in an OSINT investigation?

STEPHEN COULTHART
The way that typically I’ve taught OSINT is to kind of follow a general process that I had learned myself through trainings with Arnold Royser and Reuters Information Service. And that kind of approach has been the general approach that I’ve taken. But one of the things that I’ve gotten really interested in, Jeff, is and I’m kind of wondering too, if you’ve had any discussion about this on the program, is this idea of having some type of overall body of knowledge surrounding OSINT competencies and skills. The National Geospatial Intelligence Foundation, they have something called an essential body of knowledge or EBK. And EBK was developed, I believe, primarily by academic researchers. And the purpose was in a fast moving, somewhat technical field, technical field of Geoen. What makes someone a Geoen analyst or an imagery analyst? I think we have a strong need for that in the OSINT field. And I’m not familiar if anyone has kind of taken that approach to try to create a universal body of knowledge. But one of the things that I think that the OSI lab should do to your question is to start to, like, formally scope out what it means to be able to say that someone has the essential body of knowledge surrounding being an oath and analyst.

STEPHEN COULTHART
Are you familiar with any there are.

JEFF PHILLIPS
Thinking first of all, there’s two sides. This when I end up talking with people, there’s the the pure tools side and there are websites. There are people that just like to keep a body of the latest and greatest, and it could be hundreds of links long of these are great OSINT tools for doing the job. That’s one side of it. In terms of the actual process and how you go about it, I have seen not many, but there are, I think I believe it was through Sands. There might be some where there’s some level of courses that you can take to get trained in OSINT at a very base level. And then from there you would get into some professional services from individuals that might sell a platform that has video training, for example, or you get into vendors such as us, I mean, Authentic8, if you buy our product, has the Silo application on the back end. We have what’s called Silo training that’s once you’re a customer and trade craft skills to how to use the tools of the platform. But I don’t think there’s not a ton of that Owent certification. I’ve seen some, so I think that would be great.

AUBREY BYRON
But there has been kind of a call for that. We covered on we do an OSINT news roundup on our blog of just kind of relevant OSINT news. And there was a great paper in the Stanley Center about the need for kind of more rigor for the definition of OSINT. And he’s somewhat, maybe controversially for some of our audience, said that basically what amateur sleuths are doing on Twitter shouldn’t even be considered OSINT because it doesn’t have the same rigor necessarily as government analysts would do. And there was another paper about creating kind of an OSINT Buyers Club. But, yeah, I agree, there’s not a lot of consistency about everything from the definition to the process to anything, I think, because it’s still so newly being adapted in this space.

STEPHEN COULTHART
Yeah, one of the first things we did with the lab is working with my colleague Brian Nest Bomb, we created a definition of open source intelligence because we needed to have something. Because one of the things I’ve noticed in having discussions with stakeholders is that everyone’s kind of having this discussion about what OSINT is, but it’s not entirely clear that we’re all talking about the same thing. So the way that we define it, Aubrey, is actually, I think, kind of along the lines of the other guests that you were just speaking of that we wouldn’t consider open source intelligence, say, people doing online sleuthing simply for their own personal interest. For example, for us, there has to be some type of intelligence requirements being met, and that’s typically within an organization with a customer or a set of consumers that would use the intelligence product. But that being said, I also got done telling you earlier about how there’s these interesting kind of open source networks of people that are not really in a formal organization but grow organically, like the veterans groups that were helping Afghans out in Afghanistan. I think one of the things we have to have is a little bit of a tolerance here on ambiguity.

STEPHEN COULTHART
And certainly that’s the way that we wrote our definition white paper. We say in it like, this is one definition, this is what we say that it is, at least try to facilitate.

AUBREY BYRON
Yeah. Another good example is Elliot Higgins. Right? He didn’t have any sort of professional analyst background, and Bellingcat is now extremely respected. So, yeah, just to that point, that there is a gray area, for sure.

STEPHEN COULTHART
And if it’s okay, too, can I kind of give you the main components of how we define it?

AUBREY BYRON
Sure, yeah, that would be great.

STEPHEN COULTHART
We say that open source intelligence is legally obtained public or commercial information that’s been validated, analyzed, and disseminated to meet an intelligence requirement. So in there we have the components of publicly and commercially available information legally obtained, processed and analyzed to meet an intelligence requirement and we’re totally agnostic on the field. I mean, it could be in cybersecurity, it can be for counterterrorism, it could be at any level of government. But basically, as long as it has those components, that’s what we think of as OSINT.

AUBREY BYRON
Yeah, that’s a great definition.

STEPHEN COULTHART
I think, too, one kind of danger of leaving things fairly open and ambiguous is that it kind of opens up the possibility of I think in some cases, if we don’t kind of have some type of definition around what it is, I think it can provide an aura for folks that perhaps maybe don’t have the best of intentions. So in our white paper, we referenced that there was a recent government case where an FBI agent basically admitted in court that open source is just federal speak for a Google search. So if we don’t kind of like go in here and kind of stake as a community instead of communities that we don’t come in and kind of stake our own kind of ground here, I think we seed it for other people to use the terminology uncontested.

AUBREY BYRON
Really quick, before we wrap up, is there any advice you want to leave with the audience or final thoughts?

STEPHEN COULTHART
I think, like, if I can kind of speak to the folks that maybe are newer to the field that may or may not be students but are more new, what I would say is think less about specific tools and more about general process, which is something that we were touching on earlier. Certainly there are great tools like the one that sponsors this podcast, but I would say too that just to kind of be aware of those tools, use those tools, and also the fundamentals meat and potatoes of being good researcher are going to translate across tools. The tools will enhance the work, but the tools by themselves are not the work. The other thing you know, I’d say, too, is that it’s? And I’ve been speaking from personal experience with students and then experience myself is that when it comes down to if you’re curious about something, one fantastic thing about living in the year 2023 or living in the time you live now is that if you have a question about something, you can just contact people. And I have been amazed if I’ve had a question about OSINT tradecraft, about OSINT, about something of substance matter.

STEPHEN COULTHART
If you just reach out to people and you’re clear about what you want and you’re respectful, I think most of the time people get back to you. And I think sometimes if you’re more junior, you kind of have this feeling of reticence. You’re like, I’m not this important person. This person is never going to respond to me. But you never know unless you try. And ultimately, not only can you learn something useful but you might forge a relationship there. And then I would kind of say to the people that maybe are a little bit more established in the field to consider the role of academia, to consider that academia can potentially provide this wider perspective that otherwise we might not have so that there is something of value there.

AUBREY BYRON
That’s great.

JEFF PHILLIPS
Well, I’d like to thank our guest, Dr. Stephen Coulthart, for joining us today. If you liked what you heard, you can view transcripts and other episode info on our website, Authentic8 with an eight .com/needlestack. That’s Authentic with the number eight .com/needlestack. And be sure to let us know what you thought of the show on Twitter. That’s @needlestackpod and to like and subscribe wherever you’re listening today. We’ll be back next week with more OSINT research tips. We’ll see you then.