AI-enhanced technology for frictionless borders

Declan Trezise
So, yeah, with the liquid bomb plot, I think there were multiple planes and of the order of about 2000 people that were at risk who were who were saved by that plot being foiled. But an example where there was a failure in a border system that could have been solved by AI is the Boston Marathon Bomb.

Jeff Phillips
Welcome to Needlestack, the podcast for professional online research. I’m Jeff Phillips, your host.

Aubrey Byron
And I’m Aubrey Byron, producer and co-host. Today we’re continuing our discussion around AI and OSINT with a discussion about the role of AI in border security.

Jeff Phillips
This should be a great topic. And joining us for the discussion is Declan Trezise, vice president of Global Solutions Engineering at Babel Street. Declan, welcome to the show.

Declan Trezise
Thanks for having me, Jeff. Thanks, Aubrey. Appreciate you taking the time with me today.

Jeff Phillips
Yeah, it’s been a really interesting last couple of episodes. Obviously, artificial intelligence is a hot topic these days. But before we dive in to start us off and for those that don’t know, can you tell us a little bit about Babel Street and your role with the company?

Declan Trezise
Okay, yeah, absolutely. So Babel Street is a company with a great heritage working with government organizations, agencies, intelligence, law enforcement, and also large commercial enterprise customers. We deliver technology that allows investigators and people involved in solving and helping difficult technological problems in the security domain, in the intelligence domain. And it revolves around OSINT that we’re going to talk about a little bit later, but also around providing very competent AI components that can deliver kind of point solutions and help solve very difficult problems. Things that we’re going to talk about in border control. But yeah, understanding human language, analyzing language and text, and providing insights for analysts.

Jeff Phillips
Well, you talked a little bit about you mentioned border control.

Declan Trezise
Can you tell us a little bit.

Jeff Phillips
About how the tool is utilized by countries in terms of border control?

Declan Trezise
Yeah, absolutely. So we actually work with some large, well developed Western countries in helping them secure the border. And we do that by providing a capability to screen and check names of people as they enter and exit the country. Now this sounds like it might be a relatively straightforward problem. You think? Okay, a name is a name, but I think I’ll probably go into a bit more detail about how a name isn’t just a name and you want to combine name with other biographic information in order to check whether it’s on a watch list or a no fly list or it could even be a terrorist watch list. So, yeah, we solve that problem currently.

Aubrey Byron
Yeah. Can you, backing up a minute, just give us a little bit of an overview of modern border management and what the challenges are.

Declan Trezise
Oh, yeah, absolutely. So I first got involved with thinking about borders back in 2015. I’ve been working with a number of people across the UK home office and actually they are a customer of ours when it comes to the capabilities that I talked about screening names at the border. And over that time I think I’ve spoken to several experts in the field, work very closely with them and my information is secondhand. But I’ve been around these operations and have been delivering solutions to help. Really? What is a border? Okay, you look at a border and it’s very much about what’s it for. Well, it’s there to facilitate the free travel of innocent people that are going about their business, trying to improve the economic prosperity of their country, maybe through trade and through cargo, but also travelers for pleasure, travelers for business. Those are really 99.9% of all travel is completely innocent and well intended. Now your border has to facilitate that as swiftly, smoothly, as efficiently as possible. But also it’s there as a security measure. It’s there to prevent the bad actors from passing over, vent the bad actors from being in a place where they can commit crimes and atrocities that we don’t want.

Declan Trezise
But also there are bad actors that are bringing illicit cargo, smuggling people, amongst other things, that are not good. So your border must be able to reject those people and identify them efficiently too. So that’s what a border is for. Now, when it comes to kind of an integrated approach to border management, you need to look at it from a few different angles. So really if you can prevent somebody crossing your border before they get there, that is a great help. And if there’s information sources that allow you to get that information in advance, you can prevent the traveler leaving their destination, then your border is working effectively. So pushing the border out is a key feature of integrated border management using intelligence before somebody even travels. Now part of the integration though is getting data shared and visible and usable in time across many different agencies and producers and consumers of data. So an integrated border wouldn’t just be a single isolated agency or organization. They’d have to work in concert with other borders and they’d have to work with their local defense, security and intelligence agencies to make sure they have up to information and that then has to get to the border guard so they can make effective decisions.

Declan Trezise
So it’s the integration of people, organizations, agencies and then it comes down to the problem of identity. So when you have people at the border correctly identifying who they are and comparing that against the sources of data that you have. So yeah, several aspects of integrated border management need to be in place to be effective.

Jeff Phillips
That’s super interesting. For the last several episodes of the podcast we’ve been talking about artificial intelligence and this podcast is about OSINT.

Jeff Phillips
Now we’re talking about border security. And I can picture in my mind a little bit of how that might all connect. But can you tell us about how OSINT or publicly available information and the link to AI is all used in border security?

Declan Trezise
Yeah, sure, absolutely. So I’ve been around OSINT since before it was known as OSINT. So in my early years as a technologist, I worked on projects for collecting information and for making that usable and actionable. OSINT, as we understand it, is open source intelligence. It’s intelligence from sources that are available either publicly or commercially. And that could be anything. It could be websites, it could be social media, or it could be information that you can buy from an organization that collates and organizes it. But fundamentally, it’s data that’s out there on the Internet. Now, without some degree of artificial intelligence, you’re not really going to be able to find something that’s key and valuable for your border to mitigate the risks in a timely way. You could have hundreds of analysts on every single border trying to manually scour the Internet, and they’re not really necessarily going to find the risks that we’re worried about. So you need to have a set of analytics tools that can find the correct data, but then also refine it down to the bits that are of interest. So identifying the people and the associated risk that goes alongside them.

Declan Trezise
So OSINT would be a great source of huge amount of information, but without the analytics tools that are often AI fueled that sit on top, you’re not going to be able to see the risks that are there. And this could be things that it could be analytics, AI that runs across video content, or something that’s more close to my heart associated with things I’ve been working on for nearly ten years now is looking at text data. So we publish and write so much text on the Internet, whether it’s social media postings on Twitter, or whether it’s blogs, whether it’s instant messages between people, you cannot have a human read and understand all that. So what we want to do is get to the point where we have AI, by which I mean language models. It could be small models, there could be the emergence recently of things like chat CPT, it could be large language models, things that can read and understand text across multiple languages and pull out things of interest or even answer questions against that data in a summarized way. So that’s where we look. We look at the convergence of OSINT and text analytics and natural language processing to do smart things for border security.

Jeff Phillips
And as I’ve gone across borders and do that super fast, timely, as they take my name off my passport, being able to do that quickly as you.

Declan Trezise
Go through okay, so your name will come off the passport, Jeff. And they also need to know a little bit more about you because there’s probably several Jeff Phillips out there, and there’s probably several ways of spelling Jeff, right? You spell it with a J but somebody might spell it with a G. So it’s first of all making sure you’ve got the right person and then making sure that the information that you’re screening against is up to date and shows the right risks.

Aubrey Byron
Yeah. Can you tell us a little bit more about that name matching technology and how it works?

Declan Trezise
Yeah, it’s something that I talk about often, so yeah, stop me if I go too long. It all starts with the idea that we imagine that matching a name is a simple thing to do. If the name is on the list, then it’s a hit. But we as humans and working with technology, introduce all kinds of complexity into names all the time and mistakes can be made. Or it could even be that a name is perfectly correct in the way that it’s written, but it’s just not the way that you would write your name. And complexities are added when we take a name and we try and stuff it into a format that a machine can understand, so we’ll split it and break it. We will miss parts out. You might lose your middle names and they become an initial, or your last name might get put first in front of your first name. So by working with machines, we introduce complexity and then a big layer on top is all about language. So we’re speaking the same language for the most part. We might have the Atlantic Ocean between us, but for the most part, when I say something, you’ll understand how to write that word down, you’ll understand what that word means.

Declan Trezise
But when you move between two languages, there’s a layer of fuzziness that’s added automatically because no two languages map directly. And then there can be various interpretations as to how you would write somebody’s name. And if you’re coming from something like Korean, Chinese, Japanese or Arabic, for example, their alphabets have nothing in common with the Latin alphabet that we use in Europe. So there are adaptations and standards that some people stick to, but other people use different ones. So names can become more complex as you move from language to language. And any system that’s going to understand checking names, screening names, mapping them, is going to have to deal with all the complexities that we introduce just from communicating names between us and then the ways that they can be complicated and the phenomena of different name variations that get introduced in those processes.

Aubrey Byron
That’s fascinating. Another thing that came up was fuzzy name matching. Can you go into a little bit about that?

Declan Trezise
Yeah, fuzzy is a slightly worrying term right now. We try and add on the word smart at the same time when we talk about it. So you want it to be smart and fuzzy, not just fuzzy. Fuzzy is how we think about two names that we’re checking or so a name versus a watch list, or a name versus a passport to verify them we want to see. Okay. It’s not just a binary answer as to yes or no. Does this match? It’s about having some fuzziness in terms of how much the names overlap and how confident we can be that they’re the same or similar. If I see my name written with my first name spelled incorrectly and my middle name missing, I can be reasonably confident that they’re the same, maybe 80% confident that those two names are the same. But what we have is a degree of fuzziness that lets us say yes. I think that above a certain confidence threshold. I’m going to accept that these are the same rather than just the other binary yes or no.

Jeff Phillips
Okay, so now if I put myself and from what you’ve seen, but if I put myself in the shoes of border agents, how do you see artificial intelligence AI-powered tools such as Babel Street? How is it assisting that border agent on the ground?

Declan Trezise
Okay, so this is great. So I have in my head the idea of the perfect border. Okay, excellent. What’s the perfect border look like? The perfect border looks like completely frictionless seamless travel. You park your car, you walk in the airport, you get on your plane, you fly for 5 hours and then you get off. And at no point were you stopped or interrupted or hindered in your travel plans. Now that would be ideal, but how do we get there? And we get there by doing not just one thing. It’s not just introducing an AI agent in the middle. It’s by having kind of a comprehensive approach to it. So usually when you book a flight, you’ll do it in advance. You’ll do it maybe a couple of weeks in advance. And at that point you’re providing information to the airlines, who then provide it to the local law enforcement. They provide it to intelligence agencies. All of those systems currently exist. We call that advanced passenger information. And that enables some prescreening to happen. So potentially you could even be part of a trusted travel program sorry, where you basically give more information up front to allow more seamless travel.

Declan Trezise
And those programs already exist. There are people who can travel from the UK. To the US. They do that frequently. They’ll be on one of these trusted traveler programs. So yeah, giving information in advance helps agencies screen in advance and helps you be fast tracked, potentially if you are traveling between countries. But there will also be times when somebody wants to travel without making plans far in advance. So what would we do to get the information to help us prescreen them? We might look at their behavior, the things that they do on the Internet. We might look at the information that they make publicly available about themselves out there and maybe have that fed into a system that can do some kind of intelligent targeting, screening and fast tracking process. So if we know who somebody is, maybe they drive their car up to the airport, they get off near the terminal. Maybe there is an integrated system between law enforcement, a highways agency that identifies the car, who the person is, and says, okay, are they somebody that we already know is going to travel? Or are they somebody that we think could be about to travel and then kick in a screening process?

Declan Trezise
So we want to make it seamless so that somebody that approaches such an environment can be screened before they get through the door. And then we need some degree of understanding who they are when they’re walking through the airport so that the right person doesn’t get pulled aside. So we want to minimize the number of interventions that a border control guard would have to take. Can we use a combination of biometrics so we willingly give our biometric information up when we cross the border? Traveling into the United States, the first time I went there, I had to give my fingerprints. And traveling in Asia, I’ll give Irish scans or I’ll give face scans that allow me to come through quicker in future. So those capabilities are getting stronger all the time. And obviously AI algorithms sit behind the recognition of the biometrics that happen. So if you can identify who somebody is as they approach, do prescreening identify who they are as they walk through an airport, then you’re 90% of the way there in making this ideal border scenario. And in the background, you have an AIpowered risk engine that assesses the level of risk associated with that person, whether it’s someone on the watch list and you check their name, or that you don’t have a watch list.

Declan Trezise
And the watch list is generated on the fly based on risk factors that exist in the publicly available information that’s out there at the time. So almost a just in time risk check. So you’re not necessarily depending on something. But all of these things you can see require a large amount of coordination between providers of the information, consumers of information, law enforcement, intelligence, and the airlines as well. Or it could be the shipping lines haven’t even mentioned the idea of people traveling cross land borders or seaboarders carrying large amounts of goods.

Jeff Phillips
Either way, I didn’t even think about that as well. As to your point, goods are just shipping and what’s going on there. So with everything you mentioned there, do you see AI as being able to augment current efforts and efforts and plug into places? Or does what you’re talking about require a complete change in the way borders are managed? Is this a complete overhaul?

Declan Trezise
That’s a good question, Jeff. I like that one. So what’s currently happening and then what might be in the future is the way I look at it. Well, actually also in the past, so many countries will have existing legacy systems. They’ll be sitting on huge piles of data that don’t actually exist in an integrated fashion. The worst of it might be paper tickets, reports, passenger manifests, things like this that aren’t accessible or digitized, and countries that have been operating as ports of entry or transit venues, they will have a huge amount of historic information where there might be trapped value. So legacy systems, even that are digital, won’t always be connected, and that can be a problem. Right now, the places that we see AI being implemented is really kind of point solutions. So it’s the algorithm that texts the name, or it’s the algorithm that looks at the iris scan and makes a determination, or the one that works with the fingerprint. So it’s a point solution for AI here and a point solution for AI there. And although people get excited about it, it’s not a holistic thing. It’s not the ultimate large model for borders.

Declan Trezise
So we have large language models now where everybody’s becoming very entertained by those and we’re thinking about the way that it might apply. But ultimately, once you can move away from legacy systems and stop thinking about point solutions, that’s where you can think, okay, really we want a risk model that’s going to be holistic and consider all aspects of data pertaining to border travel, transit, the flow of people and goods across borders. And then once we have that model developed, we can ask it novel questions around the risk of individuals, companies, et cetera, and make determinations. So that’s ideally where maybe it would go. Now there are developing countries that potentially can skip the step around legacy systems. So if you’ve never had an integrated border system, or you’ve never had an attempt at making a border management system with your historic data, then maybe you’re in a better position to start from scratch and not have to rely on that. And I think that’s quite novel. So I work with Tony Smith, the chairman of Ibmasa, So, the Integrated Board of Management Association. He was former Director General of UK Borders, and in my dealings with him, he’s been absolutely invaluable in understanding some of the processes around the UK border when I’ve been working with the Home Office.

Declan Trezise
But he often travels and holds events in different countries who are at different stages in their border management journey. And some of the more agile nations tend to be those that are developing a new border management system from scratch. Ultimately, everybody still raves about the fact that the US has the best border. They don’t tend to do things by halves US combined Customs and Border Protection into a single agency. And from there they took a view that was integrated from the start. I could see other countries looking at going in that direction too.

Jeff Phillips
I like the ideal scenario you talked about. Right.

Declan Trezise
I think we all do, right? That’s what we all want. Yeah.

Aubrey Byron
If you’re a country looking, I guess, to set up a new system or how you can improve what kind of technical and ethical considerations are there for taking this approach.

Declan Trezise
So I’m very much the technologist, Aubrey, so I’m not the one to give you the ethical view very much, but I think that we do all worry that we’re giving away too much information about ourselves all the time and it’s a transaction that we make. So to what degree do you want to give away your personal information for the sake of improving your passage through a border? Well, I think everyone has to make an individual decision about that. But there’s a lot of information out there publicly available and maybe we’re not necessarily aware of how much we’ve already given away. I ask you, feel free to reach out to Babel Street and some of my colleagues will maybe show you what can be found about you out there in the publicly available information. But yeah, we give away a lot of information and we to a degree should have control of how much of our personal information we give away. But yeah, it becomes a transaction as to how much we’re prepared to give away to facilitate our own travel. That’s not to say that if somebody is unwilling to provide personal information that they are in fact a risk.

Declan Trezise
I think I’d never go so far as to look at it like that. There’s a lot of reasons why somebody might not want to give away their personal information. Sorry, what was the other aspect of your question?

Aubrey Byron
I guess for countries looking to set this up? And to your point though, I think that part of that giving away sometimes means maybe you give away more information, but you have TSA pre check now and you get to pass more quickly through. But yeah, if countries are looking to adopt more AI in their how would you recommend they start going about that?

Declan Trezise
So I think it’s look at what the thought leaders in the space are doing and think, okay, how do we adopt that but also build on top of that? My personal prioritization is around improving screening of passengers against watch lists. You can make everything easier for everyone if only the people that are genuine threats are pulled aside. So having an optimal screening system that doesn’t deliver false positive alerts has got to be high on the list. You’re going to be able to make the most of your border guards if they’re not pulling over every 10th person for a single airport, you could have hundreds and hundreds of false positive alerts every single day. So yeah, getting better at that a point solution that can improve your screening, reduce your false positive alerts, and still make sure that you don’t miss those true positive alerts, those absolutely real risks is key. So, yeah, I’d say go that way first. Now, once you have such a system that is good for screening, then your resources are free to be more smart about what you do with everything else that you’ve got. So now that you can make sure that the people on the watch list are stopped and the people that aren’t can move freely.

Declan Trezise
Well, now you think about how do I build the better watch list, how do I make better use of the information available to me? So applying AI to as we described earlier, sort through that information from the OSINT sources to look for new and novel risks associated with it and maybe build a large model from the historical data and the decision making that’s happened. A supervised model plus an unsupervised model machine learnt based off that data, combined with a decision making engine that’s going to give your kind of future border guard a better way of doing things. Because we think of border guards as people wearing uniforms, standing in an airport or standing at a port or at a border crossing. But I think that in the future that’s more going to be a role where somebody sits at a desk and understands the general level of risk at any one time and can drill down into the individual risk associated with people as they move. So yeah, assisted by a machine that can bring the riskiest things to the top of their list.

Aubrey Byron
Yeah. Can you give any examples actually of AI being used to sort of tip off border agents about a potential risk?

Declan Trezise
I can give you an interesting example. So I’ve got a couple. So I think probably you’re all familiar with the limitations around taking liquids onto flights, right? Yeah. And what we were aware, there was a Netflix documentary, like film documentary that told the whole story of the liquid bomb plot. Ultimately the liquid bomb plot was a plot that was foiled by intelligence agencies. And when you watch the film you get to the point where they’ve already identified the people that were causing out to cause harm. But a lot of the hard work happened before that. So this was back in 2006, I think, something like that. There was already AI algorithms running against text postings on forums in the internet, in the deep and dark web to identify risks against individuals who could potentially become travelers. So having AI engines that could read different languages, AI engines that could understand the risky terms that might be mentioned and associate those with people and times and flights were needed to foil that particular plot. So it was natural language processing was applied on huge amounts of collected data to pull out the names of people and then the plot that they were trying to create.

Declan Trezise
And it wasn’t simplistic things, it wasn’t words like bomb that were identified. Those are very easy with the keyword set. It’s having a semantic understanding of individual languages and being able to determine talk of chemicals in a different language. Actually those chemicals when combined together could produce a bomb. So it’s going, that extra layer which the natural language processing and text analytics can allow you to do. So that was one of the kind of an early example of AI being used to trawl through large amounts of data to mitigate a risk. The second example doesn’t have such a positive outcome. So with the liquid bomb plot, I think there are multiple planes and of the order of about 2000 people that were at risk who were saved by that plot being foiled. But an example where there was a failure in a border system that could have been solved by AI is the Boston Marathon bombing. So I think this was 2013, and unfortunately, there were two individuals, the Sarnaf brothers, who were actually already on the FBI terrorist suspect database, and they still managed to enter the United States at JFK Airport, travel to Boston and plant a bomb.

Declan Trezise
So there was a failure there in the name matching system that they were using at the time. And after a Senate inquiry, it was understood that it was failure in the name screening capability that they had when it came to taking names written in Cyrillic. So Russian names written in Cyrillic script and comparing them against an English watch list. So that system, after the inquiry and the identification of the hole in the border, because ultimately it was a hole in the border caused by a lack of capability that’s when they implemented, after some testing to prove that it could be filled, they implemented an AI smart fuzzy name matching system. In fact, it was our AI based smart fuzzy name matching system to secure CBP after, you know, what we don’t want is to have more things like that happen that trigger a need for an AI algorithm. We want to kind of preempt that if we can. But now you can know that it won’t happen again. Right. The United States has filled that hole with the best technology available today.

Aubrey Byron
That just made me think, are you using linguists in your capabilities at all if you’re trying to do these translations and match language? I’m just curious.

Declan Trezise
Yeah. So in order to build a language model that’s appropriate for name screening or any of the other capabilities that we have around analyzing text, you need to have linguists that can train the machine, and they need to take their knowledge and put it into a way that the machine can understand. Build an AI model off the back of that. So I remember at one time within the Rosette team, I think we counted there was 37 different languages spoken across about 80 people, which was quite impressive. But also, a lot of the language capability that we use, the linguists that we might use, well, it’s about having a data team that can reach out to agencies and say, okay, I need skilled linguists that can help me build this model, help me with this annotation task, and feed. That into the machine. So, yeah, ultimately it comes from humans. All of the learning that we’ll implement into something in the technology.

Aubrey Byron
That’s fascinating. Yeah, I think just like with that sort of technology, you automatically think, okay, tech people are building this, but just love to hear of my fellow liberal arts background people finding its place here, too.

Declan Trezise
No, it’s a funny intersection that technology and language, because, yeah, often it is different groups, so they’re quite sometimes quite hard to find somebody that does both. But actually, you can split the tasks such that the linguist can do the linguistic tasks and the computer scientists can do the computer tasks.

Aubrey Byron
Awesome.

Jeff Phillips
Well, Declan, sorry, this has been super interesting to know, crossing AI and OSINT with border security. I want to thank you for your time. But before we go, is there anything else you want to add about Babel Street and artificial intelligence and how you can assist borders, basically to run more smoothly and travelers to be less?

Declan Trezise
You know, I really appreciated you giving me the time to speak today on the podcast. Jeff, Aubrey, really appreciate your time. Now, I think that kind of some of my final thoughts around it is that we’re not done here. It’s not that we have the perfect system yet. We don’t have frictionless borders yet. And similar technology applies in other aspects of defense security and law enforcement, too. They’re not done. And we’re continually evolving. I think that in the old days, we worried about having too much data. Big data used to be this buzzword that we were scared of, but now we’ve kind of tamed big data. I think going forward, we’re going to have to get smarter about what we’re asking of the systems and what we want them to do. It’s a bit like in The Hitchhiker’s Guide to the Galaxy. You find out the answer to the ultimate question of life, the universe, and everything, but what’s the question that you need to be asking? So I think, yeah, there’s a lot of thinking that still needs to be done so we can envisage the best possible outcome, the seamless frictionless border. But to get there, we can do better than we’re doing now with the current point solutions.

Declan Trezise
But, yeah, watch this space, right? So we’re always trying to innovate, we’re always trying to bring capabilities that are novel and unique from research, make them into capabilities that can be consumed as enterprise software. And you’re not the only company in this space. There’s a thriving community we work in partnership with across AI, across language and complementary technologies. So watch this space and look out for upcoming announcements and look out for products and capabilities that appear. And yet we will continue to support this particular area. We’ve got some very loyal and satisfied customers in the border domain, and it’s something that’s close to our hearts. So, yeah, that’s kind of what I’d close with.

Jeff Phillips
Well, thank you again, Declan, for joining us today. To our listeners, if you liked what you heard, you can view transcripts and other episode info on our website, authentic8.com/needlestack. That’s authentic with the number eight.com slash needlestack. And be sure to let us know your thoughts on Twitter @needlestackpod and to like and subscribe wherever you’re listening today. We’ll be back next week with more on how analysts can use emerging technologies.

Declan Trezise
We’ll see you then.