The US Department of Homeland Security operated a program that sought to automate the designation of social media users as \u201cpro-terrorist\u201d based on their use of certain keywords and interaction with other targeted users, according to documents obtained by the Brennan Center for Justice and published on Tuesday.<\/p>\n\n\n\n
Project Night Fury, a partnership between the DHS and the University of Alabama at Birmingham, sought to assign potentially \u201cpro-terrorist\u201d accounts a \u201crisk score\u201d that would then affect other accounts they interacted with. The university had agreed to develop automated methods to determine whether an account linked to one already under surveillance was itself \u201cpro-terrorist,\u201d using criteria like \u201ckeyword set comparisons.\u201d The algorithm, not a human, would determine whether a user could be labeled \u201cpro-terrorist\u201d for simply clicking \u201clike\u201d on a Facebook post or retweeting an account that had already received that label.<\/p>\n\n\n\n
Despite the known limits of artificial intelligence when this system was being formulated in 2018, the entire process would have been automated, according to the documents. The university was assigned to build models to \u201cidentify key influencers of pro-terrorist thought\u201d and create an automated system to uncover bots \u201cprogrammatically generated to exert influence\u201d to spread both \u201cterrorist propaganda\u201d and \u201cforeign influence campaigns.\u201d They would then compile a list of suspect accounts and turn them over to the DHS, along with their names, emails, phone numbers, pictures, and posts.<\/p>\n\n\n\n
Missing from the contracts were definitions for important concepts like what \u201cpro-terrorist\u201d actually meant \u2013 or what a \u201crisk score\u201d would measure, whether it was the risk of becoming a terrorist or merely sympathizing with one \u2013 or an explanation of how an automated process could make such distinctions absent clear-cut definitions.<\/p>\n\n\n\n
It’s also not clear how the program would have distinguished between terrorism-adjacent content and material related to drug trafficking and \u201cdisinformation,\u201d which it also sought to track. As of the date of the contracts seen by the Brennan Center, the DHS hadn\u2019t figured out how to quantify terrorist sympathies and was seeking assistance from university researchers to \u201cidentify relevant attributes.\u201d<\/p>\n\n\n\n
The DHS Inspector General was tipped off to potential privacy violations on Project Night Fury in 2018, and until the Brennan Center\u2019s FOIA request was fulfilled, all that was known about the project came from the OIG\u2019s 2019 report, which found the agents involved with the project did not comply with department policies governing privacy and protection of sensitive information.<\/p>\n\n\n\n
While the DHS supposedly stopped work on Night Fury in 2019, it\u2019s not known how far along the project was, or whether its assets were transferred elsewhere within the law enforcement apparatus. Customs and Border Protection was recently found to be using a similar AI-powered tool called Babel X to snoop through the social media accounts of travelers at the US border.<\/p>\n\n\n\n