Everything about the subject of this article is mind-numbingly awful.
Ars Technica: The NSA’s Skynet program may be killing thousands of innocent people
The NSA have apparently built an artificial intelligence which sifts through mass-surveiled metadata of Pakistani civilians to compile a kill list, which is then executed on by America's fleet of death squads and flying robots.
[Skynet] engages in mass surveillance of Pakistan's mobile phone network, and then uses a machine learning algorithm on the cellular network metadata of 55 million people to try and rate each person's likelihood of being a terrorist.
The program collects metadata and stores it on NSA cloud servers, extracts relevant information, and then applies machine learning to identify leads for a targeted campaign …[which] likely involves another branch of the US government—the CIA or military—that executes their "Find-Fix-Finish" strategy
Not only that, civilians who try to use common-sense to avoid this survilance will be flagged, and thus presumably more likely to be selected as a false-positive.
Turning off a mobile phone gets flagged as an attempt to evade mass surveillance. Users who swap SIM cards … also get flagged … Even handset swapping gets detected and flagged
Though this sounds cartoonishly villainous anyway, it seems like they didn't even do it very well, so their AI will be potentially subject to vast false-positive rates. Which is a problem when you're compiling a kill list.
"First, there are very few 'known terrorists' to use to train and test the model," [data scientist Patrick] Ball said. "If they are using the same records to train the model as they are using to test the model, their assessment of the fit is completely bullshit. The usual practice is to hold some of the data out of the training process so that the test includes records the model has never seen before. Without this step, their classification fit assessment is ridiculously optimistic."
(This is a completely rudimentary error in machine learning practise, incidentally.)
"On whether the use of [Skynet] is a war crime, I defer to lawyers," Ball said. "It's bad science, that's for damn sure"
It's getting pretty hard to reconcile America's conduct with any definition of "good guys".