Artificial intelligence (AI) has become the foundation of everyday technologies — including smartphones, cars, banking apps, home devices and more. In the cybersecurity world, AI is powering new technologies to enhance the detection of malicious behavior and sophisticated threats. Complex models can identify attack trends much faster than previous systems.
But what if attackers could exploit the very power of AI to launch new attacks? Is it possible to subvert the AI we depend on, including cybersecurity products, to evade detection?
Research shows us that it’s not just possible, but plausible. This is what we call adversarial AI or adversarial machine learning, and it should be a growing concern for businesses and consumers as algorithms become more advanced.
Research Shows The Possibilities Of Adversarial AI
As noted in a March 2019 article (registration required) in MIT Technology Review, Dawn Song, professor and cybersecurity researcher at the University of California, Berkley, stated that adversarial machine learning could be used to attack just about any system built on the technology.
Song’s research group explored several examples of how adversarial learning can be used. For instance, in one case they demonstrated how attackers could exploit machine learning algorithms designed to automate email responses to instead “spit out sensitive data such as credit card numbers.”
Song demonstrated how computer vision systems in vehicles could be tricked by placing stickers on road signs, corrupting the dataset and tricking the algorithms powering autonomous cars into thinking stop signs were actually speed limits. The problem with this is self-evident.
Adversarial AI Attacks In Action
Researchers at Princeton recently explored how adversarial tactics applied to artificial intelligence (AI) could leave systems vulnerable.
In the report, the researchers noted, “Just as software is prone to being hacked and infected by computer viruses, or its users targeted by scammers through phishing and other security-breaching ploys, AI-powered applications have their own vulnerabilities. Yet the deployment of adequate safeguards has lagged.”
As noted in the report, there are three key types of adversarial AI attacks:
• Data poisoning at the time of model training: Attackers use AI to mark or launch their attacks.
• Adversarial inputs at runtime: Attackers alter the training data used for security AI.
• Privacy attacks: Adversaries try to gain access to private information.
Within these categories, adversarial attacks can take several forms, including false flag attacks. By manipulating data, attackers can launch cyberattacks and make them appear to come from a specific country.
Would the U.S. reaction to election hacking be different if it appeared to come from a nation like North Korea, as opposed to a global power like Russia? If the attacks on the Ukrainian power grid that resulted in power loss for more than 250,000 citizens were to happen to Israel and appeared to come from Iran, would it precipitate a physical response?
In a time of rising global tensions, these scenarios are no longer simply part of a war game. They’ve moved into the realm of reality.
Another example of adversarial attacks is the concept of deepfakes. As reported by the Financial Times (paywall), AI-powered deepfakes are already being used in everyday attacks such as fraud, as well as to manipulate videos.
Other attacks include attackers manipulating AI to carry out more authentic and devastating socially engineered attacks.
For instance, a recently reported deepfake was used to trick an executive at a U.K. energy company into wiring money to a supplier. The victim, in this case, received a phone call that he thought was his boss instructing him to initiate the transfer. The call and email that followed replicated the mannerisms, accent and diction of his boss.
As we head deep into the 2020 U.S. elections, security continues to be a major issue. In my opinion, it’s possible that adversarial AI could play a role in influencing the outcome of the elections or enable fraud in other aspects of business and daily life. For instance, emails stolen from candidates could be used to craft believable messages that are contrary to the true positions of a candidate. Think about the havoc that could cause when launched and amplified through social media.
The risks of adversarial AI should also force us to broaden the concept of an insider threat. Insiders in many cases have the ability to mess around with the training and tweak the algorithms.
In fact, such insiders might be targeted or subverted precisely for this reason. The level of trust an organization has in AI might mean these alterations are extremely difficult to detect.
Reality Check: Attacks Can Happen Anywhere
How easy is it for bad actors to launch attacks by manipulating training data and AI systems? It depends on the sophistication of the models and other factors.
However, there are plenty of people who understand the inner workings of the technology and how models are built, and they know how to manipulate AI for various purposes. If they’re properly motivated or coerced, they could become participants of adversarial AI attacks.
This is a worldwide risk factor that needs to be addressed before it becomes a major problem. Security experts and product developers need to factor in the potential for abuse when building AI models and harden those models to the extent possible.
Multilayer checks and balances that don’t rely on solitary models for decisions are important to manage this risk. Similarly, using an ensemble of machine learning approaches raises the bar for the adversary to be successful. When building models, developers need to assume the worst: that someone will try to subvert them to cause damage. Then they can at least make it more difficult to change the models in an adverse way, and they will have already mitigated the worst-case scenarios as best possible.
By taking steps today to become more aware of how adversarial AI works, everyone can be in a better position to eliminate or reduce the risks.
