{"id":2276,"date":"2023-12-04T10:52:34","date_gmt":"2023-12-04T14:52:34","guid":{"rendered":"https:\/\/ece.ncsu.edu\/?p=263101"},"modified":"2023-12-04T10:52:34","modified_gmt":"2023-12-04T14:52:34","slug":"ai-networks-are-more-vulnerable-to-malicious-attacks-than-previously-thought","status":"publish","type":"post","link":"https:\/\/my.ece.ncsu.edu\/communications\/2023\/ai-networks-are-more-vulnerable-to-malicious-attacks-than-previously-thought\/","title":{"rendered":"AI Networks Are More Vulnerable to Malicious Attacks Than Previously Thought"},"content":{"rendered":"
\"\"<\/div>\n

Artificial intelligence tools hold promise for applications ranging from autonomous vehicles to the interpretation of medical images. However, a new study finds these AI tools are more vulnerable than previously thought to targeted attacks that effectively force AI systems to make bad decisions.<\/p>\n

At issue are so-called \u201cadversarial attacks,\u201d in which someone manipulates the data being fed into an AI system in order to confuse it. For example, someone might know that putting a specific type of sticker at a specific spot on a stop sign could effectively make the stop sign invisible to an AI system. Or a hacker could install code on an X-ray machine that alters the image data in a way that causes an AI system to make inaccurate diagnoses.<\/p>\n

\u201cFor the most part, you can make all sorts of changes to a stop sign, and an AI that has been trained to identify stop signs will still know it\u2019s a stop sign,\u201d says Tianfu Wu, co-author of a paper on the new work and an associate professor of electrical and computer engineering at North Carolina State University. \u201cHowever, if the AI has a vulnerability, and an attacker knows the vulnerability, the attacker could take advantage of the vulnerability and cause an accident.\u201d<\/p>\n

The new study from Wu and his collaborators focused on determining how common these sorts of adversarial vulnerabilities are in AI deep neural networks. They found that the vulnerabilities are much more common than previously thought.<\/p>\n

\u201cWhat\u2019s more, we found that attackers can take advantage of these vulnerabilities to force the AI to interpret the data to be whatever they want,\u201d Wu says. \u201cUsing the stop sign example, you could make the AI system think the stop sign is a mailbox, or a speed limit sign, or a green light, and so on, simply by using slightly different stickers \u2013 or whatever the vulnerability is.<\/p>\n

\u201cThis is incredibly important, because if an AI system is not robust against these sorts of attacks, you don\u2019t want to put the system into practical use \u2013 particularly for applications that can affect human lives.\u201d<\/p>\n

To test the vulnerability of deep neural networks to these adversarial attacks, the researchers developed a piece of software called QuadAttacK<\/em>. The software can be used to test any deep neural network for adversarial vulnerabilities.<\/p>\n

\u201cBasically, if you have a trained AI system, and you test it with clean data, the AI system will behave as predicted. QuadAttacK<\/em>\u00a0watches these operations and learns how the AI is making decisions related to the data. This allows QuadAttacK<\/em>\u00a0to determine how the data could be manipulated to fool the AI. QuadAttacK<\/em>\u00a0then begins sending manipulated data to the AI system to see how the AI responds. If QuadAttacK<\/em>\u00a0has identified a vulnerability it can quickly make the AI see whatever QuadAttacK<\/em>\u00a0wants it to see.\u201d<\/p>\n

In proof-of-concept testing, the researchers used QuadAttacK<\/em>\u00a0to test four deep neural networks: two convolutional neural networks (ResNet-50 and DenseNet-121) and two vision transformers (ViT-B and DEiT-S). These four networks were chosen because they are in widespread use in AI systems around the world.<\/p>\n

\u201cWe were surprised to find that all four of these networks were very vulnerable to adversarial attacks,\u201d Wu says. \u201cWe were particularly surprised at the extent to which we could fine-tune the attacks to make the networks see what we wanted them to see.\u201d<\/p>\n

The research team has made QuadAttacK<\/em>\u00a0publicly available, so that the research community can use it themselves to test neural networks for vulnerabilities. The program can be found here:\u00a0https:\/\/thomaspaniagua.github.io\/quadattack_web\/<\/a>.<\/p>\n

\u201cNow that we can better identify these vulnerabilities, the next step is to find ways to minimize those vulnerabilities,\u201d Wu says. \u201cWe already have some potential solutions \u2013 but the results of that work are still forthcoming.\u201d<\/p>\n

The paper, \u201cQuadAttacK<\/em>: A Quadratic Programming Approach to Learning Ordered Top-K<\/em>\u00a0Adversarial Attacks,\u201d will be presented Dec. 16 at the Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023<\/a>), which is being held in New Orleans, La. First author of the paper is Thomas Paniagua, a Ph.D. student at NC State. The paper was co-authored by Ryan Grainger, a Ph.D. student at NC State.<\/p>\n

The work was done with support from the U.S. Army Research Office, under grants W911NF1810295 and W911NF2210010; and from the National Science Foundation, under grants 1909644, 2024688 and 2013451.<\/p>\n","protected":false},"excerpt":{"rendered":"

\"\"<\/div>\n

A study finds AI tools are more vulnerable than previously thought to targeted attacks that effectively force AI systems to make bad decisions.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"ncst_dynamicHeaderBlockName":"","ncst_dynamicHeaderData":"","ncst_content_audit_freq":"","ncst_content_audit_date":"","ncst_content_audit_display":false,"ncst_backToTopFlag":"","footnotes":""},"categories":[180],"tags":[],"class_list":["post-2276","post","type-post","status-publish","format-standard","hentry","category-research"],"displayCategory":null,"acf":{"ncst_posts_meta_modified_date":null},"_links":{"self":[{"href":"https:\/\/my.ece.ncsu.edu\/communications\/wp-json\/wp\/v2\/posts\/2276","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/my.ece.ncsu.edu\/communications\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/my.ece.ncsu.edu\/communications\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/my.ece.ncsu.edu\/communications\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/my.ece.ncsu.edu\/communications\/wp-json\/wp\/v2\/comments?post=2276"}],"version-history":[{"count":1,"href":"https:\/\/my.ece.ncsu.edu\/communications\/wp-json\/wp\/v2\/posts\/2276\/revisions"}],"predecessor-version":[{"id":2277,"href":"https:\/\/my.ece.ncsu.edu\/communications\/wp-json\/wp\/v2\/posts\/2276\/revisions\/2277"}],"wp:attachment":[{"href":"https:\/\/my.ece.ncsu.edu\/communications\/wp-json\/wp\/v2\/media?parent=2276"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/my.ece.ncsu.edu\/communications\/wp-json\/wp\/v2\/categories?post=2276"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/my.ece.ncsu.edu\/communications\/wp-json\/wp\/v2\/tags?post=2276"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}