The Adversarial ML Threat Matrix provides guidelines that help detect and prevent attacks on machine learning systems.
This article is part ofDemystifying AI, a series of posts that (try to) disambiguate the jargon and myths surrounding AI.
With machine learning becoming increasingly popular, one thing that has been worrying experts is the security threats the technology will entail. We are still exploring the possibilities: The breakdown of autonomous driving systems? Inconspicuous theft of sensitive data from deep neural networks? Failure of deep learningbased biometric authentication? Subtle bypass of content moderation algorithms?
Meanwhile, machine learning algorithms have already found their way into critical fields such as finance, health care, and transportation, where security failures can have severe repercussion.
Parallel to the increased adoption of machine learning algorithms in different domains, there has been growing interest in adversarial machine learning, the field of research that explores ways learning algorithms can be compromised.
And now, we finally have a framework to detect and respond to adversarial attacks against machine learning systems. Called the Adversarial ML Threat Matrix, the framework is the result of a joint effort between AI researchers at 13 organizations, including Microsoft, IBM, Nvidia, and MITRE.
While still in early stages, the ML Threat Matrix provides a consolidated view of how malicious actors can take advantage of weaknesses in machine learning algorithms to target organizations that use them. And its key message is that the threat of adversarial machine learning is real and organizations should act now to secure their AI systems.
The Adversarial ML Threat Matrix is presented in the style of ATT&CK, a tried-and-tested framework developed by MITRE to deal with cyber-threats in enterprise networks. ATT&CK provides a table that summarizes different adversarial tactics and the types of techniques that threat actors perform in each area.
Since its inception, ATT&CK has become a popular guide for cybersecurity experts and threat analysts to find weaknesses and speculate on possible attacks. The ATT&CK format of the Adversarial ML Threat Matrix makes it easier for security analysts to understand the threats of machine learning systems. It is also an accessible document for machine learning engineers who might not be deeply acquainted with cybersecurity operations.
Many industries are undergoing digital transformation and will likely adopt machine learning technology as part of service/product offerings, including making high-stakes decisions, Pin-Yu Chen, AI researcher at IBM, told TechTalks in written comments. The notion of system has evolved and become more complicated with the adoption of machine learning and deep learning.
For instance, Chen says, an automated financial loan application recommendation can change from a transparent rule-based system to a black-box neural network-oriented system, which could have considerable implications on how the system can be attacked and secured.
The adversarial threat matrix analysis (i.e., the study) bridges the gap by offering a holistic view of security in emerging ML-based systems, as well as illustrating their causes from traditional means and new risks induce by ML, Chen says.
The Adversarial ML Threat Matrix combines known and documented tactics and techniques used in attacking digital infrastructure with methods that are unique to machine learning systems. Like the original ATT&CK table, each column represents one tactic (or area of activity) such as reconnaissance or model evasion, and each cell represents a specific technique.
For instance, to attack a machine learning system, a malicious actor must first gather information about the underlying model (reconnaissance column). This can be done through the gathering of open-source information (arXiv papers, GitHub repositories, press releases, etc.) or through experimentation with the application programming interface that exposes the model.
The security landscape has evolved and continues to develop to address each of these threats. We have anti-malware software, web application firewalls, intrusion detection and prevention systems, DDoS protection solutions, and many more tools to fend off these threats.
For instance, security tools can scan binary executables for the digital fingerprints of malicious payloads, and static analysis can find vulnerabilities in software code. Many platforms such as GitHub and Google App Store already have integrated many of these tools and do a good job at finding security holes in the software they house.
But in adversarial attacks, malicious behavior and vulnerabilities are deeply embedded in the thousands and millions of parameters of deep neural networks, which is both hard to find and beyond the capabilities of current security tools.
Traditional software security usually does not involve the machine learning component because itsa new piece in the growing system, Chen says, adding thatadopting machine learning into the security landscape gives new insights and risk assessment.
The Adversarial ML Threat Matrix comes with a set of case studies of attacks that involve traditional security vulnerabilities, adversarial machine learning, and combinations of both. Whats important is that contrary to the popular belief that adversarial attacks are limited to lab environments, the case studies show that production machine learning system can and have been compromised with adversarial attacks.
For instance, in one case study, the security team at Microsoft Azure used open-source data to gather information about a target machine learning model. They then used a valid account in the server to obtain the machine learning model and its training data. They used this information to find adversarial vulnerabilities in the model and develop attacks against the API that exposed its functionality to the public.
Other case studies show how attackers can compromise various aspect of the machine learning pipeline and the software stack to conduct data poisoning attacks, bypass spam detectors, or force AI systems to reveal confidential information.
The matrix and these case studies can guide analysts in finding weak spots in their software and can guide security tool vendors in creating new tools to protect machine learning systems.
Inspecting a single dimension (machine learning vs traditional software security) only provides an incomplete security analysis of the system as a whole, Chen says. Like the old saying goes: security is only asstrong as its weakest link.
Unfortunately, developers and adopters of machine learning algorithms are not taking the necessary measures to make their models robust against adversarial attacks.
The current development pipeline is merely ensuring a model trained on a training set can generalize well to a test set, while neglecting the fact that the model isoften overconfident about the unseen (out-of-distribution) data or maliciously embbed Trojan patteninthe training set, which offers unintended avenues to evasion attacks and backdoor attacks that an adversary can leverage to control or misguide the deployed model, Chen says. In my view, similar to car model development and manufacturing, a comprehensive in-house collision test for different adversarial treats on an AI model should be the new norm to practice to better understand and mitigate potential security risks.
In his work at IBM Research, Chen has helped develop various methods to detect and patch adversarial vulnerabilities in machine learning models. With the advent Adversarial ML Threat Matrix, the efforts of Chen and other AI and security researchers will put developers in a better position to create secure and robust machine learning systems.
My hope is that with this study, the model developers and machine learning researchers can pay more attention to the security (robustness) aspect of the modeland looking beyond a single performance metric such as accuracy, Chen says.
Read the original post:
The security threat of adversarial machine learning is real - TechTalks
- The 12 Coolest Machine-Learning Startups Of 2020 - CRN - November 19th, 2020
- Utilizing machine learning to uncover the right content at KMWorld Connect 2020 - KMWorld Magazine - November 19th, 2020
- The way we train AI is fundamentally flawed - MIT Technology Review - November 19th, 2020
- DIY Camera Uses Machine Learning to Audibly Tell You What it Sees - PetaPixel - November 19th, 2020
- Machine Learning Predicts How Cancer Patients Will Respond to Therapy - HealthITAnalytics.com - November 19th, 2020
- This New Machine Learning Tool Might Stop Misinformation - Digital Information World - November 19th, 2020
- Fujitsu, AIST and RIKEN Achieve Unparalleled Speed on MLPerf HPC Machine Learning Processing Benchmark - HPCwire - November 19th, 2020
- SVG Tech Insight: Increasing Value of Sports Content Machine Learning for Up-Conversion HD to UHD - Sports Video Group - November 19th, 2020
- SiMa.ai Adopts Arm Technology to Deliver a Purpose-built Heterogeneous Machine Learning Compute Platform for the Embedded Edge - Design and Reuse - November 19th, 2020
- Machine learning removes bias from algorithms and the hiring process - PRNewswire - November 6th, 2020
- Using machine learning to track the pandemic's impact on mental health - MIT News - November 6th, 2020
- AI Recognizes COVID-19 in the Sound of a Cough Machine Learning Times - The Predictive Analytics Times - November 6th, 2020
- The consistency of machine learning and statistical models in predicting clinical risks of individual patients - The BMJ - The BMJ - November 6th, 2020
- PathAI and Gilead Report Data from Machine Learning Model Predictions of Liver Disease Progression and Treatment Response at AASLD's The Liver Meeting... - November 6th, 2020
- Google Introduces New Analytics with Machine Learning and Predictive Models - IBL News - November 6th, 2020
- Free Webinar | Machine Learning and Data Analytics in the Pandemic Era - MIT Sloan - November 6th, 2020
- Global Predictive Analytics Market (2020 to 2025) - Advent of Machine Learning and Artificial Intelligence is Driving Growth - PRNewswire - November 6th, 2020
- Machine learning and predictive analytics work better together - TechTarget - October 31st, 2020
- Microsoft Introduces Lobe: A Free Machine Learning Application That Allows You To Create AI Models Without Coding - MarkTechPost - October 31st, 2020
- Amwell CMO: Google partnership will focus on AI, machine learning to expand into new markets - FierceHealthcare - October 31st, 2020
- Microsoft/MITRE group declares war on machine learning vulnerabilities with Adversarial ML Threat Matrix - Diginomica - October 31st, 2020
- 93% of security operations centers employing AI and machine learning tools to detect advanced threats - Security Magazine - October 31st, 2020
- Machine Learning in Insurance Market(COVID-19 Analysis): Indoor Applications Projected to be the Most Attractive Segment during 2020-2027 - Global... - October 31st, 2020
- Leveraging Machine Learning and IDP to Scale Your Automation Program - AiiA - October 31st, 2020
- 5 machine learning skills you need in the cloud - TechTarget - October 31st, 2020
- Machine learning approach could detect drivers of atrial fibrillation - Cardiac Rhythm News - October 31st, 2020
- Vanderbilt trans-institutional team shows how next-gen wearable sensor algorithms powered by machine learning could be key to preventing injuries that... - October 31st, 2020
- Machine Learning & Big Data Analytics Education Market Size And Forecast (2020-2026)| With Post Impact Of Covid-19 By Top Leading Players-... - October 31st, 2020
- Bridging the Skills Gap for AI and Machine Learning - Integration Developers - October 23rd, 2020
- Nudges and machine learning triples advanced care conversations - Penn Today - October 23rd, 2020
- Machine Learning and AI Can Now Create Plastics That Easily Degrade - Science Times - October 23rd, 2020
- insitro Strengthens Machine Learning-Based Drug Discovery Capabilities with Acquisition of Haystack Sciences - Business Wire - October 23rd, 2020
- Revolutionizing IoT with Machine Learning at the Edge | Perceive's Steve Teig - IoT For All - October 23rd, 2020
- Mastercard Says its AI and Machine Learning Solutions Aim to Stop Fraudulent Activites which have Increased Significantly due to COVID - Crowdfund... - October 23rd, 2020
- Abstract Perspective: Long-term PM2.5 Exposure and the Clinical Application of Machine Learning for Predicting Incident Atrial Fibrillation - DocWire... - October 23rd, 2020
- Machine-Learning Inference Chip Travels to the Edge - Electronic Design - October 23rd, 2020
- Machine Learning Data Catalog Software Market share forecast to witness considerable growth from 2020 to 2025 | By Top Leading Vendors IBM, Alation,... - October 23rd, 2020
- AI and machine learning: a gift, and a curse, for cybersecurity - Healthcare IT News - October 21st, 2020
- Teaming Up with Arm, NXP Ups Its Place in the Machine Learning Industry - News - All About Circuits - October 21st, 2020
- Machine Learning Capabilities Come to the Majority of Open Source Databases with MindsDB AI-Tables - PR Web - October 21st, 2020
- Soleadify secures seed funding for database that uses machine learning to track 40M businesses - TechCrunch - October 21st, 2020
- NXP Announces Expansion of its Scalable Machine Learning Portfolio and Capabilities - GlobeNewswire - October 21st, 2020
- NXP Invests in Au-Zone to Enhance Machine Learning Capabilities - Mobile ID World - October 21st, 2020
- Factories of The Future Are Using Machine Learning Analytics to Optimize Assets - Embedded Computing Design - October 21st, 2020
- Lantronix Brings Advanced AI and Machine Learning to Smart Cameras With New Open-Q 610 SOM Based on the Powerful Qualcomm QCS610 System on Chip (SOC)... - October 21st, 2020
- EMA Webinar to Uncover How Machine Learning and Predictive Analytics Can Improve Workload Automation Outcomes - PR Web - October 21st, 2020
- How To Choose The Best Machine Learning Algorithm For A Particular Problem? - Analytics India Magazine - October 21st, 2020
- AI and Machine Learning Technologies Expected to Play a Key Role in Expanding Multi Billion Dollar Digital Banking Sector: Report - Crowdfund Insider - October 21st, 2020
- EXCLUSIVE: Amazon AI executive explains three things every business needs to address before using machine lear - Business Insider India - October 21st, 2020
- Photoshops AI neural filters can tweak age and expression with a few clicks - The Verge - October 21st, 2020
- Futurism Reinforces Its Next-Gen Business Commerce Platform With Advanced Machine Learning and Artificial Intelligence Capabilities - Yahoo Finance - October 15th, 2020
- Purebase Enhances Its Board of Advisors with An Expert on Machine Learning and Cheminformatics - GlobeNewswire - October 15th, 2020
- How to Beat Analysts and the Stock Market with Machine Learning - Knowledge@Wharton - October 15th, 2020
- Synopsys and SiMa.ai Collaborate to Bring Machine Learning Inference at Scale to the Embedded Edge - AiThority - October 15th, 2020
- Robotic Interviews, Machine Learning And the Future Of Workforce Recruitment - Entrepreneur - October 15th, 2020
- Top 8 Books on Machine Learning In Cybersecurity One Must Read - Analytics India Magazine - October 15th, 2020
- AI and Machine Learning Can Help Fintechs if We Focus on Practical Implementation and Move Away from Overhyped Narratives, Researcher Says - Crowdfund... - October 15th, 2020
- AI and Machine Learning Can Help FIs Avoid Riskbut They Have Risk of Their Own - PR Web - October 15th, 2020
- Machine learning for rowdy roadies: Cops and tech to rein in traffic offenders - Bangalore Mirror - October 15th, 2020
- Automated ATOs and cybersecurity - FCW.com - October 15th, 2020
- Experian partners with Standard Chartered to drive Financial Inclusion with Machine Learning, powering the next generation of Decisioning - Yahoo... - October 15th, 2020
- Machine Learning Answers: Facebook Stock Is Down 20% In A Month, What Are The Chances It'll Rebound? - Trefis - September 22nd, 2020
- Machine Learning in Education Market Incredible Possibilities, Growth Analysis and Forecast To 2025 - The Daily Chronicle - September 22nd, 2020
- Proximity matters: Using machine learning and geospatial analytics to reduce COVID-19 exposure risk - Healthcare IT News - September 22nd, 2020
- Global Machine Learning Market Tends To Show Steady Growth Post Pandemic With Regional Overview and Top Key Players - Verdant News - September 22nd, 2020
- PREDICTING THE OPTIMUM PATH - Port Strategy - September 22nd, 2020
- AI/ML Remains The Most In-Demand Tech Skill Post COVID - Analytics India Magazine - September 22nd, 2020
- Panalgo Brings the Power of Machine-Learning to the Healthcare Industry Via Its IHD Software - AiThority - September 15th, 2020
- Microchip Partners with Machine-Learning (ML) Software Leaders to Simplify AI-at-the-Edge Design Using its 32-Bit Microcontrollers (MCUs) - EE Journal - September 15th, 2020
- What is 'custom machine learning' and why is it important for programmatic optimisation? - The Drum - September 15th, 2020
- PODCAST: NVIDIA's Director of Data Science Talks Machine Learning for Airlines and Aerospace - Aviation Today - September 15th, 2020
- The Use of Machine Learning to Forecast Progression to Advanced AMD - DocWire News - September 15th, 2020
- How Can Machine Learning Help the Teaching Profession? - FE News - September 15th, 2020
- Global Machine Learning in Automobile Market: Development History, Current Analysis and Estimated Forecast to 2024 - The Market Correspondent - September 15th, 2020
- Using machine learning to organize the chemical diversity - Tech Explorist - September 15th, 2020
- Dashboard AI Announces Its Technology Vision for the Foodservice and Hospitality Industry - PRNewswire - September 15th, 2020
- Alfa Releases Second Paper on AI, Using Machine Learning in the Wild - Monitor Daily - September 10th, 2020
- Combatting COVID-19 misinformation with machine learning (VB Live) - VentureBeat - September 10th, 2020
- This artist used machine learning to create realistic portraits of Roman emperors - The World - September 10th, 2020
- Domino Data Lab Named a Leader in Notebook-Based Predictive Analytics and Machine Learning Evaluation by Global Research Firm - Business Wire - September 10th, 2020